Information Technology Reference
In-Depth Information
2.2
Bayesian Model Averaging
In a model space
models, the posterior probability of model M c given the
data D can be computed via Bayes' Theorem:
M
with
|M|
P ( M r ) P ( D
|
M c )
P ( M c |
D )=
.
(4)
|M|
t =1 P ( M t ) P ( D |M t )
The marginal probability of the data D given model M c , P ( D
|
M c ) is involved in com-
puting (4) and can be calculated using:
M c )= P ( θ c |
P ( D
|
M c ) P ( D
|
θ c ,M c ) c ,
(5)
where θ c is the parameter vector corresponding to model M c . Evaluating the integral
in (5) can be complicated. Approximations such as the Laplace approximation and the
approximation based on Schwarz Bayesian Information Criterion (BIC) could be em-
ployed. However, in the linear model case, as in equation (3), where the coefficient
vector for model M c ,
N ( μ c ,V c ) and σ c
χ 2 ( ν,λ ) prior is used, an
β c
Inv
analytic expression for (5) is:
Γ ν + 2 ( νλ ) 2
P ( D c ,V c ,ν,X c ,M c )=
π 2 Γ 2 |
I + X c V c X c |
1 / 2
X c μ c )
×
[ λν +( Y
( I + X c V c X c ) 1
×
X c μ c )] ν + 2 ,
×
( Y
(6)
where μ c and V c are the mean vector and variance-covariance matrix, respectively, and
ν and λ are the degrees of freedom, and scale parameter, respectively. This work will
employ (6) for computing (5) versus any information criterion based on approximations.
In cases where the model space is sufficiently large, calculating (5) for each model
is computationally infeasible. A stochastic search through the model space can be
performed using a metropolis-hastings approach. This can be accomplished by con-
structing neighborhoods around the current model M c . Typically, the neighborhoods
nbd ( M c ) consist of all models with one additional term than model M c and all models
with one less term than model M c . For a candidate model M t
nbd ( M c ) the probabil-
ity, α , of acceptance of model M t is given by.
α =min 1 , P ( M t ) P ( D
,
|
M t )
q ( M t |
M c )
q ( M c |
(7)
P ( M c ) P ( D
|
M c )
M t )
where q ( M t |
M c ) is the probability that the candidate model is M t is selected for con-
sideration given the current state is model M c . Note the neighborhood structure men-
tioned above is not appropriate when the main effect terms are required to be in the
model whenever an epistatic term is in the model. [14], [15], [16], [17] and [13] allow
the neighborhood to be all models with one main effect term more or less than M c and
all models with one epistatic effect more or less than M c . These previously proposed
Search WWH ::




Custom Search