Database Reference
In-Depth Information
3.2.1 Joint, Marginal, and Conditional Gaussians.
In
this section we interchange the terms normal and Gaussian distribution.
In both cases, we are referring to the same density function (shown in
Eq. 10.6).
1
/
2
exp
μ
)
1
(2
π
)
ρ/
2
1
2
(
x
−
μ
)
T
Σ
−
1
(
x
−
p
(
x
1
,...,x
ρ
)=
−
(10.6)
|
Σ
|
For any set of random variables that are jointly normally distributed, all
marginal and conditional distributions associated with any individual
or subgroup of variables are also normally distributed. For example,
consider the simple bivariate normal distribution
p
(
x,y
) with parameters
given in Eq. 10.7.
(
μ
x
,
Σ
x
)
Σ
x,y
p
(
x,y
)=
N
μ
y
Σ
y,x
Σ
y
The marginal distribution for
x
is computed by integrating over all pos-
sible values of
y
.Thatis,
p
(
x
)=
p
(
x,y
)
dy
. For a joint Gaussian,
marginalization or
integrating out
variables is a simple procedure, since
the result is a normal distribution we must only find the mean and
covariance matrix that specify the distribution. In this case, we sim-
ply take the mean and covariance sub-matrix corresponding only to the
variable(s) of interest and the marginal is again normally distributed.
For example, using the joint in Eq. 10.7,
p
(
x
)=
(
μ
x
,
Σ
x
).
Marginalization is the process of simply removing a variable from
our distribution. However, this process does not provide us with any
additional information about the variable of interest, it only serves to
simplify our distribution form by reducing dimensionality. Alternatively,
it may be possible to observe the value of a variable may provide us with
information about our variable of interest. In this case we are interested
in computing the
conditional distribution
. That is, the distribution over
x
given that you have observed a specific value for
y
.Inthecaseofjointly
distributed Gaussian variables, the conditional distribution is again a
Gaussian. The parameters for a conditional Gaussian are shown below.
N
p
(
x
(
m
x|y
,P
x|y
)
m
x|y
=
μ
x
+Σ
x,y
Σ
−
y
(
y
|
y
)=
N
−
μ
y
)
(10.7)
Σ
x,y
Σ
−
y
Σ
y,x
P
x|y
=Σ
x
−
(10.8)
The interpretation of these updated parameter values is quite intuitive.
For instance, Eq. 10.7 says that the updated mean is the marginal mean
of
x
corrected by some value. This correction term depends on the cou-
pling between the two variables which is encoded in the covariance term,
