Environmental Engineering Reference
In-Depth Information
solutions of the posterior distribution. For a given likelihood function, the conjugate prior
distribution will lead to a posterior distribution of the same analytical form (Raiffa and
Schlaifer 1961). One example of such conjugate priors was already seen in Illustration 1: If
the likelihood of a parameter θ is the normal distribution with mean θ and fixed standard
deviation σ
X
, the conjugate prior is the normal distrib
u
tion with mean
µ
θ
and standard
′
deviation
σ
θ
.
If
n
samples are taken with sample mean
x
,
the posterior distribution of θ is
the normal distribution with parameters:
′
(
µσ
′
/
′
2
)
+
(
nx
n
/
σ
2
)
θθ
θ
X
(5.28)
µ
′′ =
,
θ
(
1
/
σ
′
2
)
+
(
/
σ
2
)
X
−
σ
σσ
(/ )
12
1
2
n
(5.29)
′′ =
+
.
θ
′
2
θ
X
The reader is asked to apply these results to the data in Illustration 1. It can be observed that
the posterior mean depends only on the number and mean of the samples (in statistical jargon:
the sample number and sample mean are a sufficient statistic of the samples). The posterior
standard deviation depends only on the number of samples, but not on the actual outcome of
the samples. This shows that—for these distributions—the uncertainty in the posterior will
always be less than the uncertainty of the prior, independent of the measurement outcome.
Conjugate priors facilitate the choice of noninformative priors. For the above example, a
noninfor
m
ative prior is obtained by selecting ′ σ∞
θ
2
. In this case, the posterior parameters
are ′′ =
µ
θ
x
and ′′ σ
θ
X
/ . This was utilized in Illustration 3.
Additional examples of conjugate priors include the γ-distribution as the conjugate prior
of the parameter λ of the exponential distribution; and the β-distribution as the conjugate
prior of the parameter
p
of the binomial distribution. These and additional conjugate priors
are described in detail in Raiffa and Schlaifer (1961) and Fink (1997).
illustration 7: Conjugate prior of the normal distribution when
both mean and standard deviation are uncertain
We extend Illustration 1, considering now both the mean μ
φ
and the standard deviation σ
φ
of φ to be uncertain. φ is still modeled as normal distributed. For mathematical convenience,
it is beneficial to replace σ
φ
by the precision τ
1
2
/ the reciprocal of the variance.
Therefore, the parameters to estimate are θ = [μ
φ
, τ
φ
]. The conjugate prior for a normal dis-
tribution with unknown mean and precision is the normal-gamma-distribution, whose PDF
is (Raiffa and Schlaifer 1961; DeGroot 1969)
= (
σ
),
ϕ
ϕ
βλ
Γα π
τ
α
1
2
α
−
0.5
2
f
θϕϕ
(,)
µτ
=
exp(
−
βτ
)exp
−
λτµ
(
−
ν
)
(5.30)
ϕ
ϕ
ϕ
ϕ
()
2
The normal-gamma-distribution has parameters ν, λ, α, β. It is obtained by modeling τ
φ
as a gamma-distributed random variable with parameters α and β, and μ
φ
by a conditional
normal distribution given τ
φ
, with mean ν and precision τ
φ
λ.
The posterior marginal distribution of the mean μ
φ
is Student's
t
distribution with 2α
degrees of freedom:
(
)
− ′′ +
(
α
( / )
12
αλ βΓα
απΓα
/
(
′′
+
(
12
))
′′
′′
′′
λµ
′′
(
− ′′
ν )
2
1
2
ϕ
(5.31)
f
µϕ
′′
ϕ
µ
()
=
1
+
.
β
′′
2
′′
()
′′
Search WWH ::
Custom Search