Information Technology Reference
In-Depth Information
methods. Therefore, the determination of prior distribution in Bayesian learning
is extremely important. If there is no prior information and we adopt
non-information prior, with the increase of sample, the effect of sample will
become more and more salient. If the noise of sample is small, the posterior will
be increasingly close to its true value. The only matter is that large computation
is required.
6.3.3
Steps of Bayesian problem solving
The steps of Bayesian problem solving can be summarized as follows:
(1) Define random variable. Set unknown parameters as random variable or
vector, shortly by
ȶ
. The joint density
p
(
x
1
, x
2
, ..., x
n
;
ȶ
) of sample
x
1,
x
2
, ..., x
n
is
regarded as the conditional density of
with respect to
ȶ
, shortly by
x
1
, x
2
, ..., x
n
x
1
, x
2
, ..., x
n
|
ȶ
) or p(D|
ȶ
).
(2) Determine prior distribution density
p
(
(
ȶ
). Use conjugate distribution. If there
is no information about prior distribution, then use Bayesian assumption of
non-information prior distribution.
(3) Calculate posterior distribution density via Bayesian theorem.
(4) Make inference of the problem with the resulting posterior distribution
density.
p
Take the case of single variable and single parameter for example. Consider the
problem of thumbtack throwing. If we throw a thumbtack up in the air, the
thumbtack will fall down and reset at one of the two states: on its head or on its
tail. Suppose we flip the thumbtack
N
+1 times. From the first
N
observations,
how can we get the probability of the case head in the
N
+1th throwing?
Step 1
Define a random variable
. The value
ȶ
corresponds to the possible
value of the real probability of head. The density function
p
(
ȶ
) represents the
uncertainty of
. The variable of the ith result is
X
i
(
i
=1,2,...,
N
+1), and the set of
observation is
D
={
X
1
=
x
1
,...,
X
n
=
x
n
}. Our objective is to calculate
p
(
x
N
+1
|D).
p
( )
θ
p D
p D
(
|
θ
)
Step 2
According to Bayesian theorem, we have
p
(
|D)=
, where
(
)
=
Ð
p D
(
)
p D
(
|
θ
)
p
( )
θ
d
θ
, p(D|
) is the binary likelihood function of sample.
