Information Technology Reference
In-Depth Information
If
, the value of
, is known; the observation value in D is independent; and the
probability of head (tail) is
ȶ
, the probability of tail is (1-
ȶ
); then:
p
(
θ
)
θ
h
(
−
θ
)
t
p
(
ȶ
|
D
)=
(6.11)
p
(
D
)
Where
are the times of head and tail in the observation D respectively.
They are sufficient statistics of sample binary distribution.
Step 3
Seek the mean of
as the probability of case head in the
h
and
t
N
+1th toss:
Ð
p
(
X
=
heads
|
D
)
=
p
(
X
=
heads
|
θ
)
p
(
θ
|
D
)
d
θ
N
+
1
N
+
1
Ð
θ
⋅
p
(
θ
|
D d
)
θ
=
E
p(
|D)
(
)
(6.12)
Where E
p(
|D)
(
ȶ
) is the expectation of
under the distribution
p
(
|D)
Step 4
Assign prior distribution and supper parameters for
.
Common method for prior assignment is to assume prior distribution first, and
then to determine proper parameters. Here we assume the prior distribution is
Beta distribution:
Γ α
(
)
(
)
α
−
1
α
−
1
θ α
|
,
α
θ
(1
−
θ
)
p(
)=Beta
(6.13)
h
t
h
t
Γ α
(
)
Γ α
(
)
h
t
Where α
h
>0 and α
t
>0 are parameters of Beta distribution, and α = α
h
+ α
t
, and
(·) is Gamma function. To distinguish with parameter
ȶ
,
h and α
t
are called
“Supper Parameters”. Because Beta distribution belongs to conjugate family, the
resulting posterior is also Beta distribution.
Γ α
(
+
N
)
α
+
h
−
1
α
+ −
t
1
θ
(1
−
θ
)
p(
|D)=
h
t
Γ α
(
+
h
)
Γ α
(
+
t
)
h
t
(
)
θ α + +
(6.14)
To this distribution, its expectation of
has a simple form:
|
h
,
t
=Beta
h
t
α
α
Ð
θ
⋅
Beta
(
θ α
|
,
α
)
d
θ
=
(6.15)
h
t
Therefore, for a given Beta prior, we get the probability of head in the
N
+1th toss
as follows:
α
α
+
h
p X
(
=
heads
|
D
)
=
h
(6.16)
N
+
1
+
N
There are many ways to determine the supper parameters of prior Beta
distribution p(
ȶ
), such as imagined future data and equivalent samples. Other
methods can be found in the works of Winkler, Chaloner and Duncan. In the
method of imagined future data, two equations can be deduced from equation
(6.16), and two supper parameters α
h
and α
t
can be solved accordingly.
