Cryptography Reference
In-Depth Information
c
c
0
0
(a) Normal distribution
(b) Mixture normal distribution
Fig. 7.14.
Histogram of correlation values.
EM Algorithm
The expectation-maximization (EM) algorithm is a representative maximum-
likelihood method for estimating the statistical parameters of a probability
distribution [24, 25]. For a mixture normal distribution comprising two nor-
mal distributions that the correlation values, c
(f,r)
k
, follow, the EM algorithm
can estimate the probability, w
(f,r)
k
, that each c
(f,r)
k
follows N (µ
(f,r)
,σ
2(f,r)
),
µ
(f,r)
,andσ
2(f,r)
. The relationship between w
(f,r)
k
, µ
(f,r)
,andσ
2(f,r)
is given
by
1
Kβ
(f,r)
w
(f,r)
k
c
(f,r)
k
µ
(f,r)
=
,
(7.32)
k
1
Kβ
(f,r)
w
(f,r)
k
c
(f,r)2
k
σ
2(f,r)
−µ
(f,r)2
,
=
(7.33)
k
k
w
(f,r
k
is the weighting factor of N (µ
(f,r)
,σ
2(f,r)
)to
the mixture normal distribution. These parameters are sequentially updated
from initial values by iterative calculation, and µ
(f,r)
where β
(f,r)
=1/K
and σ
2(f,r)
areusedas
estimates when they are converged. The µ
(f,r)
and σ
2(f,r)
are estimated from
K correlation values c
(f,r)
k
s:
Step 1: Set the initial values of the parameters β
(f,r)
, µ
(f,r)
,andσ
2(f,r)
to
β
(f,r)
[0], µ
(f,r)
[0], and σ
2(f,r)
[0].
Step 2: Do Step 3 through Step 5 over t =1, 2,.
Step 3: For each k calculate w
(f,r)
k
[t]fromc
(f,r)
k
:
[t]=
β
(f,r)
[t]φ(c
(f,r)
; µ
(f,r)
[t],σ
2(f,r)
[t])
w
(f,r)
k
k
,
(7.34)
g(c
(f,r)
k
; β
(f,r)
[t],µ
(f,r)
[t],σ
2(f,r)
[t])
where g(c; β, µ, σ
2
) is the probability density function of the mixture
normal distribution, that is,
