Digital Signal Processing Reference
In-Depth Information
from the samples. The conditional probability from above is
⎛
⎞
T
1
2
σ
2
⎝
−
⎠
μ, σ
2
)=(2
πσ
2
)
−T/2
exp
μ
)
2
p
(
x
(1)
,...,x
(
T
)
|
(
x
(
j
)
−
j=1
and hence the log likelihood is
T
T
2
ln(2
πσ
2
)
1
2
σ
2
μ, σ
2
)=
μ
)
2
.
ln
p
(
x
(1)
,...,x
(
T
)
|
−
−
(
x
(
j
)
−
j=1
The likelihood equation then gives the following two equations at the
maximum-likelihood estimates (
μ
ML
, σ
2
ML
):
T
∂
∂μ
ln
p
(
x
(1)
,...,x
(
T
)
1
σ
2
ML
μ
ML
, σ
2
ML
)=
|
(
x
(
j
)
−
μ
ML
)=0
j=1
∂
∂σ
2
T
2
σ
2
ML
μ
ML
, σ
2
ML
)=
ln
p
(
x
(1)
,...,x
(
T
)
|
−
+
T
1
2
σ
4
ML
(
x
(
j
)
−
μ
ML
)=0
j=1
From the first one, we get the maximum-likelihood estimate for the mean
T
1
T
μ
ML
=
x
(
j
)
j=1
which is precisely the sample mean estimator. From the second equation,
the maximum-likelihood estimator for the variance is calculated as
follows:
T
1
T
σ
2
ML
=
μ
ML
)
2
.
(
x
(
j
)
−
j=1
Note that this estimator is not unbiased, only asymptotically unbiased,
and it does not coincide with the sample variance.
3.3
Information Theory
After introducing the necessary probability theoretic terminology, we
now want to define the terms entropy and mutual information. These
Search WWH ::
Custom Search