Digital Signal Processing Reference
In-Depth Information
is the Fisher information matrix. For this case, the equality in (2.136) holds if
and only if
)
∂
p
X
(
x
;
θ
)
θ
−
θ
=
I
(
θ
(2.137)
∂
θ
where
I
is a matrix composed of nonzero functions.
Some aspects concerning the estimator properties for infinite data length
(asymptotic properties) may also be of interest. For instance, Refs. [70, 165]
present relevant discussions on this topic, which is out of our main focus
of presenting estimation theory methods as a support of the optimal filtering
techniques. Therefore, we now turn our attention to the design of estimators.
We will consider the case of multiple-parameter estimation, which is more
general and more relevant to our objectives.
(
θ
)
2.5.3 Maximum Likelihood Estimation
The ML estimation method consists of finding the estimator that maxi-
mizes the likelihood function established between the observed data and the
parameters. In other words, for a given set of available measurements
x
,we
search for the parameters
θ
that provide the highest probability
p
X
(
with
which the observed data would have been generated. Thus, the ML estimator
is given by
x
|
θ
)
θ
ML
=
arg max
θ
p
X
(
x
|
θ
)
(2.138)
where the pdf
p
X
(
is the likelihood function. Hence, we need to find the
maximum of this function, which is given by its first derivative.
Due to the widespread use of exponential families of pdfs, it is very usual
to use the log-likelihood. This being the case, the solution of the likelihood
equation is given by
x
|
θ
)
)
θ
=
θ
ML
=
ln
p
X
∂
∂
(
|
x
θ
0
(2.139)
θ
When Equation 2.139 presents several solutions, one must keep the
θ
that
corresponds to the global maximum.
One important feature of the ML estimator is that it is asymptotically
efficient, that is, the ML estimator achieves the CRB when the number of
observed data tends to infinity [165].
2.5.4 Bayesian Approach
As mentioned before, the Bayesian approach assumes that the parame-
ters to be estimated are r.v.'s. In such case, we need to have some sort of
