Statistical Characterization of Signals and Systems - Unsupervised Signal Processing - page 51

deterministic, while x and θ are related via the joint pdf p X ,θ ( x , θ ) when θ is

∂ θ T random. Also, θ is statistically dependent on θ , and it holds that 0 .

Since the problem is mathematically stated, it is relevant to mention some
important statistical properties of the estimators.

/∂ θ =

2.5.2 Properties of Estimators

2.5.2.1 Bias
An estimator θ is said to be unbiased if
E θ = E { θ } (2.124)

otherwise, it is said to be biased with
Bias θ = E θ − θ = E θ

E { θ } − (2.125)

When θ is deterministic, we have E θ.

When multiple parameters are considered and the estimator θ is unbi-
ased, we have

{ θ } =

E θ

= E { }

θ (2.126)

and the bias is given by
Bias θ

E θ − θ E θ = = E { θ } − (2.127)

2.5.2.2 Efficiency
For two unbiased estimators θ and θ, we say that θ is more efficient than θ if
var θ ≤ var θ

(2.128)

Additionally, we can define the estimation error as

− θ

ε = θ (2.129)

so that the notion of efficiency is related to achieving an unbiased estimator
θ with the smallest error variance, which is given by var

var θ .
For the multiple-parameter case, the estimator θ is said to be more
efficient than θ , assuming both to be unbiased, if

( ε ) =

θ θ ≤ θ θ

C C (2.130)