Information Technology Reference
In-Depth Information
,
[ln
f
(
x
)]
2
f
(
x
)
dx <
,f
(
x
) is
• L
1
consistency: if
nh
→∞
as
n
→∞
∞
,and
|
H
S
(
X
)
f
(
x
)
continuous and sup
|
|
<
∞
u
|
K
(
u
)
du <
∞
,then
E
[
|
−
0.
• L
2
consistency: if, in addition,
(
f
(
x
)
/f
(
x
))
2
f
(
x
)
dx <
H
S
(
X
)
|
]
→
n
→∞
∞
(finite Fisher
H
S
(
X
)
information number), then
E
[
|
−
H
S
(
X
)
|
2
]
→
0.
n
→∞
•
MSE consistency, a consequence of the Parzen window MSE consistency.
Almost sure (a.s.) consistency:
H
S
(
X
)
•
→
n→∞
H
S
(
X
) a.s., under certain
mild conditions (see [159]).
F.4 Plug-in Estimate of Rényi's Entropy
Let us consider the expression of Rényi's entropy of order
α
:
α
ln
E
1
1
f
α
de
=
H
R
α
(
X
)=
α
ln
V
α
,α
≥
0
.
(F.12)
1
−
1
−
Since the information potential
V
α
is the mean of
f
α−
1
(
x
), the plug-in esti-
mator is immediately written as:
α
ln
1
(
x
i
)
=
n
1
H
R
α
(
X
)=
f
α−
1
n
1
−
n
i
=1
⎛
⎞
n
n
1
1
n
α
⎝
⎠
.
K
h
(
x
i
−
x
j
)
=
α
ln
(F.13)
1
−
i
=1
j
=1
The MSE consistency of the Parzen window estimate (see Appendix E) di-
rectly implies the MSE consistency of the
H
R
α
estimate. Note that for the
quadratic entropy with Gaussian kernel there is a difference of
√
2 in the
bandwidths of both estimates (F.9) and (F.13). This difference is unimpor-
tant in practical terms.
