Information Technology Reference
In-Depth Information
f
X|t
(x)
f
X|t
(x)
x
0
3
0
3
x
3
3
f
E
(e),
^
h=0.01
H
S
= −1.12
^
f
E
(e),
^
E
(e)
E
(e)
h=0.01
H
S
= −2.56
^
S
= −0.279
S
= 0.123
e
e
0
0
−2
0
2
−2
0
2
3
3
f
E
(e),
^
h=0.05
H
S
= −2.56
^
f
E
(e),
^
h=0.5
H
S
= −1.12
^
E
(e)
E
(e)
S
= 0.950
S
= 0.692
e
e
0
0
−2
0
2
−2
0
2
(a)
(b)
Fig. 3.28 Kernel smoothing effect for two locations of the split point: off-optimal
(a) and optimal (b). The top graphs are the Gaussian input PDFs. The middle and
bottom graphs show the theoretical (solid line) and smoothed (dotted line) error
PDFs for two different values of
h
.
where
e
+
(
x
;
λ
) is the exponential PDF with parameter
λ
, decaying for
x
≥
0.
The other PDF,
f
2
(
x
), corresponds to the optimal error PDF modeled as
f
2
(
x
;
a
)=
1
2
e
+
(
x
;
a
)+
1
2
e
−
(
x
;
a
)
,
where
e
−
(
x
;
a
) is the symmetrical of
e
+
(
x
;
a
).
Simple calculations of the Rényi entropies (for Shannon entropy, the prob-
lem is intractable) show that the respective entropies
H
1
and
H
2
are such
that
H
1
<H
2
(i.e., a maximum as in Fig. 3.28) for
λ>
2(
a
−
1).Wenow
proceed to convolve these PDFs with a Gaussian
G
h
(
·
). The resulting PDFs
are
Φ
x
+1
h
+
λ
2
e
λ
2
h
2
−λx
Φ
x
Φ
x
h
λh
2
h
f
1
(
x
)=
1
2
−
G
h
⊗
−
e
ax
1
Φ
x
+
ah
2
h
+
e
−ax
1
Φ
x − ah
2
h
f
2
(
x
)=
a
2
e
a
2
h
2
G
h
⊗
−
−
2
where
Φ
(
x
) denotes
x
−∞
g
(
u
;0
,
1)
du
, the standard Gaussian CDF.
Using these formulas and setting
λ
=2(
a
1) + 1 in order to have an
entropy maximum for the original
f
2
PDF, one may always find a suciently
large
h
such that the kernel smearing out of the tail of
f
1
will turn the entropy
of
f
2
into a minimum. This is exemplified in Fig. 3.29 for
a
=2
.
8 (
λ
=4
.
6).
−
