Information Technology Reference
In-Depth Information
f(e)
f(e)
H = ln 2 = 0.693
H = ln 2 = 0.693
2
2
1
1
H =0
1
H =0
2
H =0
1
H =0
2
e
e
0.5
1.5
-2
2
-2
2
f(e)
f(e)
H = 0
H = ln 2 = 0.693
H = -0.693
2
2
2
1
1
H =0
1
H =0
1
H =0
2
e
e
0.5
1.5
-1.5
-0.5
0.75
1.25
-2
2
-2
2
f(e)
f(e)
H = 0.550
H = ln 2 = 0.693
H = -0.693
2
2
2
1
1
H =0
1
H =0
2
H =0.406
1
e
e
1.5
-0.25
0.75
1.25
0.5
-1.75
-2
2
-2
2
Fig. 2.6
H
S
(
E
)
properties for
p
=
q
=1
/
2
. Top row (Property 1): Entropy is
invariant to partitions and translations. Middle row (Property 2): For the same PDF
family, an increase in variance implies an increase in entropy (see text). Bottom row
(Property 3): The variance of the right component decreased by 0.25/12 while the
other increased by the same amount; however, the decrease in entropy of the right
component more than compensates for the increase in entropy of the left component.
wonder whether such an agreement is universal; stating in a different way,
is it possible to present examples of PDF families for which MSE and MEE
do not agree on which family member is most concentrated? The answer is
armative as we show in the following example [212].
Example 2.5.
Let us consider the following family of continuous PDFs, with
one parameter,
α>
0:
f
(
x
;
α
)=
1
4
[
tr
(
x
;0
,α
)+
tr
(
x
;
−
α,
0) +
tr
(
x
;0
,
1
/α
)+
tr
(
x
;
−
1
/α,
0)]
,
(2.46)
where
tr
(
x
;
a, b
) is the symmetrical triangular distribution in [
a, b
], defined
for
a
≥
0 as
tr
(
x
;
a, b
)=
4(
x−a
)
a
≤
x
≤
(
a
+
b
)
/
2
(
b−a
)
2
.
(2.47)
4(
b−x
)
(
b−a
)
2
(
a
+
b
)
/
2
<x
≤
b
