Information Technology Reference
In-Depth Information
1
1
1
P
E
(e)
P
E
(e)
P
E
(e)
0.8
0.8
0.8
0.6
0.6
0.6
0.4
0.4
0.4
0.2
0.2
0.2
e
e
e
0
0
0
−2
0
2
−2
0
2
−2
0
2
(a)
H
S
=0
.
56
(b)
H
S
=0
.
04
(c)
H
S
=0
.
56
1
1
1
P
E
(e)
P
E
(e)
P
E
(e)
0.8
0.8
0.8
0.6
0.6
0.6
0.4
0.4
0.4
0.2
0.2
0.2
e
e
e
0
0
0
−2
0
2
−2
0
2
−2
0
2
(d)
H
S
=0
.
82
(e)
H
S
=0
.
83
(f)
H
S
=0
.
74
Fig. 4.4 Probability mass functions for distant (top) and close (bottom) classes in
the Stoller split setting. Figures from left to right correspond to the split position
at the left, at the location and at the right of the optimal split, respectively.
x
∗
,
P
E
(
e
) has more entropy than at any other split point (see the bottom
row of Fig. 4.4).
This behavior, observed for Gaussian classes, is in fact quite general, as
the following discussion demonstrates. The sign of
d
2
H
S
dx
2
(
x
∗
) (and thus, the
nature of the critical point
x
∗
) can be analyzed as a function of the distance
between the classes, defined as the distance
d
between their centers (medians).
For that, and w.l.o.g., let us consider that class
ω
1
is centered at 0 and the
center of class
ω
−
1
moves along the non-positive side of the real line. Then,
x
∗
=
−
d/
2.Now,
(
x
)ln
d
2
H
S
dx
2
(
x
)=
q
df
X|−
1
dx
P
−
1
−
1
−
P
−
1
−
P
1
(
x
)ln
p
df
X|
1
dx
P
1
−
−
−
P
−
1
−
P
1
1
q
2
f
X|−
1
(
x
)
P
−
1
p
2
f
X|
1
(
x
)
P
1
(
qf
X|−
1
(
x
)
pf
X|
1
(
x
))
2
−
−
−
−
.
1
−
P
−
1
−
P
1
(4.31)
From Theorems 4.1 and 4.2 one has
pf
X|
1
(
x
∗
)=
qf
X|−
1
(
x
∗
) and
P
−
1
(
x
∗
)=
P
1
(
x
∗
) therefore
