Information Technology Reference
In-Depth Information
0.8
0.8
P
e
(x'),H
S
(x')
P
e
(x'),H
S
(x')
0.7
0.7
0.6
0.6
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.2
x'
x'
0.1
0.1
−1
−0.5
0
0.5
1
1.5
2
2.5
−1
−0.5
0
0.5
1
1.5
2
2.5
3
(a)
(b)
Fig. 4.2
H
S
(dashed line) and
P
e
(solid line) plotted as functions of
x
in a two-
class Uniform problem: a)
[
a, b
]=[0
,
1]
,
[
c, d
]=[0
.
7
,
1
.
7]
and
p
=
q
=0
.
5
;b)
[
a, b
]=[0
,
1]
,
[
c, d
]=[0
.
5
,
2]
and
p
=0
.
7
.
4.3c (and similar ones) suggest that the correspondence between the MEE
solution and the min
P
e
solution depend on some sort of
qf
X|−
1
vs
pf
X|
1
balance between the classes. Second, the min
P
e
solution may correspond
to an entropy maximum depending on the distance between the classes as
illustrated in Figs. 4.3a and 4.3d.
4.1.2 SEE Critical Points
Our first goal is to compute the several risks described in Chap. 2, using for-
mulas (4.17) and analyze their critical points by relating them to the min
P
e
solution. We then focus our analysis in SEE seeking a theoretical explanation
for the behavior encountered in Example 4.2.
Let
p
=
q
=1
/
2 and recall from Theorem 4.1 that at the min
P
e
solution
f
X|−
1
(
x
∗
)=
f
X|
1
(
x
∗
). Then:
1. MSE risk
R
MSE
(
x
)=
e
e
2
P
E
(
e
)=(
2)
2
P
−
1
+(0)
2
(1
P
1
)+(2)
2
P
1
−
−
P
−
1
−
=4(
P
−
1
+
P
1
)=4
P
e
.
(4.22)
We see that
R
MSE
(
x
) depends, in this case, only on the misclassified
instances. In order to determine
x
∗
, first note that by the Fundamental
Theorem of Calculus
dF
X|t
/dx
=
f
X|t
(
x
). Therefore:
