Information Technology Reference
In-Depth Information
150
0.65
V
R
2
^
^
(h=0.5)
R
2
0.6
(h=0.001)
^
* (h=0.5)
R
2
R
2
0.55
^
* (h=0.005)
R
2
100
0.5
0.45
0.4
50
0.35
0.3
w
0
w
0
0
0.25
−1
0
1
2
3
4
−1
0
1
2
3
4
(a)
(b)
V
R
2
V
R
2
Fig. 3.30
(a)
V
R
2
(solid),
(dashed), and
(dotted) of the data splitter as
a function of
w
0
for a small value of
h
;(b)
V
R
2
V
R
2
and
for a higher value of
h
.
V
R
2
=
n
−
1
2
G
ij
+
n
1
n
2
c
−
1
i∈ω
−
1
c
1
i∈ω
1
G
ij
+2
c
i∈ω
1
G
ij
n
j∈ω
−
1
j∈ω
1
j∈ω
−
1
=
q
2
V
R
2
|−
1
+
p
2
V
R
2
|
1
+2
c
i∈ω
1
V
R
2
+2
c
i∈ω
1
G
ij
=
G
ij
.
j∈ω
−
1
j∈ω
−
1
Entropy is therefore decomposed as a weighted sum of positive class-
conditional potentials (as in the theoretic derivation of Appendix C), denoted
V
R
2
, plus a term that exclusively relates to cross-errors. Figure 3.30 compares
the behavior of
V
R
2
,
V
R
2
,and
V
R
2
as a function of the split parameter
w
0
for the same problem as in Fig. 3.28. From its inspection we first note the
minimum of
V
R
2
, corresponding to the entropy maximum at the optimal so-
lution
w
0
=0
.
5. Analyzing the behavior of
V
R
2
,and
V
R
2
as in Fig. 3.30a it is
possible to confirm the convergence of both terms towards
V
R
2
0.
If
h
is increased above a certain value,
V
R
2
,and
V
R
2
will exhibit a maximum
at
w
0
, but with an important difference: while the
V
R
2
maximum is not a
global one (for any
h
), the maximum of
V
R
2
is a global one.
Thus, to maximize the empirical information
V
R
2
, it is important not only
to maximize
q
2
V
R
2
|−
1
+
p
2
V
R
2
|
1
as for the theoretical counterpart (which
can be achieved with different PDF configurations with consequences to the
classifier performance; see Property 1 of Sect. 2.3.4) but also to maximize
with
h
→
2
c
i∈ω
1
j∈ω
−
1
G
ij
(with no theoretical counterpart), which is achieved if
the errors are concentrated at the origin. This cross-error term is due to kernel
smoothing.
Note that this analysis also justifies that a larger
h
is needed to obtain
an information maximum for a larger overlap of the class-conditional error
PDFs.
