Information Technology Reference
In-Depth Information
1
R
L
0.5
d
0
0
0.5
1
Fig. 2.5 The dependence of
R
MSE
(solid line) and
R
CE
(dashed line) on
d
(Ex-
ample 2.3).
Given the dataset symmetry, we have:
2
1
T
=1]
=
1
d
((1
R
CE
(
d
)=
−
2
E
[ln(
Y
)
|
−
d
)ln(1
−
d
)+
d
)
.
(2.35)
Figure 2.5 shows the dependence of
R
MSE
and
R
CE
on
d
.Alarge
d
corre-
sponds to a large overlap of the distributions. Note the penalization of larger
errors in accordance to Fig. 2.4, and the special cases
d
=0(maximum class
separation with Dirac-
δ
distributions) and
d
=1(no class separation). In
this example the integrability condition for
R
CE
is satisfied for the whole
codomain.
2.3 The Error-Entropy Risk
2.3.1 EE Risks
The risk functional constituting the keystone of the present topic is simply
the entropy of the error r.v.,
E
, the error entropy, EE. Using the Shannon
differential entropy, the Shannon error-entropy (SEE) risk is written as
R
SEE
(
E
)
≡
H
S
(
E
)=
−
f
(
e
)ln
f
(
e
)
de .
(2.36)
E
Other formulations of the differential entropy can, in principle, be used. One
interesting formulation is the Rényi entropy [183] of order
α
: