Information Technology Reference
In-Depth Information
1
R L
0.5
d
0
0
0.5
1
Fig. 2.5 The dependence of R MSE (solid line) and R CE (dashed line) on d (Ex-
ample 2.3).
Given the dataset symmetry, we have:
2 1
T =1] =
1
d ((1
R CE ( d )=
2 E
[ln( Y )
|
d )ln(1
d )+ d ) .
(2.35)
Figure 2.5 shows the dependence of R MSE and R CE on d .Alarge d corre-
sponds to a large overlap of the distributions. Note the penalization of larger
errors in accordance to Fig. 2.4, and the special cases d =0(maximum class
separation with Dirac- δ distributions) and d =1(no class separation). In
this example the integrability condition for R CE is satisfied for the whole
codomain.
2.3 The Error-Entropy Risk
2.3.1 EE Risks
The risk functional constituting the keystone of the present topic is simply
the entropy of the error r.v., E , the error entropy, EE. Using the Shannon
differential entropy, the Shannon error-entropy (SEE) risk is written as
R SEE ( E )
H S ( E )=
f ( e )ln f ( e ) de .
(2.36)
E
Other formulations of the differential entropy can, in principle, be used. One
interesting formulation is the Rényi entropy [183] of order α :
Search WWH ::




Custom Search