Information Technology Reference
In-Depth Information
Chapter 5
EE-Inspired Risks
In the previous chapters the behavior of classifiers trained to minimize error-
entropy risks, for both discrete and continuous errors, was analyzed. The
rationale behind the use of these risks is the fact that entropy is a PDF con-
centration measure — higher concentration implies lower entropy —, and in
addition (recalling what was said in Sect. 2.3.1) minimum entropy is attained
for Dirac-
δ
combs (including a single one). Ideally, in supervised classification,
one would like to drive the learning process such that the final distribution
of the error variable is a Dirac-
δ
centered at the origin. In rigor, this would
only happen for completely separable classes when dealing with the discrete
error case or with infinitely distant classes when dealing with the continuous
error case with the whole real line as support. For practical problems we will
content ourselves in achieving the highest possible error concentration at the
origin, namely by driving the classifier training process to an error PDF (or
PMF) with the highest possible value at the origin. Formally, the process
may be expressed as
Find
w
s.t.
w
=argmax
w
∈W
f
E
(
0
;
w
) for continuous errors
(5.1)
or
w
=argmax
w
∈
W
P
E
(
0
;
w
) for discrete errors
(5.2)
where
w
is the parameter vector of the classifier and
w
its optimal value in
the Zero-Error Density (Probability) Maximization (ZED(P)M) sense (first
proposed in the 2005 work [213]). In the following we will concentrate on
the continuous error case and explore this approach by deriving the Zero-
Error Density (ZED) risk and further evolve to a generalized risk capable of
emulating a whole family of risks [218]. As before, we focus on the two-class
case.
