Information Technology Reference
In-Depth Information
n
,
v
h
(
T,Y
) is a consistent (in the mean square sense) estimator of
v
h
(
T,Y
);moreover,if
nh
→∞
→∞
and
h
→
0,
v
h
(
T,Y
) is an asymptotically
unbiased and consistent estimator of
f
E
(0).
Another interesting property is that correntropy is closely related to Hu-
ber's M-estimation [106] via the so-called
correntropy induced metric
(CIM):
CIM
(
T,Y
)=
v
(0
,
0)
−
v
(
T,Y
)
.
(5.32)
This metric applies different evaluations of distance depending on how far
the samples are: it goes from a
L
2
metric, when samples are close, to a
L
0
metric when samples are distant [141, 174]. This close-to-distant relation is
controlled by the kernel bandwidth
h
. Thus, correntropy and by consequence
the ZED risk, can be seen as robust risk functionals with a reduced sensitivity
to outliers. This theoretical results support what was discussed at the end of
the previous section and illustrated in Example 5.5.
For additional properties of correntropy and practical applications consult,
for example, [141, 174, 196, 118, 70].
5.2 A Parameterized Risk
In Sect. 5.1.2.2 we compared the gradients of MSE, CE and ZED empirical risks
to understand their behavior in iteractive training, namely when using the gra-
dient descent (ascent for ZED) algorithm. We argued that having a free param-
eter (the kernel bandwidth
h
) gives us an extra flexibility allowing to adapt the
risk functional to the particular classification problem at hand. Moreover, we
showed that for increasingly
h
in
R
ZED
we recover the MSE gradient behavior,
getting a 2-in-1 risk functional. Would it also be possible to incorporate the CE
risk type behavior in a general parameterized family of risk functionals? The
answer has been shown to be afirmative [218] as we now present.
5.2.1 The Exponential Risk Functional
We define the (empirical)
Exponential
risk functional as
τ
exp
e
i
τ
n
R
EXP
=
(5.33)
i
=1
2
h
2
we have
for a real parameter
τ
=0. We start by noticing that for
τ
=
−
1
τ
√
2
πhn
R
ZED
=
R
EXP
,
(5.34)
