Information Technology Reference
In-Depth Information
5.1.2.3
ZED versus Correntropy
The ZED risk functional can also be derived from the framework of the
cross-
correntropy
or simply
correntropy
function, a generalized correlation function
first proposed in the 2006 work [196] (the name was chosen to reflect the
connection of the proposed correlation measure to Rényi's quadratic entropy
estimator). Later, a more general form of correntropy was proposed [141,174],
as a generalized similarity measure between two arbitrarily scalar random
variables
X
and
Y
defined as
v
(
X, Y
)=
K
(
x, y
)
f
X,Y
(
x, y
)
dxdy .
(5.27)
Here,
K
(
x, y
) is any continuous positive definite kernel with finite maximal
value. If
K
(
x, y
)=
xy
, then the conventional cross-correlation is obtained as
a special case, but the authors concentrate on the special case of a Gaussian
kernel, giving
v
h
(
X, Y
)=
G
h
(
x
−
y
)
f
X,Y
(
x, y
)
dxdy .
(5.28)
1
/
√
2
πh
2
),
reaching its maximum if and only if
X
=
Y
. To put this on the pattern recog-
nition framework, consider as before the error variable
E
=
T
Correntropy is positive and bounded (in particular, 0
<v
h
(
X, Y
)
≤
−
Y
. Then, one
can define the correntropy between
T
and
Y
as
=
G
h
(
e
)
f
E
(
e
)
de.
v
h
(
T,Y
)=
E
X,Y
{
G
h
(
T
−
Y
)
}
=
E
E
{
G
h
(
E
)
}
(5.29)
An important property now appears [141, 174]. If
h
→
0 then
v
h
(
T,Y
)
amounts to
f
E
(0),thatis,
lim
h
0
v
h
(
T,Y
)=
f
E
(0)
.
(5.30)
→
So, maximizing the correntropy (coined MCC in [141]) between the target and
output variables is equivalent to maximize the error density at the origin,
provided a suciently small
h
is considered. This is the same idea as the
ZEDM principle with risk functional given by
R
ZED
as in formula (5.3).
The empirical version of correntropy is obtained by noticing that
v
h
(
T,Y
)
is an expected value, giving the following sample estimate
n
v
h
(
T,Y
)=
1
n
G
h
(
e
i
)=
f
E
(0)
,
(5.31)
i
=1
which is precisely the empirical ZED risk of formula (5.16) with Gaussian
kernel. This estimator has some good properties as shown in [141, 174]: if
