Information Technology Reference
In-Depth Information
The fundamental result in RKHS theory is the famed
Moore-Aronszajn
theorem
[1, 17]. Let
K
denote a generic symmetric and positive definite function
of two variables defined on some space
E
. That is, a function
K
(
·, ·
)
:
E
×
E
→
R
such that it verifies:
E
.
(ii) Positive definiteness: for any finite number of
l
(
l
(i) Symmetry:
K
(
x
,
y
)=
K
(
y
,
x
)
,
∀
x
,
y
∈
∈
N
) points
x
1
,
x
2
,...,
x
l
∈
E
and any corresponding coefficients
c
1
,
c
2
,...,
c
l
∈
R
,
l
m
=
1
l
n
=
1
c
m
c
n
K
(
x
m
,
x
n
)
≥
0
.
(1.1)
These are sometimes called the Mercer conditions [16] in the kernel methods
literature. Then, the Moore-Aronszajn theorem [1, 17] guaranties that there exists a
unique Hilbert space
H
of real valued functions defined on
E
such that, for every
x
∈
E
,
(i)
K
(
x
, ·
)
∈H
and
(ii) for any
f
∈H
f
(
x
)=
f
(
·
)
,
K
(
x
, ·
)
H
.
(1.2)
The identity on Equation (1.2) is called the
reproducing property
of
K
and, for this
reason,
H
is said to be an RKHS with reproducing kernel
K
.
Two essential corollaries of the theorem just described can be observed. First,
since both
K
(
x
, ·
)
and
K
(
y
, ·
)
are in
H
, we get from the reproducing property that
K
(
x
,
y
)=
K
(
x
, ·
)
,
K
(
y
, ·
)
H
.
(1.3)
Hence,
K
evaluates the inner product in this RKHS. This identity is the
kernel trick
,
well known in kernel methods, and is the main tool for computation in this space.
Second, a consequence of the previous properties and which can be seen easily in
the kernel trick is that, given any point
x
∈
E
, the representer of evaluation in the
RKHS is
Ψ
x
(
·
)=
K
(
x
, ·
)
. Notice that the
functional transformation
Ψ
from the input
space
E
into the RKHS
evaluated for a given
x
, and in general any element of the
RKHS, is a real function defined on
E
.
A quite interesting perspective to RKHS theory is provided by Parzen's work
[22]. In his work, Parzen proved that for
any
symmetric and positive definite func-
tion there exists a space of Gaussian distributed random variables defined in the
input space of the kernel for which this function is the covariance function [20].
Notice that, assuming stationarity and ergodicity, this space might just as well be
thought of as a space of random processes. That is to say that any kernel inducing
an RKHS denotes simultaneously an inner product in the RKHS and a covariance
operator in another space. Furthermore, it is established that there exists an iso-
metric inner product-preserving mapping, a
congruence
, between these two spaces.
Consequently, the RKHS
H
induced by the kernel and the space of random vari-
ables where this kernel is a covariance function are said to be
congruent
.Thisisan
important result as it sets up a correspondence between the inner product due to a
H