Information Technology Reference
In-Depth Information
The fundamental result in RKHS theory is the famed Moore-Aronszajn
theorem [1, 17]. Let K denote a generic symmetric and positive definite function
of two variables defined on some space E . That is, a function K
( ·, · )
: E
×
E
R
such that it verifies:
E .
(ii) Positive definiteness: for any finite number of l ( l
(i) Symmetry: K
(
x
,
y
)=
K
(
y
,
x
)
,
x
,
y
N
) points x 1 ,
x 2 ,...,
x l
E
and any corresponding coefficients c 1 ,
c 2 ,...,
c l R
,
l
m = 1
l
n = 1 c m c n K ( x m , x n ) 0 .
(1.1)
These are sometimes called the Mercer conditions [16] in the kernel methods
literature. Then, the Moore-Aronszajn theorem [1, 17] guaranties that there exists a
unique Hilbert space
H
of real valued functions defined on E such that, for every
x
E ,
(i) K
(
x
, · ) ∈H
and
(ii) for any f
∈H
f
(
x
)=
f
( · ) ,
K
(
x
, · ) H .
(1.2)
The identity on Equation (1.2) is called the reproducing property of K and, for this
reason,
H
is said to be an RKHS with reproducing kernel K .
Two essential corollaries of the theorem just described can be observed. First,
since both K
(
x
, · )
and K
(
y
, · )
are in
H
, we get from the reproducing property that
K
(
x
,
y
)=
K
(
x
, · ) ,
K
(
y
, · ) H .
(1.3)
Hence, K evaluates the inner product in this RKHS. This identity is the kernel trick ,
well known in kernel methods, and is the main tool for computation in this space.
Second, a consequence of the previous properties and which can be seen easily in
the kernel trick is that, given any point x
E , the representer of evaluation in the
RKHS is
Ψ x ( · )=
K
(
x
, · )
. Notice that the functional transformation
Ψ
from the input
space E into the RKHS
evaluated for a given x , and in general any element of the
RKHS, is a real function defined on E .
A quite interesting perspective to RKHS theory is provided by Parzen's work
[22]. In his work, Parzen proved that for any symmetric and positive definite func-
tion there exists a space of Gaussian distributed random variables defined in the
input space of the kernel for which this function is the covariance function [20].
Notice that, assuming stationarity and ergodicity, this space might just as well be
thought of as a space of random processes. That is to say that any kernel inducing
an RKHS denotes simultaneously an inner product in the RKHS and a covariance
operator in another space. Furthermore, it is established that there exists an iso-
metric inner product-preserving mapping, a congruence , between these two spaces.
Consequently, the RKHS
H
induced by the kernel and the space of random vari-
ables where this kernel is a covariance function are said to be congruent .Thisisan
important result as it sets up a correspondence between the inner product due to a
H
Search WWH ::




Custom Search