Information Technology Reference
In-Depth Information
The selection of the main components (
q
out of
p
) results from an analysis of
the eigenvalues. Before describing it, it is worthwhile reviewing a technique,
similar to PCA, which is extensively used in linear algebra: singular value
decomposition (SVD) [Cichoki 1993]. That technique, which is very useful for
solving linear systems, was mentioned in the previous chapter as a tool for
calculating leverages for nonlinear models.
∈
R
n×p
matrices, there exist two orthogonal matrices
Theorem.
For al l A
∈
R
n×p
and V
∈
R
p×p
such that
U
⎛
⎝
⎞
⎠
σ
1
0
···
0
0
σ
2
···
0
U
T
AV
=
S
=
,
.
.
.
.
.
.
.
0
0
···
0
σ
m
with σ
1
≥
σ
2
≥···≥
σ
m
≥
0
,wherem
=min(
p,n
)
.
The elements of the diagonal matrix
S
are the singular values
σ
j
,ranked
in decreasing order. The singular values
σ
j
are the square roots of the eigen-
values
λ
j
of the positive symmetrical matrix
A
T
A
or matrix
AA
T
if
m<n
.
The columns of the matrix
V
associated with the change of variables are the
eigenvectors of matrix
A
T
A
.
PCA and SVD
Therefore, PCA and SVD are equivalent when operated on centered data.
Unlike diagonalization techniques for square matrices, singular value de-
composition applies to all types of matrices. The index for the 1st singular
value equal to 0 is the rank of the matrix; its condition number is the ratio of
the largest to the smallest singular value
σ
1
/σ
p
.
From the orthogonality of matrices U and V, one has
U
T
AV
=
S
A
=
USV
T
.
⇒
In a modeling application, if
A
is the matrix of centered observations (defined
in the previous chapter), matrix
US
=
AV
describes the same observations in
an “orthogonal” representation: the new inputs obtained after transformation
are not subject to linear correlation. The same technique is used for “cleaning”
signals [Davaud 1991]. In order to reduce the new inputs, matrix
U
is retained
as a new base of examples: the linear transformation thus becomes
S
−
1
V
T
x
instead of
V
T
x
.
Singular value decomposition of the matrix of centered data
X
is used to
express the inertia with respect to the singular values
σ
j
or the eigenvalues
of matrix
X
T
X
,
λ
j
p
p
I
p
=Tr
X
T
X
⇒
σ
2
j
I
p
=
λ
j
⇒
I
p
=
.
j
=1
j
=1
Search WWH ::
Custom Search