Graphics Reference
In-Depth Information
eigenvalues; if none of the eigenvalues are complex, then the matrix A can be
decomposed in the form
U T
A
=
U
Λ
(9.5)
where U is a matrix of orthogonal eigenvectors, and
is a diagonal matrix of
the eigenvalues. The eigenvectors corresponding to the smallest eigenvalues are
regarded as the least important in PCA. Setting small eigenvalues to zero in Equa-
tion (9.5) results in an approximation to A .
Eigenvalue analysis has some limitations: it only works on a square matrix,
only certain square matrices have only real eigenvalues. Furthermore the eigen-
value decomposition is numerically difficult unless the matrix is symmetric. Orig-
inally, principal component analysis was applied to a specially constructed matrix
that had properties amenable to eigenvalue analysis. PCA is generally applied to
amatrix AA T , which is necessarily symmetric. A more general matrix decompo-
sition is the singular value decomposition (SVD), which is a factorization of an
arbitrary matrix M into a product of the form
Λ
=
v 1
v
.
v n
σ 1
σ 2
= u 1
u m
V T
M
u 2
···
U
Σ
.
(9.6)
. . .
σ n
(The process of computing the SVD is an important problem in numerical linear
algebra.) The diagonal elements of
are the singular values ; normally the sin-
gular values are arranged in decreasing order of absolute value. The matrices U
and V are orthonormal square matrices consisting of the left-singular and right-
singular vectors, which are the eigenvectors of MM T and M T M , respectively.
Figure 9.9 illustrates the concept graphically. The singular value decomposition
can be regarded as an eigenvalue decomposition for a nonsquare matrix M .In
fact, there is a direct connection to the eigenvalues of MM T and M T M :
Σ
V Σ
Σ V T
M T M
T
=
,
(9.7)
U
T U T
MM T
=
ΣΣ
.
(9.8)
In other words, the singular values of M are the square roots of the eigenvalues of
M T M and MM T ; V consists of the eigenvectors of M T M ,and U consists of the
eigenvectors of MM T .
The application of the SVD to principal component analysis comes from the
interpretation of the singular values. Each singular value
σ k corresponds to a left-
singular vector and a right-singular vector, and in a manner of speaking, provides
the “weight” of those vectors. Setting
σ k to zero “turns off” the corresponding
Search WWH ::




Custom Search