Database Reference
In-Depth Information
= λ
e
=
e
λ for each eigenvector
e
and its corresponding eigenvalue λ, it follows that
M
T
ME
=
EL
.
We observed that
ME
is the points of
M
transformed into a new coordinate space. In
this space, the first axis (the one corresponding to the largest eigenvalue) is the most sig-
nificant; formally, the variance of points along that axis is the greatest. The second axis,
corresponding to the second eigenpair, is next most significant in the same sense, and the
pattern continues for each of the eigenpairs. If we want to transform
M
to a space with few-
er dimensions, then the choice that preserves the most significance is the one that uses the
eigenvectors associated with the largest eigenvalues and ignores the other eigenvalues.
That is, let
E
k
be the first
k
columns of
E
. Then
ME
k
is a
k
-dimensional representation of
M
.
EXAMPLE
11.6 Let
M
be the matrix from
Section 11.2.1
.
This data has only two dimen-
sions, so the only dimensionality reduction we can do is to use
k
= 1; i.e., project the data
onto a one-dimensional space. That is, we compute
ME
1
by
The effect of this transformation is to replace the points of
M
by their projections onto the
and fourth points, this representation makes the best possible one-dimensional distinctions
among the points.
□
11.2.3
The Matrix of Distances
examine the eigenvalues of
MM
T
. Since our example
M
has more rows than columns, the
latter is a bigger matrix than the former, but if
M
had more columns than rows, we would
actually get a smaller matrix. In the running example, we have
Like
M
T
M
, we see that
MM
T
is symmetric. The entry in the
i
th row and
j
th column has
a simple interpretation: it is the dot product of the vectors represented by the
i
th and
j
th
points (rows of
M
).
There is a strong relationship between the eigenvalues of
M
T
M
and
MM
T
. Suppose
e
is
an eigenvector of
M
T
M
; that is,
