Dimensionality Reduction - Mining of Massive Datasets - page 394

this fact, notice that the point of projection for both the first and second points is [1.5, 1.5]
in the original coordinate system, and the distance from the origin to this point is

Moreover, the new y -axis is, of course, perpendicular to the dashed line. The first point is at
distance above the new x -axis in the direction of the y -axis. That is, the distance between
the points [ 1 , 2 ] and [1.5, 1.5] is

Figure 11.3 shows the four points in the rotated coordinate system.

Figure 11.2 Figure 11.1 with the axes rotated 45 degrees counterclockwise

Figure 11.3 The points of Fig. 11.1 in the new coordinate system

The second point, [ 2 , 1 ] happens by coincidence to project onto the same point of the
new x -axis. It is below that axis along the new y -axis, as is confirmed by the fact that
the second row in the matrix of transformed points is The third point, [ 3 , 4 ] is trans-
formed into and the fourth point, [ 4 , 3 ] , is transformed to That is, they both
project onto the same point of the new x -axis, and that point is at distance from the ori-
gin, while they are above and below the new x -axis in the direction of the new y -axis.

11.2.2 Using Eigenvectors for Dimensionality Reduction

From the example we have just worked out, we can see a general principle. If M is a matrix
whose rows each represent a point in a Euclidean space with any number of dimensions, we
can compute M T M and compute its eigenpairs. Let E be the matrix whose columns are the
eigenvectors, ordered as largest eigenvalue first. Define the matrix L to have the eigenval-
ues of M T M along the diagonal, largest first, and 0's in all other entries. Then, since M T M e