Biology Reference
In-Depth Information
A Formal Proof That Principal Components are Eigenvectors of the
Variance
Covariance Matrix
This is the derivation as presented by
Morrison (1990)
. Let us suppose that we have a
set of measures or coordinates
X
5
(
X
1
,
X
2
,
X
3
...X
P
), and we want to find the vector
A
1
5
(
A
11
,
A
21
,
A
31
...
A
P
1
) such that:
Y
1
5
A
11
X
1
1
A
21
X
2
1
A
31
X
3
1?1
A
P
1
X
P
(6.24)
We would like to maximize the variance of
Y
1
:
X
X
P
P
s
Y
1
5
A
i
1
A
j
1
s
ij
(6.25)
i
5
1
j
5
1
where
s
ij
is the element on the
i
th row and
j
th column of the variance
covariance matrix
S
of the observed specimens. We can write the variance of
Y
1
in matrix form as:
s
Y
1
5
A
1
SA
1
(6.26)
Now we seek to maximize
s
Y
1
subject to the constraint that
A
1
has a magnitude of one,
ðA
1
A
1
5
which means that
To do this, we introduce a term called a Lagrange multiplier
λ
1
, and use it to form the expression:
s
Y
1
1
λ
1
ð
1
Þ:
A
1
A
1
Þ
1
(6.27)
which we seek to maximize with respect to
A
1
. Therefore, we take this new expression for
the variance of
Y
1
and set its partial derivative with respect to
A
1
to zero:
@
n
o
s
Y
1
1
λ
2
A
1
A
1
Þ
ð
1
0
(6.28)
5
1
@
A
1
Using
Equation 6.26
, we can expand the expression for the partial derivative to:
@
5
A
1
A
1
SA
1
1
λ
1
ð
2
A
1
A
1
Þ
1
0
(6.29)
@
which we now simplify to:
2
ðS
2
λ
1
IÞA
1
5
0
(6.30)
where
I
is the
P
P
identity matrix. Because
A
1
cannot be zero,
Equation 6.30
is a vector
multiple of
Equation 6.16
, the characteristic equation. In
Equation 6.30
,
3
λ
1
is the eigenvalue
and
A
1
is the corresponding eigenvector.
Given
Equation 6.30
, we can also state that:
ðS
2
λ
1
IÞA
1
5
0
(6.31)
This can be rearranged as:
SA
1
2
λ
1
IA
1
5
0
(6.32)
