SUBSPACE TRACKING FOR SIGNAL PROCESSING - Adaptive Signal Processing: Next Generation Solutions - page 215

4.2.3 Variational Characterization of Eigenvalues / Eigenvectors
of Real Symmetric Matrices

The eigenvalues of a general n n matrix C are only characterized as the roots of
the associated characteristic equation. But for real symmetric matrices, they can be
characterized as the solutions to a series of optimization problems. In particular, the
largest l 1 and the smallest l n eigenvalues of C are solutions to the following
constrained maximum and minimum problem (see e.g. [37, Sec. 4.2]).

n w T Cw and l n ¼ n w T Cw:

l 1 ¼ max

kwk 2 ¼ 1, w

min
kwk 2 ¼ 1, w

(4 : 3)

[ R [ R

Furthermore, the maximum and minimum are attained by the unit 2-norm eigen-
vectors u 1 and u n associated with l 1 and l n respectively, which are unique up to a
sign for simple eigenvalues l 1 and l n . For nonzero vectors w [ R

n , the expression

w T Cw

w T w is known as the Rayleigh's quotient and the constrained maximization and
minimization (4.3) can be replaced by the following unconstrained maximization
and minimization

w T Cw
w T w

w T Cw
w T w :

l 1 ¼ max

w = 0 , w [ R

and l n ¼ min

w = 0 , w [ R

(4 : 4)

n n

For simple eigenvalues l 1 , l 2 , ... , l r or l n , l n -1 , ... , l n - rþ 1 , (4.3) extends
by the following iterative constrained maximizations and minimizations (see e.g.
[37, Sec. 4.2])

n w T Cw ,

l k ¼ max

kwk 2 ¼ 1, w?u 1 , u 2 ,

k ¼ 2, ... , r (4 : 5)

... , u k 1 , w

[ R

n w T Cw ,

¼ k ¼ n 1, ... , nr þ 1( : 6)

min
kwk 2 ¼ 1, w?u n , u n 1 , ... , u kþ 1 , w [ R

and the constrained maximum and minimum are attained by the unit 2-norm eigen-
vectors u k associated with l k which are unique up to a sign.
Note that when l r . l rþ 1 or l n 2 r . l n 2 rþ 1 , the following global constrained
maximizations or minimizations (denoted subspace criterion )

X

r

Tr( W T CW ) ¼ max
W T W¼I r

w k Cw k max

W T W¼I r

k¼ 1

or

X

r

Tr( W T CW ) ¼ min
W T W¼I r

w k Cw k min

W T W¼I r

(4 : 7)

k¼ 1

where W ¼ [ w 1 , ... , w r ] is an arbitrary n r matrix, have four solutions (see e.g. [70]
and Exercise 4.6), W ¼ [ u 1 , ... , u r ] Q or W ¼ [ u n 2 rþ 1 , ... , u n ] Q respectively,