MCA EXIN NEURON - Neural-based Orthogonal Data Fitting: The EXIN Neural Networks - page 26

• Boundedness .As u ranges over all nonzero vectors, r ( u ) fills a region in the
complex plane which is called the field of values (also, the numerical range )
of C . This region is closed, bounded, and convex. If C = C ∗ ( self-adjoint
matrix ), the field of values is the real interval bounded by the extreme
eigenvalues [ λ min , λ max ].
• Orthogonality :

u ⊥ ( C − r ( u ) I ) u (2.4)

• Minimal residual : ∀ u = 0 ∧∀ scalar µ ,

( C − r ( u ) I ) u ≤ ( C − µ I ) u (2.5)

2.1.1 Critical Points

The following theorem gives, as a special case, the stationarity properties of RQ.

Theorem 42 (Courant-Fischer) Let C be a Hermitian n-dimensional matrix
with eigenvalues

λ n ≤ λ n − 1 ≤···≤ λ 1

Then ( Fischer-Poincare )

λ k =

min
V n − k + 1

max

V n − k + 1 r ( u , C )

(2.6)

u ∈

and ( Courant-Weyl )

λ k = max

S n − k

min
v ⊥ S n − k r (v , C )

(2.7)

, n ) ,whereV n − k + 1

( S n − k ( k = ... ) ranges over all subspaces of dimension n −

1,

k +

( n − k ) .

1

Proposition 43 (Stationarity) Let C be a real symmetric n-dimensional matrix
with eigenvalues λ n ≤ λ n − 1 ≤···≤ λ 1 and corresponding unit eigenvectors
z 1 , z 2 , ... , z n .Then

λ = max r ( u , C ) (2.8)

1

λ n = min r ( u , C ) (2.9)

More generally, the critical points and critical values of r ( u , C ) are the eigen-
vectors and eigenvalues for C ( for an alternative proof, see Section 2.5 ) .