GeMCA EXIN THEORY - Neural-based Orthogonal Data Fitting: The EXIN Neural Networks - page 208

Hence, ( ∞ , e n + 1 ) is an eigenpair with an infinite simple eigenvalue. The same
type of analysis as before estimates the remaining eigenpairs. From eq. (6.2), it
follows that

A T AA T b
b T Ab T b

u 1
u n + 1

I n u 1

u n + 1

0

= 2 γ

0 T 0

A T Au 1

A T bu n + 1 + = 2 γ u 1

⇒

b T Au 1 + b T bu n + 1 = 0

A T Au 1 − A T b b T b − 1 b T Au 1 = 2 γ u 1
u n + 1 =− b T b − 1 b T Au 1

⇒

A T I m − b b T b − 1 b T Au 1

⇒ = 2 γ u 1 (6.13)

Equation (6.13) can be solved for u 1 = 0(i.e., u parallel to e n + 1 ), as seen previ-
ously. To estimate the other eigenpairs, it can be derived from eq. (6.13) :

A T P b Au 1 = 2 γ u 1

I m − b b T b − 1 b T is the symmetric projection matrix ( P ⊥ 2

where P b

=
P b ) that projects the column space of A into the orthogonal complement of b .
Hence,

=

b

A T P ⊥ b Au 1 = 2 γ u 1 ⇒ A T P ⊥ b P b Au 1 = 2 γ u 1

⇒ P b A T P b A u 1 = 2 γ u 1 (6.14)

It can be concluded that if the singular values and right singular vectors of
matrix P b A are given by π i and v i , respectively, for i = 1, ... , n (the singular
values are sorted in decreasing order), the required eigenpairs of the DLS case

are given by the n couples π i

where ρ i =− b T b − 1 b T A v i . It can
be deduced that the first n components of the DLS eigenvectors are parallel to
the v i singular vectors. In particular, the right singular vector associated with
the smallest singular value of P b A corresponds to the DLS solution (see [51]).
Indeed, it corresponds to the last eigenpair, scaled by ρ min =− 1.

2 , v i

ρ i

6.1.2 General Case

In the case 0 <ζ < 1, R can be diagonalized by a D -orthogonal transformation

[178]. Indeed,

Z T Z

D = (6.15)