GeMCA EXIN THEORY - Neural-based Orthogonal Data Fitting: The EXIN Neural Networks - page 211

α 1 ≥ σ 1
2 ζ

(6.28)

where α i is the i th eigenvalue of matrix K [see eq.(6.18)] and σ 2 i is the i th
eigenvalue of matrix A T A (eigenvalues are sorted in decreasing order). Hence,

det A T A
2 n

n n n

σ 2
i
2 ζ

1
2 n

1 σ 2 1 α i + 1

≤ = = (6.29)

i

n n

ζ ζ

i = i = 1 i =

Hence,

det A T A
2 n

det ( K ) ≤ α 1 (6.30)

n

ζ

By using eq. (6.26) it follows that

det A T A
2 n

2
2

det A T A ≤

b ⊥

α (6.31)

1

2 n + 1 ζ

n

( 1 − ζ ) ζ

n

that is [recalling eq. (6.17)],

2
2
2 ( 1 − ζ ) =

b ⊥ γ OLS

1 − ζ

α 1 ≥ (6.32)

which is a lower bound for the eigenvector associated with the largest eigenvalue
of K . The inequality (6.27) shows that all eigenvalues of K but the largest are
bounded by σ 1 / 2 ζ , which varies from infinity (OLS), just allowing the n infinite
eigenvalues, as seen before, to σ 1 / 2for ζ = 1 (DLS). In the latter case, only
one eigenvalue of K is allowed to increase toward infinity (i.e., α 1 ). This is also
apparent from the lower bound (6.32), which tends to infinity as ζ tends to 1 [the
upper bound (6.28) tells nothing about the infinity for α 1 ]. This analysis confirms
what has been said in Section 6.1.1.2.
Another manipulation of the formulas above yields a novel inequality. Inequal-
ities (6.27) and (6.28) can be rewritten as

≥ σ 2

i
2 ζ

α i ∀ i = ...

1, , n (6.33)

which imply that

det A T A
2 n

n n n

σ 2

1
2 n

i
2

1 σ 2 1 α i

≥ = = (6.34)

i

ζ ζ

n

ζ

n

i = i = 1 i =