GeMCA EXIN THEORY - Neural-based Orthogonal Data Fitting: The EXIN Neural Networks - page 212

The remaining procedure is the same as before:

det A T A
2 n

det ( K ) ≥

α n + 1

(6.35)

n

ζ

det A T A
2 n

2
2

det A T A ≥

b ⊥

α n + 1 (6.36)

2 n + 1 ζ

n

( 1 − ζ ) ζ

n

2
2
2 ( 1 − ζ ) =

b ⊥ γ OLS

1 − ζ

α n + 1 ≤ (6.37)

Considering also that

σ 2

≤ σ 2

i
2

i − 1
2

≤ α i ∀ i = 2, ... , n (6.38)

ζ ζ

it can be deduced that for tending to zero (OLS), all the eigenvalues of matrix

K but the smallest go to infinity. Inequality (6.37) shows that the smallest eigen-
value in the OLS case cannot exceed the finite upper bound γ OLS . This study
confirms the analysis in Section 6.1.1.1. Also, if b ⊥ 2 = 0, which means that
b ∈ range ( A ) , the upper bound (6.37) implies that α n + 1 = 0 (a compatible system
of equations).
By looking again at the matrix K [see eq. (6.17)] and identifying the principal
(scalar) submatrix b T b / 2 ( 1 − ζ ) , by means of a consequence of the interlacing
theorem [199, Prob. 2, p. 225], we obtain

ζ

b T b

α n + 1 ≤

− ζ ) ≤ α 1

(6.39)

2 ( 1

Hence, defining

0
OLS

1

2 b T b

γ = E OLS (

) =

0

it follows that

0
OLS

γ OLS
1

γ

α n + 1 ≤ ≤ ≤ α 1 (6.40)

− ζ 1 − ζ

0
OLS

It can be concluded that γ /( 1 − ζ) is a better lower bound for α 1 , while

γ /( 1 − ζ) is a better upper bound for α 1 . OLS n +

6.2.1 Note on the Taylor Approximation of the Bounds of the
Eigenvalues of K

The bounds for the eigenvalues of matrix K expressed by (6.38) can be approx-
imated for ζ ∈ [0, 1] by a Taylor expansion of the first order: