GeMCA EXIN THEORY - Neural-based Orthogonal Data Fitting: The EXIN Neural Networks - page 221

It is positive and greater than or equal to 0.5. It is not greater than 1.
For ζ → 1 (DLS) the lower bound l3 and the upper bound u 3tendtofinite
values. It follows that for ζ = 1,

≤ σ 2

2

σ

i − i

i

2 ≤ α i

(6.70)

2

which is a bound for the DLS saddle critical values.

6.5.1.1 Analysis of the Strictest Inequalities for Saddles In this section
the interval between the bounds of the previous strictest inequalities is studied
for ζ − ≤ ζ ≤ ζ + and, above all, in a neighborhood of ζ = 0 . 5 (TLS).
Define ε such that

ζ = 0 . 5 − 2 (6.71)

The distance between the bounds l 1and u 1 (ζ − ≤ ζ ≤ 0 . 5 ) is given by

2
i

2 2 2

i

= σ

σ σ σ

i i

−

− ζ ) = − ε −

= (ε) (6.72)

2 ζ 2 ( 1 1 1 + ε

2

It can be seen that ( 0 ) = 0. Indeed, for TLS, α i = σ

i . As a consequence,

2

represents the absolute error in estimating α i as σ

i . It follows that

2 ε
1 − ε

2
i

=

2 σ

(6.73)

The same reasoning can be repeated for the distance between the bounds l 3

and u 2 (in this case ε ≤ 0). Hence, in the general case ζ − ≤ ζ ≤ ζ + it follows
that

| ε |
1 − ε

2

2
i

= σ

(6.74)

2

Around ζ = 0 . 5 (TLS), eq. (6.74) is approximated by

i 1 + ε

+ o ε

2 2 4

= 2 | ε | σ (6.75)

Hence, as a first approximation, the distance increases linearly with ζ .Also,

2

i . It means that the increase is larger for larger
saddles (i.e., saddles close to the maximum eigenvalue). As a consequence, the
strictest inequalities are the inequalities concerning saddles near the minimum.
Now consider the derivative of with regard to ε :

increases proportionally to σ

2

d d 1 + ε

2

= 2

2 2 σ

(6.76)

1 i

ε

− ε