GENERALIZATION OF LINEAR REGRESSION PROBLEMS - Neural-based Orthogonal Data Fitting: The EXIN Neural Networks - page 165

j < i

Z j

→ l j − t

sol locus

→ l i −

t

inf saddles locus

t = 0

Z i

sup 1 saddles locus

t → −∞

t → +∞

y i

→ l i +

t

t = l 1

max locus

sup 2 saddles locus

→ l j + t

Figure 5.4 Critical loci in the plane z i z j .( See insert for color representation of the figure.)

where

diag

λ λ λ 1

λ 1 − 2 γζ

2
λ 2 − 2 γζ

n
λ n − 2 γζ

= , , ... ,

has only positive elements in the solution locus. This confirms the reasoning that
yields Proposition 105 (i.e., the GeTLS solutions follow the OLS solution in the
locus and their distance on the curve is proportional to ζ ). These two reasonings
introduce the following theorem by explaining its first part.

Theorem 108 (Loci Relative Positions) The value of the parameter t of a
GeTLS solution ( position in the solution locus with respect to the origin, which
corresponds to the OLS solution ) is proportional to the value of ζ . In particular,
it means that t DLS ≥ t TLS ≥ t OLS = 0 . In every saddle locus and in the maximum
locus, the critical points have a position t on the corresponding locus proportional
to the parameter ζ of the GeTLS problem.

Proof. The entire theorem can be demonstrated by the help of the graphical
analysis of the zeros of g (γ , ζ ) . These zeros are given by the intersections
between h ( t , ζ) and l ( t , ζ) ,where

b T b − ζ − 1

h ( t , ζ) = + t ( 1 ) (5.109)