GENERALIZATION OF LINEAR REGRESSION PROBLEMS - Neural-based Orthogonal Data Fitting: The EXIN Neural Networks - page 185

5

4

3

2

1

0

− 1

− 2

− 3

− 4

− 5

1 1.5 2

− 2 − 1.5 − 1 − 0.5 0

w1

0.5

Figure 5.19 Evolution map of the sequential TLS EXIN for the second nongeneric TLS
benchmark problem. The initial conditions for the converging trajectories are in red; the
others are in black. The horizontal straight line passing through the red points contains the
saddle/solution locus. The dark blue thick vertical line is the divergence line. ( See insert for
color representation of the figure .)

5.5.1 DLS Problem

From the point of view of the neural approaches and, more generally, of the
gradient flows, the DLS problem ( ζ = 1) is the most difficult (for b = 0 ) for the
following reasons:
• The domain of convergence is smaller, as shown in Section 5.4.
• Extending the definition of motor gap (Definition 38) to the height distance
between a saddle i (height

2

σ

i ) and the corresponding immediately superior

infinite (height σ 2
i

= λ i − 1 ), the motor gap represents the gap of energy to
descend to the inferior critical point, and in this case it is the smallest, just
decelerating the gradient flow.
• All the infinite heights (different directions) are the lowest with respect to
the other GeTLS problems.
• As shown in Proposition 105, the DLS solution is the farthest GeTLS solu-
tion from the origin.
• As shown in Remark 98, the DLS error cost is not defined for null initial
conditions, and every gradient flow algorithm diverges for this choice of
initial conditions. Hence, there is no universal choice that guarantees the
convergence as for the other GeTLS EXIN neurons.

− 1

5.5.2 Direct Methods

Two direct methods are presented in [35]: the weighted TLS for α →∞ ,intro-
duced in Section 5.1.1, and the Householder transformation.