GENERALIZATION OF LINEAR REGRESSION PROBLEMS - Neural-based Orthogonal Data Fitting: The EXIN Neural Networks - page 174

6

saddle
repulsion cone

5

4

asymptote
barrier

saddle
repulsion cone

3

2

sl

1

min

0

max

− 1

max locus tangent
asymptote barrier

− 2

hypercrest

− 3

− 2 − 1.5 − 1 − 0.5 0

x1

.51 .52

Figure 5.10 Initial conditions map for the generic TLS benchmark problem. Green repre-
sents the initial conditions by which a gradient flow algorithm converges (i.e., the domain
of convergence). Red indicates divergence. The dark blue triangle has the critical points
as vertices. The dark blue line represents the asymptote or barrier z 1 .( See insert for color
representation of the figure .)

and repulsion of its own critical points (the same shape as for the TLS case).
In Chapter 6 the GeMCA theory is introduced: It transforms the GeTLS EXIN
problem in a MCA EXIN problem, thus allowing us to use the MCA stability
analysis.
For ζ = 0 (OLS), the domain of convergence is the entire x space. To increase
ζ , the domain of convergence becomes smaller and smaller. Indeed, in every
saddle locus, the saddles tend to the next asymptote ( t tends to the first superior
eigenvalue) and this decreases the nearly triangular convergence area between
the saddle and the corresponding barrier for every plane z n z i . In the maximum
locus, the maximum is far from the vertical asymptote (for the DLS problem it
coincides with the origin) and then the maximum locus tangent rotates toward
the solution locus, thus decreasing the domain of convergence.

5.3.12 Skew Distances and GeTLS

Total least squares correspond to orthogonal regression because it minimizes the
orthogonal distances (see Section 1.5.1 for TLS viewed from the row space and
Figure 1.1). What is the meaning of the GeTLS EXIN error cost? Can we still
speak of distances, perhaps skew, between the rows r i of the augmented data
matrix [ A ; b ] and the row space

a
b

a ∈

n , b ∈ , b = x T a

R r ( [ A ; b ] ) =