INTRODUCTION TO THE TLS EXIN NEURON - Neural-based Orthogonal Data Fitting: The EXIN Neural Networks - page 139

The asymptotes of g (γ ) are given by the squared singular values σ 2 = λ i of

i

2
i , just respecting the
interlacing theorem [eq. (1.16)], as illustrated in Figure 4.15 .

σ

thedatamatrix A . They are positioned among the zeros

4.8 FIRST CONSIDERATIONS ON THE TLS STABILITY ANALYSIS

By using the theory of Section 2.5 and Figure 4.1, two remarks on the stability
of TLS iterative algorithms can be deduced.

Remark 93 (TLS Stability) The critical directions intersect the TLS hyper-
plane in points, which are critical for the TLS problem with the same typology
of the corresponding critical direction. The cones become volumes of attrac-
tion and repulsion for each saddle. The intersections of the hypervalleys and
hypercrests with the TLS hyperplane give ( N − 2 ) -dimensional planes, which are,
respectively, made up of minima or maxima with respect to the direction of their
normal. In the three-dimensional weight space, the critical directions become crit-
ical points, and the TLS plane is divided into a zone of attraction and a zone of
repulsion (shown only partially in the figure). The straight line passing through
the saddle and the maximum is a crest along the minimum direction. The straight
line passing through the saddle and the minimum is a valley along the maximum
direction. The straight line passing through the minimum and the maximum is
a crest along the saddle direction, except for the intersection with the zone of
repulsion, which is a valley along the saddle direction.

Remark 94 (TLS Barrier) The hypercrest along the minimum direction is not
a barrier ( limit of the basin of attraction ) for the minimum. Indeed, if the initial
conditions for the learning law are chosen in either of the half-spaces delimited
by the hypercrest, the weight vector always converges to the minimum direction,
orienting according to the half-space containing the initial conditions. The same
reasoning is no more valid in the TLS case ( i.e., the crest is a barrier ) because
there is only one minimum point and therefore it is mandatory to choose the
initial conditions in the half-space containing this point. Later we demonstrate
the very important property peculiar to TLS EXIN alone that the TLS hyperplane
origin ( intersection of the TLS hyperplane with the axis corresponding to the last
coordinate of

1 ) is in the same half space as the minimum. This crest is not a
unique TLS barrier; it must be considered in conjunction with the saddle cones.

n +