INTRODUCTION TO THE TLS EXIN NEURON - Neural-based Orthogonal Data Fitting: The EXIN Neural Networks - page 118

must be normalized to have the last component of the solution equal to − 1. This
way of finding the TLS solution has two fundamental problems:

1. An additional operation of division must be made because of the normal-
ization of the output.
2. All the problems on the behavior of the MCA neurons cited above are valid
here.

If the weight vector is constrained to lie on the TLS hyperplane (i.e., losing
a degree of freedom), it can be argued that all the convergence problems, which
are the consequence of the degeneracy property of the Rayleigh quotient, are
no longer valid, because, on the TLS hyperplane , the only critical points are
the intersections with the critical straight lines, and therefore they are isolated
(it implies the global asymptotic stability of the solution on the hyperplane plus
the eventual solution ∞ ). This reasoning, illustrated in Figure 4.1 for the three-

dimensional -space, derives from Section 2.5 and will be extended further in

this and the next two chapters. Furthermore, no normalization operation would
be used.
The following MCA EXIN learning rule solves the TLS problem:

α( t ) y ( t ) y ( t )( t )

( t + 1 ) = ( t ) − ξ( t ) − (4.3)

T

( t )( t )

T

( t )( t )

x 2

Ψ 2

attraction zone

TLS hyperplane

saddle

min

x 1

− 1

e 2

e 3

Ψ 1

0

e 1

Ψ 3

max

Figure 4.1 Cost landscape stability analysis for MCA EXIN and TLS EXIN (TLS hyperplane
in light gray) in

3 space.