GENERALIZATION OF LINEAR REGRESSION PROBLEMS - Neural-based Orthogonal Data Fitting: The EXIN Neural Networks - page 172

The intersection of the hypercrest with the TLS hyperplane is given by the solu-
tion of the following system:

n + 1

1 v

= 0

i , n + 1 i

i =

n + 1 =− 1 (5.117)

Hence, the equation of the ( n − 1)-dimensional plane in the TLS hyperplane ( x
space) is

n

v i , n + 1
v n + 1, n + 1 x i

= 1 (5.118)

i = 1

which can be expressed as

n

1
− 1 / x i ,min x i

= 1 (5.119)

i = 1

Thus, the intercepts of this ( n − 1)-dimensional plane for every x i axis of the

TLS hyperplane are of opposite sign with respect to the x system origin. This
demonstrates the following theorem.

Theorem 113 (Origin and Saddle-Maximum Barrier) The origin of the TLS
hyperplane is always between the saddle-maximum hypercrest projection and the
TLS solution.

Figure 5.9 shows the case for n = 2 and the change in the position of the pro-
jection as a function of the TLS minimum position. All the analysis of this section,
Theorem 111, and all the observations about the TLS convergence demonstrate
the following fundamental theorem.

Theorem 114 (Origin and TLS Domain of Convergence) The TLS origin
belongs to the TLS domain of convergence.

As an example, Figures 5.10 and 5.11 show the domain of convergence of
the generic TLS benchmark problem given by the system (5.57). Figure 5.10
illustrates the domain of convergence just showing a certain number of initial
conditions for the sequential TLS EXIN neuron (it is also valid for every gradient
flow of the TLS energy): The initial conditions for which the neuron converges
are green. The dark blue straight line represents the asymptote/barrier z 1 .The
bottom part of the frontier is given by the asymptote, because the maximum is
near to it. For increasing x 2 , the frontier is given by the action of the saddle-
maximum line and the repulsion area of the saddle. The latter is responsible for