GENERALIZATION OF LINEAR REGRESSION PROBLEMS - Neural-based Orthogonal Data Fitting: The EXIN Neural Networks - page 161

s1

min

max

Figure 5.3 Barrier (frontier) for a two-dimensional benchmark; the focal axes of the equilevel
hyperbolas are evidentiated.

where x c (γ ) is the center of the corresponding equilevel curve, the barrier locus
is given by the two-dimensional vector

g

ζ )
λ 1 − 2 γζ v 1 + x c (γ

(γ ,

x bar =± , ζ) (5.94)

v 1 is the first column of V (i.e., the principal component of A T A ). Hence,
the barrier follows the

where

v 1 direction. The subfamily of hyperbolas for λ / 2 ζ ≤

2

γ ≤ γ saddle is not relevant from the convergence point of view because it will be

demonstrated in the following that the relative position between the saddle and
the minimum implies that only the first barrier is valid. This analysis is valid for
every gradient flow of the energy cost considered.

5.3.7.2 Higher-Dimensional Case For , the family

is composed of two subfamilies of hyperboloids according to their level with
respect to the saddle level. Extending these reasonings, focus on the subfamily
for

λ j + 1 / 2 ζ ≤ γ ≤ λ j / 2 ζ

γ ≤ γ ≤ λ / 2 ζ , made up of hypersurfaces of the form

saddle j

n

z i
k i = 1

(5.95)

i = 1