MCA EXIN NEURON - Neural-based Orthogonal Data Fitting: The EXIN Neural Networks - page 41

Consider now the case n > 3. Define the gap between the eigenvalue cor-
responding to the critical direction i 0 and the eigenvalue corresponding to the
critical direction j = i 0 as g ji 0 = λ j − λ i 0 . Then, in every hyperplane perpendic-
ular to the direction i 0 , the locus of points with the same energy as z i 0 (i.e.,
E i 0 = 0) is given by

−
j

+
j

g ji 0 ε g ji 0 ε

2
j

2
j

= 0 (2.64)

> i 0 < i 0

This homogeneous equation of the second degree specifies an ( n − 1 ) -
dimensional cone whose vertex (center) is the intersection with the i 0 direction.
If all g ji 0 's are of the same sign (i.e., the minimum and maximum directions),
then the cone is imaginary (only the center is a real point). The saddle directions
have g ji 0 's of different sign. In these cases the cone is called a real cone in the
sense that it has real points besides the center. The saddles are of the following
kind:

• Saddle with index i 0 ≥ ( n + 1 )/ 2: a saddle closer to the minimum than
to the maximum. The hyperplane ε n = constant = 0 intersects the cone
(2.64) in an ( n − 2 ) -dimensional ellipsoid if i 0 = n − 1 (i.e., the saddle
closest to the minimum). It means that the minimum direction is the axis
of the cone of repulsion of this saddle. The same hyperplane intersects the
cone (2.64) in a hyperboloid if i 0 < n − 1 (i.e., the inner saddles) (this
case is not possible for n ≤ 4). Every saddle repulsion cone intersects with
the i 0 direction as center and consists of all possible straight lines passing
through the center and through points of the surface in which it intersects
the hyperplane

ε n = =

( i 0 −

const 0. The 1 ) -dimensional plane

ε i 0 + 1 = const i 0 + 1 , ε i 0 + 2 = const i 0 + 2 , ... , ε n = const n (2.65)

intersects the cone (2.64) in an ( i 0 − 2 ) -dimensional ellipsoid. It follows that
the cone of repulsion contains all linear combinations of the eigenvectors
with smaller eigenvalues ( λ j , j > i 0 ). Every reasoning can be repeated, but
with respect to the j < i 0 directions, to describe the corresponding cones of

attraction.
• Saddle with index i 0 <( n + 1 )/ 2: a saddle closer to the maximum than
to the minimum. The same kind of reasoning also applies here, but with
respect to the maximum; for example, the saddle closest to the maximum
has a cone of attraction with the axis in the maximum direction and a cone
of repulsion that contains all linear combinations of the eigenvectors with
smaller eigenvalues ( λ j , j > i 0 ).

In every hyperplane perpendicular to the direction i 0 , the cones are always the
same, but the values of the error cost E are scaled by the denominator of eq.
(2.60), which is a function of the weight component parallel to the i 0 direction.