MCA EXIN NEURON - Neural-based Orthogonal Data Fitting: The EXIN Neural Networks - page 53

enters the MC direction faster, starting from null or small initial conditions.
Furthermore, starting the divergence (it will be shown later that all three
laws diverge) implies large moduli; in this case MCA EXIN has the lowest
weight modulus increment. In a nutshell, once the MCA direction is reached,
the negative power for the weight modulus in g acts as an inertia , unlike
LUO and OJAn, which are recursively amplified. 8
•sin 2 2 ϑ x w is a positive function with four peaks in the interval ( − π , π ].
This is one of the possible interpretations of the oscillatory behavior of the
weight modulus, which will be shown in the simulations.
• h and t depend only on the input noisy signal and on the relative positions
among the data and the weight vector, so they are impossible to control.

Summarizing: For these three neurons, the property of constant modulus (2.27)
is not correct. Notice that in [124] the choice of a unit starting weight vector in
the simulations for LUO does not reveal the increase in the modulus.

2.6.1.2 OJA, OJA + ,andFENG Following the same analysis as above, it

holds that

+ 2 α ( t ) + O α

( t )

2
2

2
2

2

w ( t + 1 ) = w ( t )

(2.102)

where

!

( t ) w ( t ) − 1

2
2

y 2 for OJA

y 2

2 w ( t )

− 1 for OJA +

2 2

2

=

( t ) − w ( t )

2 1

)

"

2

y 2 w ( t ) − ( t for FENG

For these neurons the property (2.98) is no longer valid and eq. (2.102) then
depends on α ( t ) . This implies a larger increment than for MCA EXIN, OJAn, and
LUO (remember that α is in general less than 1). For OJA, the sign of depends
only on the value of the squared modulus; hence, the increment in the squared
modulus is always decreasing or increasing according to the initial conditions.
This fact is no longer valid for OJA + and FENG, where an appropriate value of
y 2 can change the sign of

.

T x

is pointing in the MC
direction, whose projection on the data is as small as possible. In this case, for
FENG it holds that

Near convergence, y = w <ε with ε low, because w

+ O α

( t )

+ 2 α (

t )

2
2

2
2

2

w ( t + 1 ) −→ w ( t )

(2.103)

λ n

where eq. (2.31) has been used. If λ n is low, the increase in the squared modulus

can be significant. On the contrary, if λ n < 1, the OJA + weight modulus remains

2
2

constant ( w ( t ) → 1 at convergence).

8 As is made clear later, a too strong divergence prevents the learning law from having a reliable
stop criterion.