MCA EXIN NEURON - Neural-based Orthogonal Data Fitting: The EXIN Neural Networks - page 80

LUO

OJAn

OJA

MCA EXIN

MCA EXIN

Figure 2.16 Line fitting for noise variance equal to 0.5: plot of the index parameter
(expressed in decibels). The values are averaged using a temporal mask with width equal to
500 iterations, except in the first 500 iterations in which the mask is equal to the maximum
number of iterations. ( See insert for color representation of the figure .)

1. About OJA, there is sudden divergence. Indeed, from (2.119), t ∞ decreases
for increasing p 0 . It has been noticed that for increasing n , the sudden divergence
is anticipated. About FENG, there is instability divergence. Indeed, as shown in
(2.171), the fluctuations depend only on the modulus and the data inputs. The
modulus is large and remains so because of the too large oscillations, which
prevent the weights from approaching the MC direction. As a consequence, this
generates more and more instability until the finite time divergence. Obviously,
for increasing n , the divergence is anticipated. About LUO, two explanations
are possible: sudden and instability divergence. However, in these experiments
the divergence is of the instability type, because it is accompanied by very large
oscillations and certainly anticipates the sudden divergence. Indeed, the weight
modulus increases very much [see (2.100)], so it decreases the dynamic stability
interval st accordingly [see (2.159)], generating large oscillations which can
generate the divergence. However, the fact of having a dynamic instability sub-
space which is the contrary of the OJAs (see Figures 2.12 and 2.13) and the
fact that the weight modulus increases depends only on α

2 (see Section 2.6.1)
certainly explains the fact that LUO diverges for higher n than OJA. Obviously,
for increasing n , the LUO divergence is anticipated. Figures 2.20 to 2.21 show
some of these experiments for, respectively, n = 5, 9, 11, and 100. Figure 2.18
shows the large fluctuations of FENG and that MCA EXIN has the fastest tran-
sient. Figure 2.19 shows the very good accuracy of MCA EXIN, the sudden
divergence of OJA, and the unacceptable accuracy of FENG. Figure 2.20 con-
firms that FENG undergoes the instability divergence (notice the peaks before the