VARIANTS OF THE MCA EXIN NEURON - Neural-based Orthogonal Data Fitting: The EXIN Neural Networks - page 95

The second simulation deals with the benchmark problem in [195], already
solved in Section 2.10. As seen before, at first the points are preprocessed. Then
the NMCA EXIN and NOJA + learnings are compared (at each step the input
vectors are picked up from the training set with equal probability). Both neurons
have the same initial conditions [0 . 05, 0 . 05] T and the same learning rate used

α( t ) . .

in [195]: that is, start 0025 at the first 500

iterations, and then keep it constant. In the end, the estimated solution must
be renormalized. Table 3.1 shows the results (averaged in the interval between
iteration 4500 and iteration 5000) obtained for

at 0 01, linearly reduce it to 0

2

5 and without outliers.
They are compared with the singular value decomposition (SVD)-based solution
and the LS solution. It can be concluded that the robust neural solutions have
the same accuracy as that of the corresponding linear solutions. Second, another
training set is created by using the same 500 points, but without additional noise.
Then two strong outliers at the points x = 22, y = 18 and x = 11 . 8, y = 9 . 8are
added and fed to the neuron every 100 iterations. Table 3.2 shows the results:
NOJA + is slightly more accurate than NMCA EXIN. This example is of no
practical interest because the assumption of noiseless points is too strong. The
last experiment uses the same training set, but with different values of additional
Gaussian noise and the same two strong outliers. The learning rate and the initial
conditions are always the same. The results in Table 3.3 are averaged over 10
experiments and in the interval between iteration 4500 and iteration 5000. As
the noise in the data increases, the NMCA EXIN neuron gives more and more
accurate results with respect to the NOJA + neuron. As in the first simulation,

σ = 0 .

Table 3.1 Line fitting with Gaussian noise
a 1 a 2 ε w

True values 0 . 25 0 . 5

LS 0 . 233 0 . 604 0 . 105

SVD 0 . 246 0 . 511 0 . 011

OJA + 0 . 246 0 . 510 0 . 012

NOJA + 0 . 248 0 . 506 0 . 006

NMCA EXIN 0 . 249 0 . 498 0

Table 3.2 Line fitting with strong outliers only

a 1 a 2 ε w

True values 0 . 25 0 . 5

LS 0 . 216 0 . 583 0 . 089

SVD 0 . 232 0 . 532 0 . 036

OJA + 0 . 232 0 . 532 0 . 036

NOJA + 0 . 244 0 . 492 0 . 010

NMCA EXIN 0 . 241 0 . 490 0 . 013