GENERALIZATION OF LINEAR REGRESSION PROBLEMS - Neural-based Orthogonal Data Fitting: The EXIN Neural Networks - page 196

asymptotes coincident with the γ = 0 axis. It implies that γ min = 0 ∀ ζ . Thus,
eq. (5.51) yields the coincidence of all GeTLS solutions. The corresponding
parameter t min = 0 ∀ ζ , and therefore this common solution is the minimal
L 2 norm solution (see Proposition 105 and Theorem 108; the point at t = 0
is the origin of the solution locus and is nearest the origin of the reference
system x ).

Corollary 131 (Neural Solutions) For the underdetermined unidimensional
problem, the GeTLS EXIN neuron yields the same solution independent of its
parameter

ζ .

The following simulation deals with the benchmark problem taken from
[22, Ex. 1]:

2 − 1403 1 2

1
− 4

x

51 − 312 0 = (5.146)

1 − 21 − 5 − 1 4

The minimal L 2 norm solution, obtained by using MATLAB, is

x = x = x = [0 . 088, 0 . 108, 0 . 273, 0 . 505, 0 . 383, − 0 . 310] T (5.147)

The only comparable results in [22, Ex. 1] have a settling time of about
250 ns for the OLS problem (clock frequency

=

100 MHz, which implies one
iteration every 10 ns). Table 5.2 shows the results for GeTLS EXIN. The initial
conditions are null; to avoid the divergence, the accelerated DLS neurons have
ζ = 99. Every batch is composed of the coefficients of the three equations.

The learning rate for the nonaccelerated GeTLS EXIN is

0 .

t γ . Note the
acceleration of the SCG and, above all, of the BFGS methods (recall Remark

α( t ) = α / 0

Table 5.2 Underdetermined linear system benchmark a
ζ α 0 γ Time (ns)

GeTLS EXIN seq. 0 0.09 0.51 200

GeTLS EXIN seq. 0.5 0.09 0.51 750

GeTLS EXIN seq. hyp. 0.09 0.51 1100

GeTLS EXIN batch 0 0.1 0.8 300

GeTLS EXIN batch hyp. 0.1 0.8 1650

SCG GeTLS EXIN 0 — — 270

SCG GeTLS EXIN 0.5 — — 270

SCG GeTLS EXIN 0.99 — — 330

BFGS GeTLS EXIN 0 — — 200

BFGS GeTLS EXIN 0.5 — — 200

BFGS GeTLS EXIN 0.99 — — 300

a seq., sequential; hyp., hyperbolic scheduling.