GENERALIZATION OF LINEAR REGRESSION PROBLEMS - Neural-based Orthogonal Data Fitting: The EXIN Neural Networks - page 183

Figure 5.18 Plot of the weights of SCG and BFGS TLS EXIN for the first nongeneric TLS
benchmark problem. The initial conditions are null. The two learning laws stop at different
iterations. ( See insert for color representation of the figure .)

the asymptote z 2 and then diverge in the v 2 direction. Figure 5.17 shows the
dynamic behavior of TLS EXIN and TLS GAO for null initial conditions: The
two weights w 1 and w 2 are always coincident because the weights trajectory is the
bisector of the first quadrant. TLS EXIN is faster and more accurate. Figure 5.18
shows the behavior of SCG and BFGS TLS EXIN for null initial conditions (the
weight components are coincident). They are faster than the other two neurons
(1 epoch = 3 iterations) and, as usual, BFGS is best.
The second benchmark set of equations [93, Ex. 1] is

x 1
x 2

10
010 − 4
00

1
0
1

(5.130)

≈

1, 0] T

Here, v = [0, − and is parallel to the TLS hyperplane and to its axis x 2 .

3

Note that v = v = 0. The solution (using the SVD) is

n + 1, n + 1 3,3

2
√ 5

1 ,0 T

x = (5.131)

−

being x T 1 T

⊥ v 3 . SCG TLS EXIN and BFGS TLS EXIN, with null ini-
tial conditions, reach the solution (precision = 10 − 15 ), in, respectively, six and
four epochs. To explain problems regarding the close-to-nongeneric case, perturb

;−