TOTAL LEAST SQUARES PROBLEMS - Neural-based Orthogonal Data Fitting: The EXIN Neural Networks - page 13

Theorem 25 (Nongeneric Unidimensional TLS Solution) Let ( 1.3 ) be the
SVD of [ A ; b ] , and assume that v n + 1, j = 0 for j = p + 1, ... , n + 1 ,p ≤ n. If

σ p − 1 > σ p and v n + 1, p = 0 ,then

[ A ;

b ] = U ˆ

V T

ˆ

where = diag (σ 1 , ... , σ p − 1 ,0, σ p + 1 , ... , σ n + 1 ) (1.41)

with corresponding nongeneric TLS correction matrix

A ; b ] = σ p u p v

T
p

[ (1.42)

solves the nongeneric TLS problem ( 1.39 ) and

v n + 1, p v 1, p , ... , v n , p T

1

x =− (1.43)

exists and is the unique solution to Ax =

b.

Proof. See [98, p. 72]).

If v n + 1, n + 1 = 0 ∧ v n + 1, n = 0 ∧ σ n − 1 > σ n ,then ( 1.43 ) becomes

Corollary 26

x
− 1

v n
v n + 1, n

=− (1.44)

Theorem 27 (Closed-Form Nongeneric TLS Solution) Let ( 1.2 )[ respec-
tively, ( 1.3 )] be the SVD of A ( respectively, [ A ; b ] ) , and assume that v n + 1, j = 0
for j = p + 1, ... , n + 1 ,p ≤ n. If σ p − 1 > σ p and v n + 1, p = 0 , the nongeneric
TLS solution is

A T A − σ

p I n − 1

2

A T b x = (1.45)

Proof. See [98, p. 74].

Remark 28 The nongeneric TLS algorithm must identify the close-to-nongeneric
or nongeneric situation before applying the corresponding formula. In the follow-
ing it will be shown that the TLS EXIN neuron solves both generic and nongeneric
TLS problems without changing its learning law ( i.e., automatically ) .