TOTAL LEAST SQUARES PROBLEMS - Neural-based Orthogonal Data Fitting: The EXIN Neural Networks - page 12

= 0 ⇒ b ⊥ u with u ∈ U (σ j ) v n + 1, j (1.33)

= u with u ∈ U (σ j )

v n + 1, j = 0 ⇐⇒ u j (1.34)

= 0 ⇒ b ⊥ u with u ∈ U (σ j ) v n + 1, j (1.35)

b ⊥ u

with u ∈ U (σ j ) v n + 1, j = 0 ⇒ (1.36)

If σ j is an isolated singular value, the converse of relations ( 1.33 ) and ( 1.35 ) also
holds.

Proof. See [92, Th. 1 - 3].

( See [ 98 ] . ) If σ n > σ n + 1 and v n + 1, n + 1 = 0 ,then

Corollary 23

± v n
0

,

σ n + 1 = σ n , u n + 1 =± u n , b , b , b ⊥ u n v n + 1 = (1.37)

The generic TLS approximation and corresponding TLS correction matrix

minimizes

b F but does not satisfy the constraint b ∈ R ( A ; A ) and therefore

does not solve the TLS problem (1.9). Moreover, (1.37) yields
A ;

b v n + 1 = 0 ⇒ A v n = 0

(1.38)

Then the solution v n describes an approximate linear relation among the columns
of A instead of estimating the desired linear relation between A and b . The sin-
gular vector v n + 1 is called a nonpredictive multicollinearity in linear regression

since it reveals multicollinearities in A that are of no (or negligible) value in pre-
dicting the response b . Also, since b ⊥ u = u n + 1 , there is no correlation between
A and b in the direction of u = u n + 1 . The strategy of nongeneric TLS is to
eliminate those directions in A that are not at all correlated with the observation

vector b ; the additional constraint x

1 ⊥ v n + 1 is then introduced. Latent root
regression [190] uses the same constraint in order to stabilize the LS solution in
the presence of multicollinearities. Generalizing:

−

Definition 24 (Nongeneric Unidimensional TLS Problem) Given the set
( 1.4 ) ,let ( 1.3 ) be the SVD of [ A ; b ]; the nongeneric TLS problem searches for

[ A ; b ] − [ A ;

b ] F

b ∈ R ( A )

min subject to and

[ A ;

b ] ∈ m × ( n + 1 )

x
− 1

(1.39)

⊥ v j , j = p + 1, ... , n + 1 ( provided that v n + 1, p = 0 )

Once a minimizing [ A ;

b ] is found, then any x satisfying

Ax = b

(1.40)

is called a nongeneric TLS solution ( the corresponding nongeneric TLS correction
is [

A ; b ] = [ A ; b ] − [ A ; b ] ) .