GENERALIZATION OF LINEAR REGRESSION PROBLEMS - Neural-based Orthogonal Data Fitting: The EXIN Neural Networks - page 178

where σ is the angle between x and its ( n + 1 ) th component (in the figure, vector
b represents this last direction, which is parallel to the OLS residuals). Then

− cos σ
d GeTLS =

cos α
d OLS

Considering that α = π − (σ + γ ) , we obtain

cos σ
d GeTLS =

cos (σ + γ )
d OLS

Hence, if it assumed that the GeTLS error function (5.6) represents a sum of
squared distances,

d GeTLS = d OLS

cos 2 σ

E GeTLS =

cos 2 (σ + γ )

cos 2 σ 1

2

2 E OLS

= 2 E OLS

(σ + γ ) = cos 2 ( 1 − ζ ) + ζ p

where p = x T x . It follows that

σ ( 1 − ζ ) + ζ p

cos 2 (σ + γ ) = cos 2 (5.123)

This formula does not depend on p , except for ζ = 0, which implies that γ = 0
(OLS).

Summarizing: The GeTLS error function can be represented approximately as
a sum of skew distances only with regard to an orthogonal regression framework.

5.4 NEURAL NONGENERIC UNIDIMENSIONAL TLS

In Section 1.7 we presented the theoretical basis of the nongeneric unidimensional
TLS problem (i.e., when

σ > σ =···= σ 1 , p ≤ n and all v = 0, i = p p + 1 n + n + 1, i

p −

p + 1 critical points (the lowest) go to infinity [i.e., the corresponding zeros in
g

+ 1, ... , n + 1). From a geometric point of view, it means that the last n

2
p

(γ ) coincide with the first n − p + 1 asymptotes]. The saddle associated to σ

is the first critical point not coincident with an asymptote. Indeed,

σ p − 1 > σ p > σ p = σ p + 1 =···= σ n + 1 (5.124)

(see property (1.32) and [(98, p. 75)]. This saddle is the nongeneric TLS solu-
tion, because its position in the TLS hyperplane [eq. (4.50)] corresponds to the
closed-form solution (1.45). The singular value σ p + 1 represents the degree of