GENERALIZATION OF LINEAR REGRESSION PROBLEMS - Neural-based Orthogonal Data Fitting: The EXIN Neural Networks - page 176

x

s

d OLS

r i

g

a

b

Figure 5.12 Geometry of the skew distance d GeTLS in the row space.

It follows that

cos 2

( 1 + p ) α = 2 ( 1 − ζ ) + 2 ζ p = 2 − 2 ζ ( 1 − p )

2
1 + p −

1 p

1 + p 2

−

cos 2

α = ζ

2 1 − p

1

sin 2 α = 1 − cos 2

α = 1 −

p + p 2 ζ 1 + +

1 p

1 + p (

−

= 2 ζ − 1 ) (5.120)

which depends on p except for ζ = 0 . 5, which implies that α = 0. That is the
TLS case, in which the orthogonal distances are used; indeed, we know that in
this case α does not depend on p .

From eq. (5.120), other considerations can be drawn. From

p −→ 0
−→

sin 2 α 2 ζ − 1 with ζ ∈ [0 . 5, 1] (5.121)

p −→ ∞

−→

sin 2 α 1 − 2 ζ with ζ ∈ [0, 0 . 5] (5.122)

it can be deduced that during the first part of the transient, in the case of null
initial conditions, p is close to 0, and for ζ ∈ [0 . 5, 1], as a first approximation,
the GeTLS error cost represents a sum of skew distance whose slope depends on
the value of ζ . The same is true for large p , but for ζ ∈ [0, 0 . 5] . These two linear
functions of ζ are visible in Figure 5.13 (they are the borders of the hypersurface),
which plots eq. (5.120) together to the hyperplane sin 2 α = 0, which is the lowest

limit of the admissible space (the squared sine cannot be negative). Obviously,
when p = 1, the distances are orthogonal. Figure 5.14 shows the derivative of