Information Technology Reference
In-Depth Information
Remark 10
(
See
[
74
]
.
)
The ridge regression is a way of regularizing the
solution of an ill-conditioned LS problem
[
114, pp. 190ff.
]
; for example, the
minimization of
b
−
Ax
2
2
2
2
,where
µ
is a positive scalar, is solved by
+
µ
x
x
LS
(µ)
=
A
T
A
+
µ
I
−
1
A
T
b, and
x
LS
(µ)
2
becomes small as
µ
becomes large.
But x
TLS
=
x
LS
(
−
σ
2
n
+
1
)
, which implies that the TLS solution is a deregularizing
procedure, a reverse ridge regression. It implies that the condition of the TLS
problem is always worse than that of the corresponding LS problem.
Remark 11
Transforming
(
1.18
)
as
T
z
−
1
1
z
g
2
n
with
g
=
T
U
T
b
,
z
=
V
T
x
=
σ
g
T
2
2
+
b
−
1
(1.19)
Then
T
1
I
z
=
g and
σ
2
2
. Substituting z in the latter
−
σ
2
n
+
2
n
+
+
g
T
z
=
b
1
expression by the former yields
g
T
T
n
+
1
I
−
1
g
2
n
+
1
−
σ
2
2
2
σ
+
=
b
(1.20)
This is a version of the TLS secular equation
[
74,98
]
.
Remark 12
If
v
n
+
1,
n
+
1
=
0
, the TLS problem is solvable and is then called
generic.
Remark 13
If
σ
p
> σ
p
+
1
=···=
σ
n
+
1
, any vector in the space created by the
right singular vectors associated with the smallest singular vector is a solution
of the TLS problem
(
1.9
)
; the same happens in the case m
<
n
(
underdetermined
system
)
, since then the conditions
σ
m
+
1
=···=
σ
n
+
1
=
0
hold.
Remark 14
The TLS correction
[
A
;
b
]
−
[
A
;
b
]
F
is always smaller in norm
than the LS correction
b
b
2
.
−
1.5.1 OLS and TLS Geometric Considerations (Row Space)
The differences between TLS and OLS are considered from a geometric point of
view in the row space
R
r
(
[
A
;
b
]
)
(see Figure 1.1 for the case
n
=
2).
If no errors are present in the data (EIV model), the set
Ax
≈
b
is compatible,
rank[
A
;
b
]
=
n
, and the space
R
r
(
[
A
;
b
]
)
is
n
-dimensional (hyperplane). If errors
occur, the set is no longer compatible and the rows
r
1
,
r
2
,
...
,
r
m
are scattered
around the hyperplane. The normal of the hyperplane, corresponding to the minor
component of [
A
;
b
], gives the corresponding solution as its intersection with the
hyperplane
x
n
+
1
=−
1.
Search WWH ::
Custom Search