Information Technology Reference
In-Depth Information
V
11
V
12
V
21
V
22
n
d
=
v
1
,
...
,
v
n
+
d
,
v
i
n
+
d
,
V
T
V
=
VV
T
V
=
∈
=
I
n
+
d
n
d
0
1
m
×
(
n
+
d
)
,
=
=
(σ
1
,
...
σ
n
+
t
)
∈
diag
,
0
2
n
,
2
=
diag
(σ
n
+
1
,
...
,
σ
n
+
t
)
∈
(
m
−
n
)
×
d
,
σ
1
≥···≥
σ
n
+
t
≥
0
For convenience of notation,
σ
i
=
0if
m
<
i
≤
n
+
d
.
u
i
,
σ
i
,
v
i
and
(
u
i
,
σ
i
,
v
i
)
are, respectively, the
singular triplet
of
A
and [
A
;
B
].
Most of the topic is devoted to the unidimensional
n
×
t
=
min
{
m
−
n
,
d
}
,
1
=
diag
(
σ
1
,
...
,
σ
n
)
∈
(
d
=
)
1
case; that is,
Ax
=
b
(1.4)
m
. However in Section 1.6 we deal with the multidi-
mensional case. The problem is defined as
basic
if (1) it is unidimensional, (2)
it is solvable, and (3) it has a unique solution.
m
×
n
where
A
∈
and
b
∈
1.4 ORDINARY LEAST SQUARES PROBLEMS
Definition 1 (OLS Problem)
Given the overdetermined system
(
1.4
)
, the least
squares
(
LS
)
problem searches for
m
b
−
b
2
to
b
∈
R
(
A
)
min
b
∈
subject
(1.5)
where R
(
A
)
is the column space of A. Once a minimizing b
is found, then any x
satisfying
Ax
=
b
(1.6)
is called an
LS solution (
the corresponding LS correction is
b
=
b
−
b
)
.
Remark 2
Equations
(
1.5
)
and
(
1.6
)
are satisfied if b
is the orthogonal projec-
tion of b into R
(
A
)
.
Theorem 3 (Closed-Form OLS Solution)
If rank
(
A
)
=
n, eqs.
(
1.5
)
and
(
1.6
)
are satisfied for the unique LS solution given by
x
=
(
A
T
A
)
−
1
A
T
b
=
A
+
b
(1.7)
(
for an underdetermined system, this is also the minimal L
2
norm solution
)
.
Search WWH ::
Custom Search