Information Technology Reference
In-Depth Information
2
b
are estimated under the spherical Gaussian
assumption in order to compute the corresponding
2
a
and
experiment, variances
σ
σ
ζ
by (5.3):
a
σ
ζ
=
(5.5)
a
2
b
σ
+
σ
The optimal solution of problem (5.2) is given by the minimization of the
following cost function:
T
2
(
Ax
−
b
)
(
Ax
−
b
)
(
1
−
ζ )
+
ζ
x
T
x
1
E
GeTLS EXIN
(
x
)
=
(5.6)
The difference with the weighted (scaled) TLS, which would need a unique
scalar parameter going to infinity in order to have the DLS solution, as shown in
Section 5.1.1, is apparent in eq. (5.6). The same holds true for the one-parameter
generalization in [21]. A similar problem can be found if the augmented matrix
[
A
;
b
] is pre- and postmultiplied by particular matrices, as in [74]. However, the
need for a finite parameter in the numerical algorithms limits the accuracy. This
does not happen in the GeTLS EXIN formulation because DLS is represented
by a finite value, as a consequence of the fact that the parameter appears directly
in the denominator of the cost function (5.6).
For
ζ
=
0 this cost is the classical OLS sum-of-squared error; for
ζ
=
0
.
5this
cost is eq. (1.22). The validity of this expression for the DLS is demonstrated in
Section 5.2.4.
5.2.1 GeTLS EXIN Linear Neuron
The function (1.22) can be regarded as the cost function of a linear neuron whose
weight vector is
x
(
t
)
. The weight vector converges to the required solution during
training. In particular,
m
E
(
i
)
(
x
)
E
GeTLS EXIN
(
x
)
=
(5.7)
i
=
1
where
a
i
x
b
i
2
−
2
1
2
1
2
δ
E
(
i
)
(
x
)
=
x
T
x
=
(5.8)
(
1
−
ζ )
+
ζ
(
1
−
ζ )
+
ζ
x
T
x
Hence,
dE
(
i
)
dx
2
a
i
(
1
−
ζ )
+
ζ
x
T
x
−
δ
δ
ζ
x
=
(5.9)
(
x
T
x
2
1
−
ζ )
+
ζ
and the corresponding steepest descent discrete-time formula is
+
ζα(
)
x
2
x
(
t
+
1
)
=
x
(
t
)
−
α (
t
) γ(
t
)
a
i
t
) γ
(
t
(
t
)
(5.10)
Search WWH ::
Custom Search