Information Technology Reference
In-Depth Information
X
(
k
+1)=
A
[
w
]
X
(
k
)+
V
(
k
+1)
,
where
A
[
w
] depends linearly on
w
,
(
V
k
) is a white vector noise and where
⎛
⎝
⎞
⎠
a
1
a
2
..
k
10
..
0
01
..
0
. . . .
0
00
.
10
X
(
k
)=[
X
(
k
);
...
;
X
(
k
−
p
+ 1)]
and
A
(
w
)=
.
The stochastic approximation theory is valid in that general Markov frame
and provides the consistency statement about the almost sure convergence of
the recursive estimate.
There exist as well recursive versions of second-order optimization algo-
rithms (Newton rule). The estimates are consistent too. Their convergence
may be proved in the general stochastic approximation framework [Ljung
1983]. They are of special interest for linear models, because they provide
a powerful way to speed up convergence. Recall (see Chap. 2) the Newton
formula,
J
[
w
∗
]
,
where
H
J
[
w
∗
] is the Hessian matrix of the cost function. The elements of
the Hessian matrix of a function of several variables are the second partial
derivatives, and its symmetry is guaranteed by the inversion Schwarz rule.
The formulation of Newton formula leads to the recursive relation
w
=
w
∗
−
H
J
[
w
∗
]
−
1
∇
H
Φ
[
w
(
k
)]
−
1
w
(
k
+1)=
w
(
k
)
−
∇
Φ
[
w
(
k
)]
.
In the case of a strictly convex function (e.g., for a quadratic criterion),
this matrix is definite positive thus invertible. In the previous example of the
AR(
p
) model, it is equal to the covariance matrix of the stationary random
vector
X
k
.
The recursive second-order algorithm combines the second-order optimiza-
tion of
J
and the recursive estimation of covariance matrix
R
,
w
(
k
+1)=
w
(
k
)+
γ
k
+1
ϑ
(
k
+1)
R
(
k
)
−
1
X
(
k
)
R
(
k
+1)=
R
(
k
)+
γ
k
+1
X
(
k
+1)
X
(
k
+1)
T
.
That method is called recursive prediction error method (RPEM). It is fully
described in [Ljung 1983], with emphasis on the applications to identification.
The RPEM method may be extended to nonlinear models. It may be used for
neural network adaptive learning when the learning data are provided on-line
by an experimental process or by a simulation.
Search WWH ::
Custom Search