Information Technology Reference
In-Depth Information
In all the simulations of this chapter, the following numerical stop criterion has
been chosen.
Define
w(
k
)
=
w(
k
)
−
w(
k
−
1
)
(2.123)
n
where
w
∈
and
k
is the iteration; then the learning law stops when
w(
k
)
∞
≡
max
i
|
w
i
(
k
)
|
<ε
(2.124)
for
T
iterations, where the
L
∞
or Chebyshev norm has been taken into account;
T
is, in general, equal to the number of vectors of the training set or to a predefined
number in case of online learning and
ε
is very small.
Another interesting stop criterion, which exploits the MCA EXIN ODE
because of its slow divergence, has been devised, inspired by [78, p. 140].
Instead of testing the convergence of the sequence
{
r
(w
,
R
)
}
, which is not
suitable for comparing different algorithms because of its dependence on the
sequence
{
w (
t
)
}
, this criterion is based on the squared length of the gradient of
r
(
w
,
R
)
:
=
w
−
2
2
2
w
−
2
2
2
2
−
r
2
grad
r
(w
,
R
)
R
w
(w
,
R
)
(2.125)
2
2
and the
Considering the fact that grad
r
(
w
w
,
R
)
is inversely proportional to
2
stopping criterion cannot depend on
2
is necessary.
Furthermore, to avoid the dependence on the scale of
R
[if
R
is multiplied by a
constant,
r
w
2
, a normalization by
w
(w
,
R
)
; see eq. (2.2), and grad
r
(w
,
R
)
is also multiplied by the same
constant], the following stop criterion holds:
1
2
r
−
2
w
−
2
2
2
r
−
2
2
2
2
2
grad
r
(w
,
R
)
w
(w
,
R
)
=
R
w
(w
,
R
)
−
<ε
(2.126)
where
ε
is a prescribed threshold. If the initial guess
w
(
t
0
)
is selected very near
an eigenvector, the test values will be small in the first few iterations. Hence, it
is recommended that the process not be stopped until the test values are below
the threshold for several consecutive iterations. The same problem may happen
even if the learning trajectory crosses the solution without staying on it.
2.6.3 Limits of the Feng Neuron
Even if it converges to a solution [see eq. (2.31)], the FENG learning law has
some unacceptable problems: the high oscillations around the solution, which
prevent the choice of a reliable stop criterion, and the difficulty in controlling
the rate of convergence (see Theorem 71). Figure 2.10 shows a comparison with
MCA EXIN in the computation of the smallest eigenvalue of a matrix
R
, whose
Search WWH ::
Custom Search