Information Technology Reference
In-Depth Information
The global asymptotic convergence of eq. (2.30) is established by the following
theorem.
Theorem 50 (Asymptotic Stability) Let R be positive semidefinite with mini-
mum eigenvalue λ n of multiplicity 1 and corresponding unit eigenvector z n .Then
it holds that
1
λ n z n
lim
→∞ w (
t
)
(2.31)
t
Proof. See [56, pp. 2127-2129].
In the simulations, the large oscillations of the weights around the solution
are apparent. According to the authors, this sensitivity of the estimate is caused
by the statistical fluctuation and error. In this chapter we show the true reasons
for this phenomenon.
2.3 MCA EXIN LINEAR NEURON
There are various ways of demonstrating the MCA EXIN learning law [30]. It
can be derived from the generalization of the TLS EXIN learning law [24,36] to
a vectorial space of one more degree of freedom. Another way (followed here)
is to find a stochastic learning law that has a well-defined averaging ODE, which
is exactly the opposite of the reasoning regarding the other learning rules. This
ODE represents the gradient flow of the Rayleigh quotient of the autocorrelation
matrix R ( = E x ( t ) x T
( t ) )on
N
− { 0 } .
As already noted, the input vector x is assumed to be generated by such an
ergodic 4 information source that the input sequence is an independent stochas-
tic process governed by p ( x ) . According to the stochastic learning law, the
weight vector
w
is changed in random directions and is uncorrelated with x .The
averaged cost function is
E y 2
w
T R
, R ) = w
w
E [ J ]
= r (w
=
(2.32)
T
2
2
w
w
T x . Then, up to a constant, the gradient flow of E [ J ] ,
which is then the averaging MCA EXIN ODE, is given by
where, as usual, y = w
R w
I
T
d w ( t )
dt
1
w ( t )
( t ) R w ( t )
w ( t )
=−
w ( t )
2
2
2
2
1
w
=−
[ R r (w , R ) I ] w
(2.33)
2
4 The information source generating the vectors x is ergodic if the temporal average over a typical
sequence x ( t ) is the same as the average over the probability distribution p ( x ) .
Search WWH ::




Custom Search