Information Technology Reference
In-Depth Information
Averaging the behavior of the weight vector after the first critical point has
been reached ( t t 0 ), the following formula can be written:
w( t ) = w( t ) 2 z n
t t 0
(2.104)
where z n is the unit eigenvector associated with the smallest eigenvalue of R .
From eq. (2.96) the discrete law can easily be deduced for updating the weight
modulus. This discrete law can be regarded as the discretization of the following
ODE:
w) E
2
T
y 2
w
d w
w
= s 2
T
(w
yx
w w
(2.105)
dt
T
2
Without loss of generality, the input data are considered as Gaussian; after some
matrix algebra, which can be found in [181], the following equation is derived:
( p ) λ
+ λ n tr ( R ) p
dp
dt = s 2
2
n
(2.106)
where λ n is its smallest eigenvalue of R , the input data autocorrelation matrix.
Solving this differential equation for s ( p ) = 1 (OJAn), with the initial condition
p ( 0 ) = 1 for the sake of simplicity, yields
p ( t ) = e λ n + λ n tr ( R ) t
(2.107)
and considering that the quantity in brackets is never negative, it follows that
p
(
t
) →∞
when
t
→∞
exponentially
(2.108)
Hence, the norm of
w(
t
)
diverges very fast. This divergence also arises when
s
(
p
) =
p (LUO); indeed,
1
p ( t ) =
1
(2.109)
2 λ
) t
n
+ λ n tr
(
R
In this case the divergence happens in a finite time ( sudden divergence ); that is,
p
(
t
) →∞
when
t
t
(2.110)
where
1
t =
2 λ
)
(2.111)
n
+ λ
n tr
(
R
Search WWH ::




Custom Search