Information Technology Reference
In-Depth Information
which is anticipated in case of noisy data). It cannot be stopped reliably
and is very sensitive to outliers. It is a low variance/high bias algorithm
and the computational cost per iteration is the best. It works very badly for
medium-dimensional data.
OJA +
. It has a very slow convergence to the MC direction and its squared
weight modulus change is of order O ( α )
. The weights converge only for
1,
there is sudden divergence, which is anticipated in case of noisy data). It
cannot be stopped reliably and is very sensitive to outliers. It is a low
variance/high bias algorithm. It works very badly for medium-dimensional
data.
FENG . It is the worst learning law because of the too large oscillations
(the dynamic stability does not improve when the MC direction is reached
because it depends only on p and the learning rate). It has a slow con-
vergence (same as LUO) to the MC direction and is very sensitive to the
outliers. It cannot be stopped reliably and can diverge for near-singular
matrices. It works very badly for medium-dimensional data.
λ n <
λ m >
1 (in case of initial conditions of modulus greater than 1 and
MCA EXIN is by far the best MCA learning law. It has the best convergence
to the MC direction, the slower divergence, and does not have problems of either
sudden or instability divergence. It works very well in high-dimensional spaces
and has been used in real applications. It is robust to outliers because of the
presence of the squared modulus of the weight in the denominator of its weight
increment. However, there also exists a variant (NMCA EXIN [28]), which is
robust to outliers because it implements the theory of the M -estimators [91], and
it is presented in Chapter 3. MCA EXIN can be stopped reliably. It is a high
variance/low bias algorithm with very good dynamics because of its inertia for
large weights. It has the same computational cost as the other MCA laws, except
OJA.
The best possible choice for the initial conditions should be as low as possible.
However, there is a limit for MCA EXIN: too large oscillations in the transient
(not divergence!). In this case, a variant, called MCA EXIN
, has been presented
in [24], which allows null initial conditions and works better than MCA EXIN.
It is described in Chapter 5.
+
Search WWH ::




Custom Search