Information Technology Reference
In-Depth Information
Table 2.4 Divergence of the MCA learning laws for autocorrelation matrices of
increasing size
a
dim
LUO
FENG
OJA
OJAn
OJA
+
3
conv.
conv.
conv.
conv.
conv.
7
conv.
conv.
conv.
conv.
conv.
9
conv.
413
conv.
conv.
conv.
10
conv.
460
conv.
conv.
conv.
15
conv.
531
conv.
conv.
conv.
18
conv.
371
conv.
conv.
conv.
20
conv.
700
180
conv.
65
25
conv.
27
27
conv.
26
30
conv.
27
27
1848
17
40
conv.
400
17
975
12
50
conv.
370
6
670
12
60
conv.
540
3
550
14
70
conv.
260
7
520
12
80
545
220
5
400
6
90
8
7
8
355
8
100
8
8
8
250
5
a
The numbers show at which iteration the weights go to infinity; conv., convergence.
•
LUO
. The weight modulus increases as before, but starts from lower values.
Hence, there is always instability divergence, but for higher
n
. The experi-
ments have shown that it is very sensitive to the learning rate, as expected
from (2.159).
•
OJA
. With this choice of initial conditions, the weight modulus quickly
16
decreases to 0 (see Section 2.6.2.2) and then the stability subspace is smaller
(see Figure 2.13), which implies an increase in the oscillations and the insta-
bility divergence. For increasing
n
, the divergence is anticipated. Probably
it is due to the fact that it is more difficult to fit data with a linear hyper-
plane because of the problem of empty space [10] caused by the increasing
dimensionality of data and accentuated by the fact that this law is only an
approximate RQ gradient, which implies that the OJA algorithm does not
reach the MC direction and then diverges very soon.
•
OJAn
. The same analysis for LUO is valid here. However, for increasing
n
,
since the first iterations, there are large fluctuations. This could be expected
from eq. (2.159), which, unlike LUO, does not depend on
p
, so the algorithm
cannot exploit the low initial conditions. This instability is worsened by
the problem of the empty space; indeed, the algorithm does not reach the
MC direction, so the oscillations increase because on average,
ϑ
does
x
w
not approach
st
remains small enough to cause the instability
divergence for values of
n
smaller than those in the case of LUO.
π/
2and
16
Proportional to
α
(see Section 2.6.2.1).
Search WWH ::
Custom Search