Information Technology Reference
In-Depth Information
Tabl e 6 . 10 Results for the experiments with the Reber grammar using the topol-
ogy (7:0:2(2,2):7). ANS stands for Average Number of Sequences necessary to con-
verge.
η
=0
.
1
η
=0
.
2
η
=0
.
3
3
] % conv. ANS (std) [
10
3
] % conv. ANS (std) [
10
3
]%conv.
ANS (std) [
10
MMSE
19.2 (26.1)
41
46.0 (77.2)
53
30.1 (50.7)
61
MEE h=1.3
57.2 (67.5)
43
44.3 (58.8)
18
28.4 (12.9)
5
MEE h=1.4
20.2 (27.8)
55
70.5 (112.4)
22
108.9 (130.5)
10
MEE h=1.5
33.6 (55.3)
59
68.1 (109.8)
36
31.7 (32.7)
19
MEE h=1.6
26.5 (41.9)
65
38.3 (47.2)
42
58.4 (82.7)
25
MEE h=1.7
32.3 (57.2)
53
48.4 (83.8)
48
55.6 (82.9)
26
MEE h=1.8
24.5 (48.8)
60
54.4 (88.3)
61
43.4 (82.0)
40
MEE h=1.9
51.8 (76.7)
49
70.6 (134.5)
66
62.1 (108.1)
66
MEE h=2.0
44.6 (79.3)
48
48.3 (85.8)
68
44.3 (70.2)
41
Reber
grammar
T
T
B
E
P
P
Reber
grammar
Fig. 6.18
A finite-state machine for the embedded Reber grammar.
it with some diculty, since, as opposed to the Reber grammar problem,
there is the need to retain information for long time lapses.
In this case the experiments reported in [6] were similar to the ones de-
scribed for the Reber grammar but the learning rates used were 0.1 and 0.3.
The train and test sets had also 500 strings each. The topology used was
7:0:4(3,3,3,3):7 (the coding is the same as in the Reber grammar example).
The experiments were also repeated 100 times. The results are in Table 6.11.
The last dataset consists of a series of strings from the grammar
A
n
B
n
.
Valid strings consist of
nA
symbols followed by exactly
nB
symbols. The
network is trained with only correct strings, and
n
from 1 up to 10. It is
considered that the network converged if it is able to correctly classify all the
strings in both training and test sets, using less than 50 000 sequences for
learning. In the first experiment reported in [6] the correct strings were used
for the test set for
n
=1
...
50. In the second experiment the correct strings
were used for
n
=1
...
100. In both experiments the network topology was
3:0:2(1,1):3. Both experiments were repeated 100 times for
η
=1.Theresults
obtained are in Table 6.12.
