Information Technology Reference
In-Depth Information
Item Summary
Generalization Test
itm
err
%tot
%itm
correl
uniq
In addition to all of the above receptive field measures
of the network's performance, we can perform a behav-
ioral test of its ability to generalize in a spatially invari-
ant manner, using the two objects (numbers 18 and 19 in
figure 8.12) that were not presented to the network dur-
ing training. We can now train on these two objects in
a restricted set of spatial locations and sizes, and assess
the network's ability to respond to these items in novel
locations and sizes. The sweep test above indicates that
the network should generalize, because it produced spa-
tially invariant responses in layer V4 for these novel
stimuli. Thus, presumably the bulk of what the net-
work needs to do is learn an association between these
V4 representations and the appropriate output units, and
good generalization should result to all other spatial lo-
cations.
In addition to presenting the novel objects during
training, we also need to present familiar objects; other-
wise the network will suffer from
catastrophic interfer-
ence
(see chapters 7 and 9 for more discussion of this is-
sue). The following procedure was used. On each trial,
there was a 1 in 4 chance that a novel object would be
presented, and 3 in 4 chance that a familiar one was pre-
sented. If a novel object was presented, its location was
chosen at random from a
6
x
6
grid at the center of the
visual field (i.e., 36 possible locations, out of a total of
256, or roughly 14% of the locations), and its size was
chosen at random to be either 5 or 9 pixels (i.e., 2 out
of the 4 possible sizes, or 50% of the sizes). If a famil-
iar object was presented, then its size and position was
chosen completely at random from all the possibilities.
This procedure was repeated for 60 epochs of 150 ob-
jects per epoch, with a learning rate of .001. Although
the network started getting the novel objects correct af-
ter only 3 epochs, the longer training ensured that the
new knowledge was well consolidated before testing.
Testing entailed another sweep analysis after training
on the new objects (table 8.2). The detailed results are
contained in the file
objrec.swp.err
. In particu-
lar, pay attention to the
%itm
column, which shows the
errors as a function of total presentations of that item
(object). The overall results were that object 18 had
roughly 15 percent errors (85% correct) during testing
0
38
1.71
3.71
96.02
5.93
1
110
4.96
10.74
88.53
9.53
2
22
0.99
2.14
97.54
3.97
3
142
6.40
13.86
90.04
5.76
4
66
2.97
6.44
92.70
7.57
5
39
1.75
3.80
96.19
7.18
6
42
1.89
4.10
96.03
5.71
7
176
7.94
17.18
87.56
7.74
8
57
2.57
5.56
94.41
5.99
9
150
6.76
14.64
86.93
8.54
10
99
4.46
9.66
90.65
6.95
11
106
4.78
10.35
91.99
4.96
12
159
7.17
15.52
87.13
8.26
13
64
2.88
6.25
94.30
6.23
14
55
2.48
5.37
93.80
5.93
15
229
10.33
22.36
83.39
6.26
16
82
3.70
8.00
92.28
9.25
17
65
2.93
6.34
92.94
8.27
18
162
7.31
15.82
85.10
10.74
19
353
15.92
34.47
73.74
9.16
size
Size Summary
0
921
41.56
17.98
86.35
7.19
1
353
15.92
6.89
89.03
7.19
2
290
13.08
5.66
94.75
7.20
3
652
29.42
12.73
92.12
7.21
itm,size
Item by Size Summary
18,0
27
1.21
10.54
84.98
10.72
18,1
70
3.15
27.34
82.99
10.73
18,2
31
1.39
12.10
84.67
10.74
18,3
34
1.53
13.28
87.76
10.75
19,0
122
5.50
47.65
66.16
9.14
19,1
102
4.60
39.84
70.40
9.16
19,2
33
1.48
12.89
79.15
9.16
19,3
96
4.33
37.5
79.24
9.18
Tab le 8 . 2 :
Summary of sweep testing results after generaliza-
tion training on the novel items (18 and 19) in a limited num-
ber of positions and sizes. Columns are as in previous table,
with additional data for novel items in different sizes shown.
The
%itm
column for the novel items (15.82 and 34.47) gives
the generalization error, which is not bad considering the lim-
ited range of training locations and sizes (covering only 7% of
the total possible). The Item by Size data shows that size gen-
eralization to the untrained sizes (1 and 3) is also fairly good
(27.34 and 13.28 for object 18, 39.84 and 37.5 for 19). More
errors are also generally made in the smaller sizes. Finally,
there is some evidence of interference from the generalization
training in comparison with table 8.1.
Search WWH ::
Custom Search