Perception and Attention - Computational Explorations in Cognitive Neuroscience - page 242

a quarter of the V1 layer, with neighboring columns
again having half-overlapping receptive fields. Next,
the V4/IT layer represents just a single hypercolumn
of units ( 10x10 or 100 units) within a single inhibitory
group, and receives from the entire V2 layer. Finally,
the output layer has 20 units, one for each of the differ-
ent objects.
Now, let's build and connect the network.

erable amount of generalization during training, some
of which is due to the weight sharing, but probably not
all of it (weight sharing doesn't help with size invari-
ance, for example).
Because this network is relatively large, only the ac-
tual weight values themselves have been saved in a file.

Load the weights using LoadNet on the control
panel. Then, StepTrain a couple of times to see the
minus and plus phases of the trained network as it per-
forms the object recognition task. To enhance the ex-
citement of seeing the network perform this difficult task,
you can set the net_updt to CYCLE_UPDT and see the
activation patterns evolve as the network settles over cy-
cles of processing.
You should see that the plus and minus phase output
states are the same, meaning that the network is cor-
rectly recognizing most of the objects being presented.
To view a record of this performance, we can open a
training log.

, !

Press BuildNet on the objrec_ctrl overall con-
trol panel.
You will see the network fill in with units, and it will
also be connected.

Now, switch to r.wt , and click on units in the V1
layer.
You should see that each V1 unit receives from a set
of 4 input units, arranged either horizontally or verti-
cally. You may also be able to see that each hypercol-
umn of V1 units has a potential receptive field of a 4 x 4
patch of the input layers (as indicated by the slightly
darker gray color of those units), and neighboring hy-
percolumns receive from half-overlapping such patches.

Do View , TRAIN_LOG , and press the rewind VCR
button in the log to see prior results.
The sum_se column of this log shows the error for
each pattern. Although you may see an occasional er-
ror, this column should be mostly zeroes (the errors are
usually associated with the smallest sizes, which are
just at the low end of the resolution capabilities of the
network given its feature detectors). Thus, the network
shows quite good performance at this challenging task
of recognizing objects in a location-invariant and size-
invariant manner.

Now click on V2 units.
You should see that their connectivity patterns are
similar in form, but larger in size.
Now, let's see how the network is trained.

First, go back to viewing act in the networks display.
Then, do StepTrain in the control panel.
You will see the negative phase of settling for the
input image, which is one of the shapes shown in fig-
ure 8.12 at a random location and size.

, !

Press StepTrain again to see the plus phase. You
can then continue to StepTrain through a series of in-
puts to get a feel for what some of the different input
patterns look like.
Because it takes more than 24 hours for this net-
work to be trained, we will just load the weights from
a trained network. The network was trained for 460
epochs of 150 object inputs per epoch, or 69,000 ob-
ject presentations. However, it took only roughly 200
epochs (30,000 object presentations) for performance to
approach asymptote. This corresponds to the network
having seen each object in each location and size (i.e.,
1024 locations and sizes per object for 18 objects) only
twice (assuming a perfect uniform distribution, which
was not the case). Thus, it is clear that there is a consid-

Activation-Based Receptive Field Analysis

Unfortunately, it is not particularly informative to watch
the patterns of activity in the various layers of the net-
work in response to the different inputs. Further, be-
cause most of the units are not directly connected to the
input, we can't just view their weights to easily see what
they are representing. We can instead generate an acti-
vation based receptive field for the hidden units in the
network.
The activation based receptive field shows how a
unit's activity is correlated with another layer's activity
patterns, as measured over a large sample of patterns.
For example, we might want to know how a V2 unit's