Perception and Attention - Computational Explorations in Cognitive Neuroscience - page 231

the negative-valued ones in the “off-center” input layer.
The absolute values capture the fact that off-center neu-
rons get more excited (positively) when the input is
darker (more negative in the filtered images) in their
center than in their surround.
One critical aspect of any model is its scale relative
to the actual brain. This model is intended to simulate
roughly one cortical hypercolumn, a relatively small
structural unit in the cortex. As we discussed previ-
ously, a hypercolumn in V1 is generally thought to con-
tain one full set of feature dimensions, e.g., orienta-
tions, sizes, etc., for a given area of the retina. How-
ever, a hypercolumn has many thousands of neurons, so
our model of one hypercolumn is considerably reduced.
The main factor in determining the scale of the model
is not its raw number of units, but rather its patterns of
connectivity (and inhibition), because these determine
how units interact and whether they process the same or
different input information.
The present model's connectivity reflects the follow-
ing aspects of a hypercolumn: (1) All of the units re-
ceive from the same set of LGN inputs (actual V1 neu-
rons within a hypercolumn probably have different in-
dividual connectivity patterns, but they receive from
roughly the same part of the LGN); (2) The lateral
connectivity extends to a relatively large portion of the
neighboring units; (3) All of the units compete within a
common inhibitory system (i.e., one kWTA layer). The
kWTA average-based inhibitory function ensures that
no more than about 10 percent of the units can be active
at any given time, which is critical for specialization of
the units, as discussed in chapter 4.
One consequence of the model's scale is that only a
small patch of an overall image is presented to the in-
puts of the network, because a hypercolumn similarly
processes only one small patch of the overall retinal in-
put. In contrast with many other models of V1 that re-
quire initial spatially topographic connectivity patterns
to obtain specialization in the V1 units (e.g., the arbor
function in the Miller et al., 1989 and subsequent mod-
els), the V1 units in our model are also fully connected
with the input layers. If we were simulating a larger
patch of visual cortex in the model, we would proba-
bly need to include spatially restricted topographic con-
nectivity patterns, and we would expect that this rough

initial topography would be refined over learning into
more well-defined hypercolumn structures.
The mechanism for inducing topographic represen-
tations in the model relies on excitatory interactions
between neighboring units. These interactions were
implemented by having a circle of connections sur-
rounding each unit, with the strength of the connections
falling off as a Gaussian function of distance. Thus,
if a given unit becomes active, it will tend to activate
its (closest) neighbors via the lateral excitation. Over
learning, this will cause neighboring units to become
active together, and therefore develop similar repre-
sentations as they will have similar conditionalized re-
sponses to subsets of input images.
As discussed in chapter 4, this neighborhood excita-
tion is the key idea behind Kohonen networks (Koho-
nen, 1984; von der Malsburg, 1973), which explicitly
clamp a specific activation profile onto units surround-
ing the single most active unit in the layer. The main
advantage of the current approach is that it has consid-
erably more flexibility for representing complex images
with multiple features present. This flexibility is due
to the use of the kWTA inhibitory function, which just
selects the best fitting (most excited) units to be active
regardless of where they are located (though this is ob-
viously influenced by the excitatory neighborhood con-
nectivity). In contrast, the Kohonen function essentially
restricts the network to representing only one feature
(and its neighbors) at a time. Furthermore, the current
model is closer to the actual biology, where a balance
between lateral excitation and inhibition must be used
to achieve topographic representations.
In our model, the lateral excitatory connectivity
wraps around at the edges, so that a unit on the right
side of the hidden layer is actually the neighbor of a
unit on the left side (and the same for top and bottom).
This imposes a toroidal or doughnut-shaped functional
geometry onto the hidden layer. This wrap-around is
important because otherwise units at the edges would
not have as many neighbors as those in the middle, and
would thus not get activated as much. This is not a prob-
lem in the cortex, where the network is huge and edge
units constitute a relatively small percentage of the pop-
ulation.