Related Work - Hierarchical Neural Networks for Image Interpretation - page 57

Because the excitatory neurons cannot inhibit each other directly, each exci-
tatory cell is accompanied by an inhibitory interneuron. The units in such an E-I
pair are reciprocally connected. The lateral connection pattern to the interneurons is
such that similar orientations in columns that are not aligned are suppressed. While
the excitatory neurons have a transfer function which saturates for high inputs, the
transfer function of the inhibitory neurons does not saturate. Hence, if the network's
activity becomes too high inhibition exceeds excitation and lowers the activity again.
Initially, the network's activities are determined by the direct visual inputs
within the classical receptive fields of the units. As the network dynamics evolves,
the activities are quickly modulated by contextual influences mediated by the recur-
rent lateral connections. Li analyzed the network dynamics and demonstrated that
the network performs the tasks of texture segmentation, contour enhancement, and
perceptual pop-out. This is also illustrated in the figure.
Li [142] recently proposed that the contextual interactions in this V1 model,
which make consistent stimuli more salient, represent bottom-up attentional effects.
This mechanism discards inconsistent stimuli and focuses the limited resources of
the higher visual system to the most salient objects.
Networks with Spiking Neurons. Lateral coupling of spiking neurons can be used
to produce coherent firing. For instance, Hopfield and Brody [102, 103] proposed
a network where different features that belong to the same object are laterally cou-
pled. The coupling uses balanced excitation and inhibition and thus has little effect
on firing rates. In this network, synchronization occurs if the features have approxi-
mately the same activity. The synchronized firing of neuron groups is recognized by
a neuron in a higher layer that has short integration times and acts as a coincidence
detector. Time-warp invariant recognition of real-world speech data was demon-
strated in the network. However, Hopfield and Brody used a limited vocabulary of
only ten words.
A similar idea was applied by Henkel [94] to the problem of stereovision. He
used arrays of local disparity estimators with slowly changing parameters. Neigh-
boring cells are coupled laterally. In this network, smooth changes of disparity pro-
duce coherent firing that represents dense disparity maps. Local ambiguities are
resolved and noise is filtered out by the lateral interactions.
3.2.2 Models with Vertical Feedback
While horizontal connections mediate lateral interactions within an area, vertical
connections link areas of different degrees of abstraction. The connections from
lower areas, that are closer to the visual input and represent less complex features,
to higher areas are called feed-forward or bottom-up links. They serve feature ex-
traction purposes. The connections in the reverse direction are called top-down or
feedback links. They expand abstract representations to less abstract ones.
In the following, some models that involve vertical feedback are reviewed.