Neural Abstraction Pyramid Architecture - Hierarchical Neural Networks for Image Interpretation - page 75

but it needs a larger number of examples to determine the parameters of more ab-
stract representations.
In the dyadic case, where the resolution between the layers is reduced by a factor
of two in both dimensions, four hypercolumns correspond to a single hypercolumn
in the next higher layer. To make specific backward projections possible, each of the
four lower-level hypercolumns has to maintain its own backward projection. Thus,
in this case the backward weights are shared with only one fourth of the feature cells.
This connection structure can be viewed as distributed storage of a single larger pro-
jection that is computed in the reverse direction. For instance, when a reversed back-
ward projection has a size of 2 × 2, it covers all lower-level hypercolumns without
overlap. Such a projection is realized as four different 1 × 1 backward projections
with offsets (0,0) that are distributed between the 2 × 2 corresponding lower-level
hypercolumns.
A special case for the forward/backward projections is the drop of resolution to
1 × 1 hypercolumns at the top of the pyramid. Here, the offsets are usually chosen in
such a way that a complete connection structure between the topmost layer and the
layer below it is achieved. No weight sharing is possible in this case.

4.2.3 Discrete-Time Computation

The activities of the feature cells are computed at discrete points t of time. They are
accessed either directly or as a buffered copy. The cell activities must be initialized
and can be clamped to input values. Care must be taken to handle border effects.
Update Order. All feature cells are computed in a predetermined order. Usually,
the update of the activities proceeds layer by layer in a bottom-up manner. This is
done to speed up the forward flow of information. Within the layers, the features are
sometimes assigned to groups. For instance, excitatory and inhibitory features can
constitute two different groups. The fixed update order can assure that all features
of one group are updated before the first feature of another group is updated. This
makes fast lateral interactions, like fast inhibition, possible.
Direct Access. All weights of a projection access activities from the same point
of time t ∗ , described by the function T kl ( t ) . For direct access, activities that have

already been computed in the same time step are used: T kl ( t ) = t . This is pos-
sible only if the earlier update of the sources can be ensured. The direct access
is commonly used for forward projections and for fast lateral projections, like the
ones from excitatory to inhibitory features. This fast inhibition prevents a delay
between monosynaptic excitation and disynaptic inhibition. Fast inhibition is bio-
logically plausible since inhibitory synapses typically contact neurons mostly near
to the soma or even at the soma, while excitatory synapses typically connect to more
distant parts of the dendritic tree which induces some delay.
Buffered Access. Of course, not all projections can be realized with direct access.
It is also possible to access the activity of a feature cell from the previous time
step: T kl ( t ) = t − 1 . This buffered access is used for backward projections and for