Neural Abstraction Pyramid Architecture - Hierarchical Neural Networks for Image Interpretation - page 76

delayed lateral connections, e.g. from inhibitory features to excitatory ones. It can
also be used for forward projections, e.g. to compare activities of two consecutive
time instances in order to detect activity change.
Initialization. At the beginning of the computations when t = 0 , it is not possible
to access the activities of the previous time step. For this reason, the activities must
be initialized before the iterative update starts. The initialization of a feature-array
k in layer l is done uniformly. All its feature cells are set to the activity a kl . Thus,
the uniform initialization adds only a single parameter to the template that describes
the computation of feature kl .
Network Input. To present input to the network some of its cells are not computed
by basic processing elements, but are clamped to static or dynamic inputs. In this
case, predetermined components of the input vector are copied to the cell outputs.
In general, different feature cells of an input array will receive different activities.
The input cells are accessed in the same way as all other cells.
Due to the recurrent network connectivity, inputs can occur at any layer. Signal-
like inputs, such as images, are presented at the lower layers of the pyramid, while
higher-level feature cells can be clamped to abstract features, such as class indica-
tors.
Network Output. Analogously, any feature cell can serve as a network output.
Outputs are not treated differently during the update of network activities. In par-
ticular, their activity is fed back into the network and influences other feature cells
and hence the output activities at later instances of time. Output cells play a special
role for supervised learning when predetermined components of a target vector are
compared to the activity of output units.
Feature cells which are neither input nor output play the role of hidden pro-
cessing elements. They maintain intermediate representations and mediate between
inputs and outputs.
Border Handling. The computation of the source hypercolumn's address may
yield positions that are outside the feature arrays. To ease the handling of such bor-
der effects, the arrays are framed with a border, such that all weights have a source
that is either a valid feature cell or part of the frame. The activity of frame cells is
determined after all feature cells of its feature array have been computed. Different
update modes are implemented. The easiest mode is to set the frame to a constant
value, e.g. to zero. In this case, it must be ensured that no discontinuity is created
between the feature cells and the frame cells. Another common update mode is to set
the frame cell activities to copies of feature cell activities. For instance, the feature
cells can be copied such that cyclic wrap-around border conditions are achieved.
In this case, it must be ensured that no discontinuities occur between the opposite
sides of the feature array. Other less common possibilities of border updates are the
fade-out of activity with increasing distance from the feature cells or the copying
with reflection at the border. The frame cells are accessed in the same way as all
other cells.