Neural Abstraction Pyramid Architecture - Hierarchical Neural Networks for Image Interpretation - page 91

Activity

1

5

4

0.5

3

0 Time

2

1

0

(a) (b) (c)
Fig. 4.19. Different update methods: (a) buffered, (b) unbuffered, (c) activity-driven.

" #!

1 ,a i + a t −

i− 1 + a t +

− 1
2

a t +1

i +1

= max 0 , min (4.10)

,

i

2

where t ±

denotes the time the source activity was updated: either in the previous
step t or in the same step ( t + 1) . Neighboring cells are connected by excitatory
links and the activity is mapped to [0 , 1] using a negative bias and saturation. The
update modes differ in the handling of the activities of neighboring cells.
• Buffered update is conservative. All cells of the array have to be computed in
time step t before the resulting activities can be used in step ( t + 1) . This makes
the result independent of the update sequence within a time step. All cells can be
computed in parallel since no dependencies exist. On the other hand, the buffered
dynamics is relatively slow because information travels horizontally with a speed
of only one cell per time step.
• Unbuffered update computes the cells in a predetermined order, here from left
to right. The resulting activity a t +1

i− 1 of a cell is used immediately to compute
the activity a t + i of its right neighbor. The unbuffered dynamics converges much
faster since information travels the full array length from left to right within the
same time step. However, the information flow from right to left is still slow
which results in an undesired asymmetric response of the system.
• Activity Driven Update uses the same unbuffered strategy to speed up conver-
gence. It prevents undesired asymmetric responses by making the update se-
quence dependent on the array activity. The cells are updated in the order of
their activity from the last time step with the most active cell updated first. Fast
communication occurs now from the more active to the less active parts of the
cell array. Since the activities represent confidences of feature presence in image
interpretation tasks, the image parts that are easy to interpret are updated first,
which in turn biases and speeds up the interpretation of the more ambiguous im-
age parts. If multiple interpretations compete, the one that first receives support
from the context is likely to win the competition.
Activity-driven update also speeds up computation because cell activities that
become zero will not get active again and hence do not need to be updated any more.