Unsupervised Learning - Hierarchical Neural Networks for Image Interpretation - page 109

Edges EdgeSum LineSum

3.48
0.94

4

3.46
0.85

6

2.99
0.45

9

2.60
0.35

15

E kl
I kl

Line( k ) Excitatory Weights Best Stimuli

Fig. 5.3. Learning a hierarchy of sparse features - line features. Four of the 16 features were
chosen that respond to approximately horizontal lines. The other features respond to lines of
other orientations.

Fig. 5.4. Learning a hierarchy of sparse features - stroke features. Shown are the eight best
stimuli of eight features that detect horizontal strokes with different curvature. The upper part
of the figure shows the sparse activity of all 32 stroke features.

Figure 5.3. They receive input mostly from the pair of horizontal step edges. The
lower horizontal edge feature is accessed by the lower part of the forward projec-
tions, while the upper horizontal edge is accessed by the upper part of the projection.
Step edges of other orientations contribute less to horizontal line features. The ac-
cess to the Layer 1 background feature that is done by the upper and the lower row
of projection weights is also interesting.
The 32 stroke features at Layer 3 are not as easy to describe as the line features.
They react to different local line shapes. Figure 5.4 shows the eight best stimuli
for eight of the stroke features that react to approximately horizontal lines. In addi-
tion to the orientation preference, some of the stroke features are also sensitive to
line curvature. Others seem to react to line endings. The feature in the lower right
corner is stimulated optimally by two parallel horizontal lines. It responds to the
background between the lines. The figure also shows in its upper part the activity of