Information Technology Reference
In-Depth Information
Fig. 5.5. Learning a hierarchy of sparse features - curve features. Shown are the eight best
stimuli of the 16 first features. They respond to typical digit parts. The upper part of the figure
shows the sparse activity of all 64 curve features.
all stroke features when the input from Figure 5.2 is presented to the network. One
can see that the representation is already quite sparse.
Figure 5.5 shows the eight best stimuli of the first 16 of the 64 curve features
that reside on Layer 4 of the pyramid. They detect typical digit parts, such as open
and closed loops, line crossings, and major strokes. It can be observed that for most
curve features all of the best stimuli belong to the same digit class. The activity of the
curve features is sparse since not all typical configurations of strokes are contained
in a single digit.
The best stimuli of some of the top-layer digit features are shown in Figure 5.6.
For the left side of the figure, digit features that react best to one of the ten digit
classes were selected. The right side shows digit features that were selected because
they react to examples from different classes. They seem to focus on some aspect of
the digit, such as to the presence of a vertical line or to a characteristic curve. One
must ask the question: 'What do the best stimuli have in common?' to find out what
a specific feature cell detects.
The emerging feature detectors do not represent all possible combinations of
substructures, but only the typical ones. The more frequent combinations of lower
level features are represented by multiple similar features with greater detail than
the less frequent ones. When going up in the hierarchy of representations, the cor-
relation of feature activities with the digit class increases. This is remarkable since
no class information has been presented to the system so far.
Search WWH ::




Custom Search