Unsupervised Learning - Hierarchical Neural Networks for Image Interpretation - page 111

Fig. 5.6. Learning a hierarchy of sparse features - digit features. Shown are the eight best
stimuli. For the left column features were chosen that correspond to a single class. The right
column shows features that focus on some other aspect of a digit.

5.4 Digit Classification

In the following experiments, the usefulness of the learned sparse features for digit
recognition is investigated. First, only two layers of the pyramid are constructed.
The resulting representation is based on features that represent oriented lines. It
has the same total size as the input image. Table 5.2 shows the performance of a
KNN classifier and two feed-forward neural networks (FFNN) that were trained
with backpropagation using the digit's gray values and the extracted lines as fea-

Table 5.2. Learning a hierarchy of sparse features - classification of low-level features. Zero-
reject substitution rates of different classifiers.

features Gray Lines

classifier TST VAL TST VAL

KNN 15 2.98 2.87 4.53 4.36

FFNN 1024 − 10 5.82 6.48 2.04 2.14

FFNN 1024 − 64 − 10 2.49 2.65 1.90 2.04