Introduction - Hierarchical Neural Networks for Image Interpretation - page 12

1.2 Organization of the Thesis
The thesis is organized as follows:
Part I: Theory
Chapter 2. The next chapter gives some background information on the human
visual system. It covers the visual pathways, the organization of feature maps, com-
putation in layers, neurons as processing units, and synapses as adaptable elements.
At the end of the chapter, some open questions are discussed, including the binding
problem and the role of recurrent connections.
Chapter 3. Related work is discussed in Chapter 3, focussing on two aspects of
the proposed architecture: hierarchy and recurrence. Generic signal decompositions,
neural networks, and generative statistical models are reviewed as examples of hier-
archical systems for image analysis. The use of recurrence is discussed in general.
Special attention is paid to models with specific types of recurrent interactions: lat-
eral, vertical, and the combination of both.
Chapter 4. The proposed architecture for image interpretation is introduced in
Chapter 4. After giving an overview, the architecture is formally described. To illus-
trate its use, several small example networks are presented. They apply the architec-
ture to local contrast normalization, binarization of handwriting, and shift-invariant
feature extraction.
Chapter 5. Unsupervised learning techniques are discussed in Chapter 5. An un-
supervised learning algorithm is proposed for the suggested architecture that yields
a hierarchy of sparse features. It is applied to a dataset of handwritten digits. The
produced features are used as input to a supervised classifier. The performance of
this classifier is compared to other classifiers, and it is combined with two existing
classifiers.
Chapter 6. Supervised learning is covered in Chapter 6. After a general discus-
sion of supervised learning problems, gradient descent techniques for feed-forward
neural networks and recurrent neural networks are reviewed separately. Improve-
ments to the backpropagation technique and regularization methods are discussed,
as well as the difficulty of learning long-term dependencies in recurrent networks. It
is suggested to combine the RPROP algorithm with backpropagation through time
to achieve stable and fast learning in the proposed recurrent hierarchical architec-
ture.
Part II: Applications
Chapter 7. The proposed architecture is applied to recognize the value of postage
meter marks. After describing the problem, the dataset, and some preprocessing
steps, two classifiers are detailed. The first one is a hierarchical block classifier that
recognizes meter values without prior digit segmentation. The second one is a neural