Neural Networks: An Overview - Neural Networks: Methodology and Applications - page 53

Fig. 1.33. LeNet, a neural network that performs feature extraction and classifica-
tion

64 hidden neurons receives information from a “receptive field” of 5 5 pix-

els. Those sets of 64 neurons are called feature maps , for the inputs to a given
map have the same weights (this is known as the “shared weights” technique,
described in Chap. 2): thus, the same operator acts locally on a 25-pixel area
of the picture, so that the outputs of a group of 64 neurons are the results
of the application of the same operator to the receptive fields. The local op-
erator technique is classical in picture processing, but the present approach
is original in that these operators are not engineered, but are “discovered”
through training by examples. The same technique is iterated by a second
layer of operators that act on the results of the first layer. Thus, 12 maps of
16 hidden neurons are produced by 192 neurons that provide the represen-
tation of the digit. Classification is performed by a final layer of 30 hidden
neurons, followed by 10 output neurons using a 1-out-of- N code: the number
of outputs is equal to the number of classes, output neuron number i must be
active if the input digit belongs to class i , and inactive otherwise.
Thus, the network performs, automatically and simultaneously, feature
extraction and classification, whereas those operations are usually performed
in a sequential fashion. The flexibility of the method has a price: given the
size of the network, training is demanding, and, because of the large number
of parameters, the network will be prone to overfitting.
In order to solve the same problem, a very different approach was im-
plemented [Knerr 1992], which consists in performing a more elaborate pre-
processing of the picture, in order to extract discriminant characteristics that
lead to a relatively simple classifier. Preprocessing consists of edge extraction,
followed by normalization, which produce 4 feature maps of 64 elements, hence

×