Application of Artificial Neural Networks - Digital Signal Processing in Power System Protection and Control - page 257

= = Training data

= = Test data

good
generalization
(learning)

concept
(truth)

bad
generalization
(memorization)

Fig. 12.14 Illustration of good and bad training results

• Initial values of synapse weights and biases (usually random values),
• The training algorithm itself (depends on the ANN type),
• Generalisation versus memorisation dilemma.
Having chosen the ANN type for given task (e.g. multilayer perceptron) one
should decide on the size of the neural network. It has been proven that infinitely
large neural network with just a single hidden layer is capable of approximating
any continuous function [ 10 ]. In practice ANNs with one or two hidden layers are
in use, with the number of neurons dependent on the number of ANN input signals
and the size of training set. The neural network size is sometimes chosen arbi-
trarily, but it can also be optimized, e.g. with use of a genetic procedure (see
Chap. 13 ).
The number of input signals and their choice is crucial for proper operation of
the ANN. It is clear that the signals delivered to the input layer of the neural
network should contain the highest amount of information about the events to be
classified. Their choice should be done individually for given task, at best by an
expert in the field. Both the training and testing sets should be representative in the
sense of containing all possible situations that may happen in real life.
The last of the above-mentioned problems (generalization vs. memorization)
results from the very fact that the well trained ANN is expected to operate cor-
rectly not only for all the patterns from the training data set, but also for all other
possibly appearing data that were not presented to the ANN during training. Such a
feature means generalization of the acquired knowledge, which is different from
focusing on the training cases only (memorization). The difference is illustrated in
Fig. 12.14 , where the ANN is assumed to be able differentiating the circle points
from the cross points. The two categories of points belong to respective parts of the
plane that can easily be separated by the black solid line representing the desired
division of the plane. When the ANN is ''overtrained'', it may become focused on