Application of Artificial Neural Networks - Digital Signal Processing in Power System Protection and Control - page 256

The above-described procedure of training is called ''supervised training''.
Independent of the ANN complexity the training is organized as follows:
• A set of associated input-output pairs are prepared in advance and then pre-
sented to a net which ''learns'' a model of the process.
• The neurons' weights are iteratively adjusted until the desired accuracy level is
achieved.
• A recursive algorithm based on a gradient-search optimization method applied
to an error function is executed either in incremental or in batch mode.
• The net errors are backpropagated form output layer to hidden layers (the
Backpropagation (BP) procedure is needed for multilayer networks) [ 38 ].
• Convergence of the process is checked—the procedure is stopped when the
MSE value becomes lower than certain threshold or when a predefined number
of iterations (epochs) is exceeded.
The supervised training procedure based on minimization of the network mean
square output error may sometimes have problems with convergence to the global
minimum within reasonable time or may get stuck in a local minimum. To
overcome the problems and assure faster convergence and more accurate results
additional supporting techniques have been developed, including variable learning
rate and ''momentum'' (new step is affected by previous step), that are well
described in [ 4 ]. It is also recommended to randomize the data set before each
iteration and to apply cross-validation (checking network response for testing data
set during training, learning is stopped when the error for testing data set starts to
rise) to get better results.
The unsupervised training (data self-organization concept) is another procedure
that is utilized e.g. in case of the SOM networks. The training procedure is
organized as follows:
• Given inputs are compared with previously encountered patterns.
• If they are similar to any of the patterns, they will be placed in the same
category, otherwise a new category (cluster) will be assigned.
• Category proliferation is controlled by a threshold.
• After the learning (cognition phase) the user defines or labels the clusters
according to some criterion.
Detailed ANN training algorithms are related to particular network type and
version and are not described here due to space limitation.
For a reader of this topic it is good to know what are the problems and issues that
should be addressed when an ANN-based solution is to be developed for an appli-
cation at hand. Generally, one should take into consideration the following points:
• ANN structure type,
• ANN size (number of layers and neurons in particular layers),
• Neuron activation function (may be different in given layers),
• Number and type of input signals,
• Representative set of patterns,