Neural Networks: An Overview - Neural Networks: Methodology and Applications - page 12

1.1.2 The Training of Neural Networks

Training is the algorithmic procedure whereby the parameters of the neurons
of the network are estimated, in order for the neural network to fulfill, as
accurately as possible, the task it has been assigned.
Within that framework, two categories of training are considered: super-
vised training and unsupervised training.

1.1.2.1 Supervised Training

As indicated in the previous section, a feedforward neural network computes
a nonlinear function of its inputs. Therefore, such a network can be assigned
the task of computing a specific nonlinear function. Two situations may arise:

• The nonlinear function is known analytically: hence the network performs

the task of function approximation,

• The nonlinear function is not known analytically, but a finite number of

numerical values of the function are known; in most applications, these
values are not known exactly because they are obtained through measure-
ments performed on a physical, chemical, financial, economic, biological,
etc. process: in such a case, the task that is assigned to the network is
that of approximating the regression function of the available data, hence
of being a static model of the process.

In the vast majority of their applications, feedforward neural networks with
supervised training are used in the second class of situations.
Training can be thought of as “supervised” since the function that the net-
work should implement is known in some or all points: a “teacher” provides
“examples” of values of the inputs and of the corresponding values of the out-
put, i.e., of the task that the network should perform. The core of Chap. 2 of
the topic is devoted to translating the above metaphor into mathematics and
algorithms. Chapters 3, 4, 5 and 6 are devoted to the design and applications
of neural networks with supervised training for static and dynamic modeling,
and for automatic classification (or discrimination).

1.1.2.2 Unsupervised Training

A feedforward neural network can also be assigned a task of data analysis
or visualization: a set of data, described by a vector with a large number of
components, is available. It may be desired to cluster these data, according
to similarity criteria that are not known a priori. Clustering methods are well
known in statistics; feedforward neural networks can be assigned a task that is
close to clustering: from the high-dimensional data representation, find a rep-
resentation of much smaller dimension (usually 2-dimensional) that preserves
the similarities or neighborhoods. Thus, no teacher is present in that task,