Neural Networks: An Overview - Neural Networks: Methodology and Applications - page 20

1.1.4.2 To What Extent Is Parsimony a Valuable Property?
In the context of nonlinear regression and generalization, parsimony is indeed
an important asset of neural networks and, more generally, of any model that
is nonlinear with respect to its parameters. We mentioned earlier that most
applications of neural networks with supervised learning are modeling appli-
cations, whereby the parameters of the model are adjusted, from examples ,so
as to fit the nonlinear relationship between the factors (inputs of the model)
and the quantity of interest (the output of the model). It is intuitive that
the number of examples requested to estimate the parameters in a significant
and robust way is larger than the number of parameters : the equation of a
straight line cannot be fitted from a single point, nor can the equation of a
plane be fitted from two points. Therefore, models such as neural networks,
which are parsimonious in terms of number of parameters, are also, to some
extent, parsimonious in terms of number of examples; that is valuable since
measurements can be costly (e.g., measurements performed on an industrial
process) or time consuming (e.g., models of economy trained from indicators
published monthly), or both.
Therefore, the actual advantage of neural networks over conventional non-
linear modeling techniques is their ability of providing models of equivalent
accuracy from a smaller number of examples or, equivalently, of providing
more accurate models from the same number of examples. In general, neural
networks make the best use of the available data for models with more than
2 inputs.
Figure 1.42 illustrates the parsimony of neural networks in an industrial
application: the prediction of a thermodynamic parameter of a glass.
1.1.4.3 Classification (Discrimination)
Classification (or discrimination) is the task whereby items are assigned to
a class (or category) among several predefined classes. An algorithm that
automatically performs a classification is called a classifier .
In the vocabulary of statistics, classification is the task whereby data that
exhibit some similarity are grouped into classes that are not predefined; we
have mentioned above that neural networks with unsupervised learning can
perform such a task. Therefore, the terminology tends to be confusing. In the
present topic, we will try to make the distinction clear whenever the context
may allow confusion. In the present section, we consider only the case of
predefined classes.
Classifiers have a very large number of applications for pattern recognition
(handwritten digits or characters, image recognition, speech recognition, time
sequence recognition, etc.), and in many other areas as well (economy, finance,
sociology, language processing, etc.). In general, a pattern may be any item
that is described by a set of numerical descriptors : an image can be described
by the set of the intensities of its pixels, a time sequence by the sequence of