Information Technology Reference
In-Depth Information
1
Neural Networks: An Overview
G. Dreyfus
How useful is that new technology? This is a natural question to ask whenever
an emerging technique, such as neural networks, is transferred from research
laboratories to industry. In addition, the biological flavor of the term “neural
network” may lead to some confusion. For those reasons, this chapter is de-
voted to a presentation of the mathematical foundations and algorithms that
underlie the use of neural networks, together with the description of typical
applications; although the latter are quite varied, they are all based on a small
number of simple principles.
Putting neural networks to work is quite simple, and good software devel-
opment tools are available. However, in order to avoid disappointing results, it
is important to have an in-depth understanding of what neural networks really
do and of what they are really good at. The purpose of the present chapter is
to explain under what circumstances neural networks are preferable to other
data processing techniques and for what purposes they may be useful.
Basic definitions will be first presented: (formal) neuron, neural networks,
neural network training (both supervised and unsupervised), feedforward and
feedback (or recurrent) networks.
The basic property of neural networks with supervised training, parsimo-
nious approximation, will subsequently be explained. Due to that property,
neural networks are excellent nonlinear modeling tools. In that context, the
concept of supervised training will emerge naturally as a nonlinear version
of classical statistical modeling methods. Attention will be drawn to the nec-
essary and su
cient conditions for an application of neural networks with
supervised training to be successful.
Automatic classification (or discrimination) is an area of application of
neural networks that has specific features. A general presentation of automatic
classification, from a probabilistic point of view, will be made. It will be shown
that not all classification problems can be solved e
ciently by neural networks,
and we will characterize the class of problems where neural classification is
most appropriate. A general methodology for the design of neural classifiers
will be explained.
