Neural Networks: An Overview - Neural Networks: Methodology and Applications - page 25

Therefore, neural networks may be advantageous in any application that
requires finding, in a machine learning framework, a nonlinear relation be-
tween numerical data.
Under what conditions is such an approach recommended?

• The first condition is necessary but not su cient: since neural network

design is essentially a problem in statistics, a set of examples, that sample
the space of inputs appropriately, and that are in appropriate number,
must be available.

• After gathering the data, one should make sure that a nonlinear model is

necessary, since the design of a linear model is much simpler and faster
than the design of a neural model. Therefore, if no prior knowledge on the
quantity to be modeled is available, one should first try out a linear model;
if it turns out that a linear model is too inaccurate, despite the fact that
all relevant factors are present in the inputs, then the model designer may
rightly resort to nonlinear models such as neural networks.

• If the appropriate examples are available, and if a nonlinear model is neces-

sary, then one should decide whether the use of neural networks, instead
of polynomials for instance, is advisable. Parsimony is the relevant choice
criterion here: as mentioned above, the number of parameters of the first
connection layer (between inputs and hidden neurons) increases linearly
with the number of variables, whereas it increases exponentially for poly-
nomial approximation (there exist, however, statistical tests that may, to
some extent, limit the combinatorial explosion of parameters in polynomial
modeling). Therefore, neural networks are advantageous when the number
of variables is large, i.e., empirically, larger than or equal to 3.

To summarize, if appropriate data sets are available, neural networks can
be used with advantage in all applications that require the estimation of the
parameters of a regression function with three variables or more. If the number
of variables is smaller, nonlinear models that are linear with respect to their
parameters, such as polynomials, radial basis functions with fixed centers and
standard deviation, wavelets with fixed translations and dilations, may be as
accurate, and require a simpler implementation.
If the available data are not numerical (e.g., symbolic), they cannot be
processed directly by a neural network. Some appropriate preprocessing is
required in order to make data numerical (techniques evolved from the theory
of fuzzy sets may be appropriate).

1.2.2 How to Design Neural Networks?

Neural networks are nonlinear parameterized functions, which can approx-
imate any nonlinear function. Therefore, approaching a regression function
from examples requires finding a neural network for which the sum, over all
examples used for training, of the squared modeling errors (the least squares