Modeling with Neural Networks: Principles and Model Design Methodology - Neural Networks: Methodology and Applications - page 149

designed. Then one should perform experimental planning, taking advantage
of the results obtained during the design of the model, with emphasis on
confidence intervals: the presence of large confidence intervals in an area of
input space may be due to an inappropriate number of examples in that area.
Therefore, measurements should be performed in the areas of input space
where confidence intervals are too large.

2.6.4.5 Conclusion

The design of a good model requires a systematic, principled methodology. We
have shown that such a methodology exists, which can be applied for designing
essentially any nonlinear model, including, but not limited to, neural networks.
Its principles are the following:

• Neural networks are parsimonious approximators, that can be advanta-

geously used for models having more than two variables; for models with
less than two variables, models that are linear with respect to their para-
meters, such as polynomials, give excellent results and are trained more
easily.

• Whether the model is linear or nonlinear with respect to its parameters,

the first step consists in an analysis of the input data, in order to find a
data representation that is as compact as possible, and in a subsequent
input selection in order to select only the candidate variables that are
really relevant.

• A model architecture is subsequently chosen (number of monomials for a

polynomial model, number of hidden neurons for a neural model, etc.),
and the parameters of the model are estimated (training). Those tasks
are performed from the simplest architecture (linear model), gradually
increasing the complexity of the models.

• For each architecture the best model is selected, and the “best” models

of the different structures are mutually compared, until the final choice is
performed.

2.7 Dynamic Black-Box Modeling

The previous section discussed the design of static models, i.e., models that
implement a static input-output mapping. Those models are very useful for
modeling a process in a steady state, or for finding relations between time-
independent data.
In the present section, we discuss dynamic models, whose inputs and out-
puts are related through differential equations, or, for discrete-time systems,
by recurrent equations or difference equations. In the present chapter, we con-
sider only discrete-time systems because the vast majority of real applications
of neural networks involve computers or digital integrated circuits, which are