Modeling with Neural Networks: Principles and Model Design Methodology - Neural Networks: Methodology and Applications - page 86

and of explaining. Scientific research strives to build knowledge-based mod-
els, whenever possible: the design of a knowledge-based model requires that
a theory be available, whereas the design of a black-box model requires that
measurements be available. Thus, Newton's theory of gravitation generated a
knowledge-based model of the motion of celestial bodies.
Semiphysical or gray-box models stand between knowledge-based and
black-box models: they embody both equations resulting from the applica-
tion of a theory, and empirical results from a black-box model.
At present, most neural network applications are black-box models; there-
fore, the first part of the present chapter is devoted to black-box modeling.
However, it will be shown that it can be very advantageous to use neural
networks as semiphysical models.

2.1.2 Static vs. Dynamic Models

A static model is made of algebraic equations only (e.g., a feedforward neural
network); by contrast, a dynamic model obeys differential (or partial differen-
tial) equations where time is the variable, and possibly algebraic equations as
well. We will first consider the design of static models. The design principles
for dynamic models (e.g., recurrent neural networks) will be explained next;
Chaps. 4 and 5 deal in more detail with dynamic modeling and control.

2.1.3 How to Deal With Uncertainty? The Statistical Context of
Modeling and Machine Learning

Before studying the design and implementation of a static black-box model, it
may be useful, for the benefit of the reader who is not familiar with those tech-
niques, to state the assumptions that underlie black-box modeling. Assume
that the quantity of interest y p is measurable, scalar 1 , and that one knows,
or suspects, that it depends, in some unknown, deterministic way, on one or
several measurable quantities called factors that can be gathered into a vector
x (which is a scalar if a single factor is involved in the modeling). In general,
the measurable factors do not provide a complete description of the evolu-
tion of the quantity of interest: the latter is also subject to disturbances that
are not measured (often not measurable). Two kinds of disturbances must be
considered,

• deterministic disturbances: putting a cold dish into a temperature-regulated

oven disturbs the temperature of the latter;

• noise: the noise inherent to the measurement of the quantity of interest

y p , for instance the noise of the sensor that measures the temperature of
the oven, disturbs the measurement.

1 The extension to the modeling of a vector does not involve any special di culty.