Modeling with Neural Networks: Principles and Model Design Methodology - Neural Networks: Methodology and Applications - page 93

temperature measurement apparatus, variations of the external temperature,
exo- or endo thermal reactions that may take place in the fluid. We did not
try to model the relations between the measured temperature and the factors
that may have an influence on the latter, since those factors were assumed to
be constant.
The problem of modeling that is addressed in the present chapter is more
complex. We want to find the mathematical relations between the quantity
of interest and the factors that may have an influence on it. If such relations
are available, then one can perform predictions about the evolution of the
quantity of interest as a function of its factors: for instance, if a relation is
found between the temperature of the oven and the intensity of the electrical
current in the heating resistors, then one can predict the temperature that will
be reached if a given intensity is flown into the resistors. One of the di culties
of modeling arises from the fact that all factors are not necessarily measured,
and possibly are not measurable: therefore, the statistical framework is still
appropriate, just as in the previous section.
2.3.1 Regression
Consider a measurable quantity y p , which depends on a set of factors that are
the components of a vector x . As in the previous section, it is convenient to
view the results of the measurements of y p as realizations of a random variable
Y , and to view the measured factors as realizations of a random vector 2 X .
Therefore, an estimate of the expectation value of the random variable Y for
a given realization x of the random vector X is sought; it is denoted by
E Y ( x ). That quantity is a function of x , called regression function (or simply
regression) of the random variable Y .
Since, as shown in the previous section, the expectation value is a quantity
that can only be estimated, but cannot be known exactly, the regression func-
tion is also unknown and can only be estimated; some of its characteristics,
such as the variance of Y for a given realization of X , or a confidence interval
on Y for a given realization of X , can be estimated. Thus, the model that
is sought is an estimation of the regression function; since neural networks
with supervised training are nonlinear parsimonious approximators as shown
in Chap. 1, they are good candidates as models of the quantity of interest if
the regression function is nonlinear.
In order to estimate the regression function from measurements of the
vector of factors (or input vector of the model, or variables of the model),
one must first make an assumption as to the regression function: the simplest
one is the linear (or a ne) assumption: it is assumed that, in the domain of
variation of the variables, a model that is linear or a ne with respect to the
latter can account satisfactorily for the behavior of the quantity of interest. If
2 A random vector is a vector, the components of which are random variables; each
component has its own probability distribution.