Neural Networks: An Overview - Neural Networks: Methodology and Applications - page 17

property (e.g., the boiling point) and “descriptors” of the molecules (e.g., the
molecular mass, the number of atoms, the dipole moment, etc.); such a model
allows predictions of the boiling points of molecules that were not synthesized
before. Several similar cases will be described in this topic.
Black-box models, as defined above, are in sharp contrast with knowledge-
based models, which are made of mathematical equations derived from first
principles of physics, chemistry, economics, etc. A knowledge-based model
may have a limited number of adjustable parameters, which, in general, have
a physical meaning. We will show below that neural networks can be building
blocks of gray box or semi-physical models, which take into account both
expert knowledge—as in a knowledge-based model—and data—as in a black-
box model.
Since neural networks are not really used for function approximation, to
what extent is the above-mentioned parsimonious approximation property
relevant to neural network applications? In the present chapter, a cursory
answer to that question will be provided. A very detailed answer will be
provided in Chap. 2, in which a general design methodology will be presented,
and in Chap. 3, which provides very useful techniques for the reduction of
input dimension, and for the design, and the performance evaluation, of neural
networks.

1.1.4.1 Static Modeling

For simplicity, we first consider a model with a single variable x . Assume that
an infinite number of measurements of the quantity of interest can be per-
formed for a given value x 0 of the variable x . Their mean value is the quan-
tity of interest z p , which is called the “expectation” of y p for the value x 0 of
the variable. The expectation value of y p is a function of x , termed “regres-
sion function”. Since we know from the previous section that any function
can be approximated with arbitrary accuracy by a neural network, it may
be expected that the black-box modeling problem, as stated above, can be
solved by estimating the parameters of a neural network that approximates
the (unknown) regression function.
The approximation will not be uniform, as defined and illustrated in the
previous section. For reasons that will be explained in Chap. 2, the model
will perform an approximation in the least squares sense: a parameterized
function (e.g., a neural network) will be sought, for which the least squares
cost function

N

y p ( x k )

g ( x k , w ) 2 J ( w )= 1

2

−

k =1

x k ,k =1to N is minimal. In the above relation, { } is a set of measured values

Ly p ( x k ) ,k =1to N

of the input variables, and as set of corresponding

measured values of the quantity to be modeled. Therefore, for a network that
has a given architecture (i.e., a given number of inputs and of hidden neurons),

{ }