Modeling with Neural Networks: Principles and Model Design Methodology - Neural Networks: Methodology and Applications - page 95

The above three tasks are described in the next three sections, building up
a complete design methodology that is essentially applicable to any nonlinear
model, be it neural or otherwise.

2.4 Input Selection for a Static Black-Box Model

When a model is designed from measurements, the number of variables must
be as small as possible, for each additional input generates additional para-
meters. In a neural model, each input gives rise to a number of parameters
that is equal to the number of hidden neurons. Therefore, it is necessary

• to find an input representation that is as compact as possible,

• to select all relevant factors as inputs to the model, but only the relevant

ones: the presence of input variables that are not relevant (i.e., whose
contribution to the output is smaller than the contribution of disturbances)
creates useless parameters and generates input variations that are not
significant, hence will generate modeling errors.

Input selection has two different sides,

• reduction of the dimension of the representation space for the variables of

the model,

• rejection of inputs that are not relevant.

2.4.1 Reduction of the Dimension of Representation Space

This first step of the input selection process considers only the inputs, irre-
spective of the quantity to be modeled; it aims at finding a data representation
that is as compact as possible. Consider the example shown on Fig. 2.2: two
data sets, corresponding to an input vector x of dimension 3, are displayed in
that space; for the right-hand side data set, the points are essentially aligned,
which means that the intrinsic data dimension is actually 1, instead of 3.
Through an appropriate change of variables, after which all points are borne
by a single axis, a one-dimensional representation of the data can be found.
That change of variables can be obtained through principal component analy-
sis (abbreviated as PCA, see for instance [Jollife 1986]). Similarly, for the
second data set, each point can be described by its curvilinear abscissa on a
curve: here again, the dimension of the representation can be reduced through
an appropriate processing of the data, such as curvilinear component analysis
or self-organizing maps [Kohonen 2001]. Those techniques are described in
detail in Chaps. 3 and 7.