Neural Networks: An Overview - Neural Networks: Methodology and Applications - page 27

Fig. 1.13. A schematic representation of the spot welding process
The questions of input selection, and of model selection as well, are by
no means specific to neural networks: they are of great importance for all
modeling techniques, whether linear or nonlinear. It will be shown, in Chap. 2,
that model selection techniques that were developed for linear models can be
extended to nonlinear models such as neural networks.
1.2.2.2 Data Collection
Before training, observations must be collected in order to build the training
set, as well as the validation and test sets, which will be defined below. Those
observations must be numerous enough, and they must be typical of the situa-
tions that will be encountered by the network when in use. When the number
of factors (model inputs) exceeds two or three, sampling the input domain in
a regular and systematic way is generally not feasible because combinatorial
explosion arises. Therefore, it is usually important to design the experiments
as e ciently as possible: experimental design is an important part of model
design. This is generally more di cult for nonlinear models than for linear
ones. Some elements will be given in the “Experimental design” section of
Chap. 2.
1.2.2.3 The Number of Hidden Neurons
The discrepancy between the neural approximation and the function to be ap-
proximated is inversely proportional to the number of hidden neurons [Barron
1993]; unfortunately, this result, as well as other theoretical results such as the
Vapnik-Cervonenkis dimension (or VC-dimension) [Vapnik 1995] (described in
Chap. 6) will only provide loose bounds or estimates of the number of hidden
neurons. At present, no result allows the model designer to find the appro-
priate number of hidden neurons given the available data and the desired
performance. Therefore, it is necessary to make use of a specific methodol-
ogy. In the following, we will first define the problem of designing a nonlinear
black-box static model, with emphasis on feedforward neural network design.