Modeling Methodology: Dimension Reduction and Resampling Methods - Neural Networks: Methodology and Applications - page 230

• networks generated automatically are su ciently well adjusted, even in the

most di cult cases, where the number of examples is small. The statistics
provided by the bootstrap allow for the automatic control of the early
stopping of training, and provide sound statistics for the generalization
error;

• the second point is associated with the problem of the size of the input

space. Even in the example of the relation application of

12

,afew
hundred points are enough to represent the relation. In many problems,
nonlinear relations may be approximated easily from a low density of ex-
amples. It should be noted that from a certain level of complexity, networks
created and adjusted using the same sample appear to be equivalent. Dif-
ferent networks may be adapted to represent the same relation.

in

R R

Within the context of statistical learning theory, the adjustment of models
may be monitored, hence optimized, by bootstrapping. That approach should
be compared with more formal methods based on the theory of [Vapnik 1995],
the goal being the adaptation of the computing capacity (VC dimension) of
the model to the data. In that context, statistical resampling methods provide
real solutions, which can easily be implemented, and can run in reasonable
time on present-day computers.

References

1. A. Cichoki, R. Unbehauen, Neural Networks for Optimization and Signal
Processing, Wiley, 1993
2. P. Demartines, Analyse de donnees par reseaux de neurones auto-organisees,
thesis at the Institut National Polytechnique de Grenoble
3. Patrick Davaud, Traitement du Signal Concepts et Applications, Hermes, 1991
4. Bradley Efron, Robert J. Tibshirani, An introduction to the Bootstrap, Chap-
man & Hall, 1993
5. Jeanny Herault, Christian Jutten, Reseaux de neurones et traitement du signal,
Hermes, 1993
6. Vincent Pilato, Application des reseaux de neurones aux methodes de mesure
basees sur l'interaction rayonnement matiere, thesis Universite Paris-Sud, 4/11/
1998
7. Gilbert Saporta, Probabilites Analyse des donnees et Statistique, Editions Tech-
nip, 1990
8. Vladimr N. Vapnik, The Nature of Statistical Learning Theory, Springer, 1995
9. Vincent Vigneron, Methodes d'apprentissage statistiques et problemes
inverses—Applications a la spectrographie, thesis for the Universite d'Evry Val
d'Essonne, 5/5/1997
10. Thomas H. Wonnacott, Ronald J. Wonnacott, Statistique Economie-Gestion-
Sciences-Medecine, Economica, 4th issue, 1990