Modeling with Neural Networks: Principles and Model Design Methodology - Neural Networks: Methodology and Applications - page 157

a perfectly appropriate architecture, and an arbitrarily large training set, one
cannot obtain a modeling error equal to the noise if an inappropriate noise
assumption is made.

Modeling a Process with State Noise

We consider a computer-simulated process, which obeys the following equa-
tion:

y p ( k )= 1

y p ( k

T
a + by p ( k

− − 1)

− 1)

+ T c + dy p ( k

u ( k

− 1)

− 1) + b ( k ) .

a + by p ( k − 1)

It is thus a process with state noise, whose deterministic part is the same
as above: it will be modeled by a feedforward neural network with 5 hidden
neurons, as above. Again we make the two noise assumptions (output noise
and state noise).

Output Noise Assumption

We first make the (wrong) assumption that the noise is output noise. The ideal
model would be a recurrent one. Figure 2.37 shows the modeling error after
training a recurrent neural network with 5 hidden neurons. The modeling
error is clearly not white noise: the modeling error contains deterministic
information that the training of the model was unable to capture. Here again,
the failure is not due to a technical problem (too few or too many neurons,
ine cient training algorithm, inappropriate training data): it is due to the
fact that the model has a wrong structure, following the wrong assumption
that was made at the beginning.

Fig. 2.37. Modeling error for a process with state noise after training according to
the output noise assumption