Modeling with Neural Networks: Principles and Model Design Methodology - Neural Networks: Methodology and Applications - page 163

ν is the order of the model. Therefore, that form is the minimal state-space
representation; if the state vector is in the form
z ( k )=[ y ( k ) ,y ( k ν +1)] T ,

− 1) ,...,y ( k −

the canonical form is an input-output model: the output is the only quantity
involved in the state vector.
Two cases must be considered:

• A black-box model is sought: then a model should be designed under the

canonical form, since there is no reason to choose another form;

• A semiphysical model is sought, taking into account prior knowledge: the

latter may lead to a model that is not in canonical form; then, prior to
training, the predictor should be put in canonical form, which is always
possible. The section entitled “Casting dynamic models into a canonical
form” is devoted to that problem.
In the following sections, the model is always assumed to be in its canonical
form.
We first discuss the training of feedforward models, then the training of
recurrent models.

2.7.3.1 Nonadaptive (Batch) Training of Feedforward
Input-Output Models: Directed (Teacher-Forced) Training

Under the state noise assumption, the ideal model is a feedforward (static)
model whose inputs are the control inputs and the measured process outputs
at the previous n time steps. The training is called directed by the process,
or teacher-forced, since the measured process outputs are input to the model
during, as shown on Fig. 2.43. Thus, the model is permanently “driven” by the
process outputs. The training of that model is exactly similar to the training
of a static model. The training set is a sequence of N input-output pairs
{ z k , y k } ,where N is the length of the training sequence,

z k =[ u ( k ) ,u ( k

n +1)] T , − 1) ,...,u ( k − m +1) ,y p ( k ) ,y p ( k − 1) ,...,y p ( k −

y k = y p ( k +1) .
The Dumb Predictor Pitfall

In directed training, the measured outputs of the process are input to the
model, at each time step. Therefore, deceptively good results can readily be
obtained, if the quality of the model is assessed by carelessly superimposing
graphically the measured output and the predicted output. Actually, a “dumb
predictor” made of a simple unit time delay, i.e., a predictor that states that
the process output at time k +1 will be equal to the process output measured
at time k , may provide excellent results, if the process output does not vary
much during a sampling period, i.e., if the sampling frequency is high enough.
Therefore, after training by teacher forcing, the results should always be com-
pared to those of the dumb predictor. Disappointments are not infrequent.