Neural Identification of Controlled Dynamical Systems and Recurrent Networks - Neural Networks: Methodology and Applications - page 272

input of the model. Thus, a model for the closed loop controlled system has
been constructed.
If we plug the output state into the state input as we have just done
in the previous paragraph, then, we get a neural simulator of the closed loop
controlled dynamical system. That architecture may be used, as shown above,
to predict the behavior of the system within a finite horizon.
Control systems are considered in detail in the next chapter.
4.5.3 Classical Recurrent Network Examples
In the two previous examples, recurrent networks that were displayed were
of the input-output type, the output being fed back to the input with a unit
time delay. In Chap. 2, we mentioned that state-space models are more general
and more parsimonious than input-output models. They are used in “black
box modeling” and also in “gray box modeling” where algebraic and differen-
tial equations that express domain knowledge are taken into account in the
structure of the network.
It should be remembered that, in recurrent networks, the delays must be
fully specified in order to avoid ambiguity in the networks dynamics. In the
appendix of the chapter we emphasize the importance of the delay distribution
with an example. The following should also be remembered (Chap. 2):
Rule. For a recurrent neural network to be causal, any cycle of the network
graph must have a nonzero delay.
Other examples of recurrent neural networks, having structures of differ-
ent complexities, were presented in Chap. 2. We present here two classical
types of recurrent neural networks. They are interesting from a historical and
didactical point of view since they are often quoted as examples. They are not
commonly used in practical applications.
4.5.3.1 The Elman Network
The Elman network is a layered neural network. It was suggested in the late
eighties to model contextual phenomena for applications in linguistic analy-
sis. [Elman 1990]. A lot of research involving recurrent neural networks in
linguistics has been performed in that period with a cognitive perspective.
Notice that in contrast with the modeling of a physical system, a context is
generally unknown, and it is not possible to determine it using a differential
equation or a variational principle. Hidden Markov Models turned out to be
an e cient, albeit complex, tool in speech analysis. The Elman model is re-
lated to those ideas: its purpose is to represent the context (i.e., the state of
the system) in a hidden layer. Actually, it is not possible to represent it as the
network output since it is impossible to compare it with any desired output.
A diagram of the Elman recurrent network is displayed on Fig. 4.16.