Neural Networks: An Overview - Neural Networks: Methodology and Applications - page 10

Fig. 1.6. The canonical form of a recurrent neural network. The symbol q − 1 stands
for a unit time delay

where Φ and Ψ are nonlinear vector functions, e.g., neural networks, and
where x is the state vector. As in the linear case, the state variables are the
elements of the minimal set of variables such that the model can be described
completely at time k + 1 given the initial values of the state variables, and the
inputs from time 0 to time k . It will be shown in Chap. 2 that any recurrent
neural network can be cast into a canonical form, as shown on Fig. 1.6, where
q − 1 stands for a unit time delay. This symbol, which is usual in control theory,
will be used in throughout this topic, especially in Chaps. 2 and 4.

Property. Any recurrent neural network, however complex, can be cast into a
canonical form, made of a feedforward neural network, some outputs of which
(termed state outputs) are fed back to the inputs through unit delays [Nerrand
1993].

For instance, the neural network of Fig. 1.5 can be cast into the canonical
form that is shown on Fig. 1.7. That network has a single state variable (hence
it is a first-order network): the output of neuron 3. In that example, neuron
3 is a hidden neuron, but it will be shown below that a state neuron can also
be an output neuron.

Further Details

At time kT , the inputs of neuron 4 are u 2 [( k − 1) T ]and x [( k − 1) T ]=

y 3 [( k 1) T ]; just as in the origi-

nal (non-canonical) form, the inputs of neuron 3 are u 1 ( kT ) ,u 2 [( k

− 1) T ]); therefore, its output is y 4 [( k −

− 1) T ] ,

y 4 [( k − 1) T ]; therefore, its output is y 3 ( kT ); the inputs of neuron 5 are