Neural Identification of Controlled Dynamical Systems and Recurrent Networks - Neural Networks: Methodology and Applications - page 237

Fig. 4.3. Trajectory of a Van der Pol oscillator. In figure ( a ) a limit cycle is observed.
In figure ( b ), the trajectory is perturbed by an additive random input in the equation
Let us introduce now uncertainty in the model. We assume that, at any
time, there is a probability of 0.1 of heading into the wrong direction,
P [ f ( a )= b ]=0 . 9 , P [ f ( a )= c ]=0 . 1 ,
and so on.
The picture of that random dynamics is outlined on Fig. 4.4.
The state trajectory is no longer deterministic. That random dynamical
system, or stochastic process, is called a Markov chain. In the long time limit,
the behavior of a Markov chain is quite different from that of a deterministic
dynamical system. In that simple example, the state does not depend on the
initial state and it is straightforward to show that it is distributed according
to the uniform law in the limit of large time. That probability law is called
the stationary distribution of the Markov chain.
The dynamics of a Markov chain can be conveniently described by a matrix
representation. The state set is ordered, and a matrix is built, whose rows
are the transition probabilities: the elements of row i are the values of the
Fig. 4.4. Diagrams of random dynamical evolutions on the triangle. ( a )Peri-
odic dynamics with random disturbance ( b ) Point attractor dynamics with random
disturbance