Graphics Programs Reference
In-Depth Information
The average time spent by the process in state j in a period of fixed duration
τ at steady-state, v j (τ), can be obtained as the product of the steady-state
probability of state j times the duration of the observation period:
v j (τ) = η j τ
The average time spent by the process in state i at steady-state, between
two successive visits to state j, v ij , can be shown to equal the ratio of the
steady-state probabilities of states i and j:
v ij = η i
η j
The term visit can be explained as follows. We say that the process visits
state i at time n if X n = i. The quantity v ij is called visit ratio since it
indicates the average number of visits to state i between two successive visits
to state j.
Matrix Formulation
The use of a matrix notation can be particularly convenient, since it sim-
plifies many of the above results. Define the transition probability matrix
P = [p ij ], whose entries are the transition probabilities, as well as the vector
η = [η i ], whose entries are the limiting probabilities.
If the finite DTMC is ergodic, the steady-state probability distribution can
be evaluated from the matrix equation:
η = η P
that is the matrix form of ( A.19a ). As before, the normalizing condition
( A.19b ) must be used in conjunction with (A.23) to identify the unique
steady-state probability distribution.
As a final comment on the properties of DTMCs, we remark that, as said
before, the sojourn time in state i is a geometrically distributed random
variable SJ i with
P { SJ i = k } = p k−1
(1 p ii )
k = 1, 2, ··· (A.24)
Indeed, given that the process is in state i, at each step the probability of
leaving state i is (1 p ii ), and the choice is repeated independently at each
The average number of steps spent in state i before going to another state
each time the process enters state i is then:
1 p ii
E [SJ i ] =
Search WWH ::

Custom Search