Stochastic process fundamentals - Modelling with Generalised Stochastic Petri Nets - page 268

The average time spent by the process in state j in a period of fixed duration
τ at steady-state, v j (τ), can be obtained as the product of the steady-state
probability of state j times the duration of the observation period:

v j (τ) = η j τ (A.21)

The average time spent by the process in state i at steady-state, between
two successive visits to state j, v ij , can be shown to equal the ratio of the
steady-state probabilities of states i and j:
v ij = η i
η j

(A.22)

The term visit can be explained as follows. We say that the process visits
state i at time n if X n = i. The quantity v ij is called visit ratio since it
indicates the average number of visits to state i between two successive visits
to state j.

A.3.2 Matrix Formulation

The use of a matrix notation can be particularly convenient, since it sim-
plifies many of the above results. Define the transition probability matrix
P = [p ij ], whose entries are the transition probabilities, as well as the vector
η = [η i ], whose entries are the limiting probabilities.
If the finite DTMC is ergodic, the steady-state probability distribution can
be evaluated from the matrix equation:

η = η P (A.23)

that is the matrix form of ( A.19a ). As before, the normalizing condition
( A.19b ) must be used in conjunction with (A.23) to identify the unique
steady-state probability distribution.
As a final comment on the properties of DTMCs, we remark that, as said
before, the sojourn time in state i is a geometrically distributed random
variable SJ i with

P { SJ i = k } = p k−1

(1 − p ii ) k = 1, 2, ··· (A.24)

ii

Indeed, given that the process is in state i, at each step the probability of
leaving state i is (1 − p ii ), and the choice is repeated independently at each
step.
The average number of steps spent in state i before going to another state
each time the process enters state i is then:

1
1 − p ii

E [SJ i ] = (A.25)