Graphics Programs Reference

In-Depth Information

The average time spent by the process in state j in a period of fixed duration

τ at steady-state, v
j
(τ), can be obtained as the product of the steady-state

probability of state j times the duration of the observation period:

v
j
(τ) = η
j
τ

(A.21)

The average time spent by the process in state i at steady-state, between

two successive visits to state j, v
ij
, can be shown to equal the ratio of the

steady-state probabilities of states i and j:

v
ij
=
η
i

η
j

(A.22)

The term visit can be explained as follows. We say that the process visits

state i at time n if X
n
= i. The quantity v
ij
is called visit ratio since it

indicates the average number of visits to state i between two successive visits

to state j.

A.3.2

Matrix Formulation

The use of a matrix notation can be particularly convenient, since it sim-

plifies many of the above results. Define the transition probability matrix

P = [p
ij
], whose entries are the transition probabilities, as well as the vector

η = [η
i
], whose entries are the limiting probabilities.

If the finite DTMC is ergodic, the steady-state probability distribution can

be evaluated from the matrix equation:

η = η P

(A.23)

that is the matrix form of (
A.19a
). As before, the normalizing condition

steady-state probability distribution.

As a final comment on the properties of DTMCs, we remark that, as said

before, the sojourn time in state i is a geometrically distributed random

variable SJ
i
with

P
{
SJ
i
= k
}
= p
k−1

(1
−
p
ii
)

k = 1, 2,
···
(A.24)

ii

Indeed, given that the process is in state i, at each step the probability of

leaving state i is (1
−
p
ii
), and the choice is repeated independently at each

step.

The average number of steps spent in state i before going to another state

each time the process enters state i is then:

1

1
−
p
ii

E [SJ
i
] =

(A.25)

Search WWH ::

Custom Search