Stochastic process fundamentals - Modelling with Generalised Stochastic Petri Nets - page 267

A subset A of the state space S is said to be closed if no transition is possible
from states in A to states in A (the complement of set A), i.e., if
X X

p ij = 0 (A.15)

j∈A

i∈A

A DTMC is said to be irreducible if no proper subset of S is closed. This
implies that every state of the chain can be reached from every other state.
It can be shown that all states of a finite and irreducible DTMC are re-
current. Note that an irreducible chain cannot comprise absorbing states.
Moreover, either all states are aperiodic, or they all have the same period d.
In the former case the chain itsef is said to be aperiodic, and it is said to be
periodic with period d in the latter case.
A distribution { z i , i ∈ S } defined on the DTMC states is said to be sta-
tionary if

X

z j p ji ∀ i ∈ S

z i = (A.16)

j∈S

that is if, once this distribution is reached, then for all successive steps this
is the distribution of the process.
Define the limiting probabilities { η j , j ∈ S } as

η j = lim

n→∞ η j (n)

(A.17)

It can be shown that for all finite, aperiodic, homogeneous DTMCs the
above limits exist. Moreover, if the DTMC is irreducible, the limits
are independent of the initial distribution { η j (0),j ∈ S } , and since all states
are recurrent,

η j > 0 ∀ j ∈ S (A.18)
and the limiting probabilities { η j ,j ∈ S } form a stationary distribution.
In the latter case all states are said to be ergodic, and the DTMC itself is
said to be ergodic; moreover the limiting probabilities { η j ,j ∈ S } are unique.
They can be obtained by solving the system of linear equations

P

i∈S η i p ij ∀ j ∈ S

η j = (a)

(A.19)

P
j∈S η j = 1

(b)

Furthermore, in this case a very simple relation exists between the state
limiting probabilities and the state mean recurrence times:

1
M j

η j = (A.20)

The limiting probabilities af an ergodic DTMC are often called also equilib-
rium or steady-state probabilities.