Stochastic process fundamentals - Modelling with Generalised Stochastic Petri Nets - page 272

If the CTMC is time-homogeneous, the transition probabilities only depend
on the time difference τ = θ − t, so that we can simplify the notation by
writing

p ij (τ) = P { X(t + τ) = j | X(t) = i } (A.29)

to denote the probability that the process be in state j after an interval of
length τ, given that at present it is in state i.
Also in this case, summing p ij (τ) over all possible states j in the state space
S we must obtain as a result 1 for all values of τ.
It can be shown that the initial distribution, together with the transition
probabilities, allows the computation of the joint PDF of any set of random
variables extracted from the process. This implies that the complete prob-
abilistic description of the process only depends on the initial distribution
and on the transition probabilities.

A.4.1 Steady-state distribution

Let η i (t) = P { X(t) = i } , and define the limiting probabilities { η j ,j ∈ S } as

η j = lim

t→∞ η j (t)

(A.30)

The conditions for the existence of the steady-state distribution depend, as in
the discrete-time case, on the structure of the chain and on the classification
of states.
Define

p ij (∆t)
∆t

i 6 = j q ij = lim

∆t→0

(A.31)

p ii (∆t) − 1
∆t

q ii = lim
∆t→0

Such limits can be shown to exist under certain regularity conditions. The
intuitive interpretation of the above two quantities is as follows. Given that
the system is in state i at some time t, the probability that a transition
occurs to state j in a period of duration ∆t is q ij ∆t + o(∆t). The rate at
which the process moves from state i to state j thus equals q ij . Similarly,
− q ii ∆t + o(∆t) is the probability that the process moves out of state i
towards any other state in a period of duration ∆t. Thus, − q ii is the rate at
which the process leaves state i. We shall assume that q ij is finite ∀ i,j ∈ S.
Note that

X

q ij = 0 ∀ i ∈ S (A.32)

j∈S

Define h j to be the first hitting time of state j, i.e., the instant in which
the process enters state j for the first time after leaving the present state.
Moreover let

f ij = P { h j < ∞| X(0) = i } (A.33)