Stochastic process fundamentals - Modelling with Generalised Stochastic Petri Nets - page 274

A.4.3 Sojourn and Recurrence Times

Sojourn times in CTMC states are exponentially distributed random vari-
ables, as was previously noted. In particular, denoting by SJ i the sojourn
time in state i ∈ S, it can be shown that

f SJ i (τ i ) = − q ii e q ii τ i τ i ≥ 0

(A.42)

The average sojourn time in state i is thus

E [SJ i ] = − 1
q ii

(A.43)

Moreover, if we define the forward recurrence time at time t, φ(t), as the
amount of time that the process will still spend in the state occupied at t,
or equivalently, as the length of the time period from t to the next change
of state, i.e.

φ(t) = min { θ > 0 : X(t + θ) 6 = X(t) } (A.44)
due to the memoryless PDF of sojourn times in states, it is possible to show
that

P { φ(t) > x | X(t) = i } = e q ii x x ≥ 0 (A.45)

Hence the forward recurrence time is exponentially distributed with the
same PDF as the sojourn time.
The same can be shown to be true for the backward recurrence time, defined
as the time the process has already spent in the present state at time t, pro-
vided that the set T in which the parameter t takes values is T = ( −∞ , ∞ ).
The distribution of the backward recurrence time is instead a truncated ex-
ponential in the usual case T = [0, ∞ ). In any case, both the forward and
the backward recurrence times are exponentially distributed in the limit for
t →∞ .

A.4.4 The Embedded Markov Chain
Consider the sample function of the CTMC { X(t),t ≥ 0 } shown in Fig. A.4.
The stochastic sequence { Y n ,n ≥ 0 } is a DTMC, and it is called the embed-
ded Markov chain (EMC) of the process X(t).
The transition probabilities of the EMC, r ij , are defined as

r ij = P { Y n+1 = j | Y n = i } (A.46)

Note that, for all times t,

r ij = P { X[t + φ(t)] = j | X(t) = i } (A.47)

Using the properties of CTMCs it is easy to show that

P { Y n+1 = j,θ n+1 − θ n > τ | Y n = i } = r ij e q ii τ , τ ≥ 0 (A.48)