This preview shows page 1. Sign up to view the full content.
Unformatted text preview: h (km
and hence pnh (i, j ) > 0. >0 1 , km ) > 0 for 1 m n 128 CHAPTER 4. CONTINUOUS TIME MARKOV CHAINS In discrete time a stationary distribution is a solution of ⇡ p = ⇡ . Since
there is no ﬁrst t > 0, in continuous time we need the stronger notion: ⇡ is said
to be a stationary distribution if ⇡ pt = ⇡ for all t > 0. The last condition
is di cult to check since it involves all of the pt , and as we have seen in the
previous section, the pt are not easy to compute. The next result solves these
problems by giving a test for stationarity in terms of the basic data used to
describe the chain, the matrix of transition rates
q (i, j ) j 6= i
Q(i, j ) =
where i = P j 6=i q (i, j ) is the total rate of transitions out of i. Lemma 4.3. ⇡ is a stationary distribution if and only if ⇡ Q = 0.
Why is this true? Filling in the deﬁnition of Q and rearranging, the condition
⇡ Q = 0 becomes
⇡ (k )q (k, j ) = ⇡ (j ) j
k6=j If we think of ⇡ (k ) as the amount of sand at k , the righ...
View Full Document
This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell University (Engineering School).
- Spring '10
- The Land