# After a score unc needs an average of 15 minute to

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: h (km and hence pnh (i, j ) > 0. >0 1 , km ) > 0 for 1 m n 128 CHAPTER 4. CONTINUOUS TIME MARKOV CHAINS In discrete time a stationary distribution is a solution of ⇡ p = ⇡ . Since there is no ﬁrst t > 0, in continuous time we need the stronger notion: ⇡ is said to be a stationary distribution if ⇡ pt = ⇡ for all t > 0. The last condition is di cult to check since it involves all of the pt , and as we have seen in the previous section, the pt are not easy to compute. The next result solves these problems by giving a test for stationarity in terms of the basic data used to describe the chain, the matrix of transition rates ( q (i, j ) j 6= i Q(i, j ) = j=i i where i = P j 6=i q (i, j ) is the total rate of transitions out of i. Lemma 4.3. ⇡ is a stationary distribution if and only if ⇡ Q = 0. Why is this true? Filling in the deﬁnition of Q and rearranging, the condition ⇡ Q = 0 becomes X ⇡ (k )q (k, j ) = ⇡ (j ) j k6=j If we think of ⇡ (k ) as the amount of sand at k , the righ...
View Full Document

## This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell University (Engineering School).

Ask a homework question - tutors are online