22 chapter 1 markov chains theorem 114 suppose that

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 1 2/3 1 1/3 11 0 1/2 1 The answer is given by the fourth row of A 1 : (0.5, 0.1875, 0.1875, 0.125) = (1/2, 3/16, 3/16, 1/8) Thus the long run fraction of time the player hits a shot is ⇡ (HH ) + ⇡ (M H ) = 0.6875 = 11/36. At this point we have a procedure for computing stationary distribution but it is natural to ask: Is the matrix always invertible? Is the ⇡ we compute always 0? We will prove this in Section 1.7 uing probaiblistic methods. Here we will give an elementary proof based on linear algebra. 22 CHAPTER 1. MARKOV CHAINS Theorem 1.14. Suppose that the k ⇥ k transition matrix p is irreducible. Then P there is a unique solution to ⇡ p = ⇡ with x ⇡x = 1 and we have ⇡x > 0 for all x. Proof. Let I be the identity matrix. Since the rows of p I add to 0, the rank of the matrix is k 1 and there is a vector v so that vp = v . Let q = (I + p)/2 be the lazy chain that stays put with probability 1/2 and otherwise takes a step according to p. Since vp = p we have vq = v . Let r...
View Full Document

This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell University (Engineering School).

Ask a homework question - tutors are online