This preview shows page 1. Sign up to view the full content.
Unformatted text preview: 1 2/3 1
1/2 1 The answer is given by the fourth row of A 1 : (0.5, 0.1875, 0.1875, 0.125) = (1/2, 3/16, 3/16, 1/8)
Thus the long run fraction of time the player hits a shot is
⇡ (HH ) + ⇡ (M H ) = 0.6875 = 11/36.
At this point we have a procedure for computing stationary distribution but
it is natural to ask: Is the matrix always invertible? Is the ⇡ we compute always
0? We will prove this in Section 1.7 uing probaiblistic methods. Here we will
give an elementary proof based on linear algebra. 22 CHAPTER 1. MARKOV CHAINS Theorem 1.14. Suppose that the k ⇥ k transition matrix p is irreducible. Then
there is a unique solution to ⇡ p = ⇡ with x ⇡x = 1 and we have ⇡x > 0 for
all x. Proof. Let I be the identity matrix. Since the rows of p I add to 0, the rank
of the matrix is k 1 and there is a vector v so that vp = v .
Let q = (I + p)/2 be the lazy chain that stays put with probability 1/2
and otherwise takes a step according to p. Since vp = p we have vq = v . Let
View Full Document
This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell University (Engineering School).
- Spring '10
- The Land