51 19 exit times since getting a tails increases the

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: note that (I = (I + r + r2 + r3 + · · · ) r)(I + r + r2 + r3 + · · · ) (r + r2 + r3 + r4 · · · ) = I If T is the duration of the game then T = Ex T = (I P y r) To solve the problem now we note that 0 1 1 .4 0 1 .4A I r = @ .6 (I r) 0 .6 1 1 N (y ) so 1 (1.24) 1 0 19/13 10/13 = @15/13 25/13 9/13 15/13 1 4/13 10/13A 19/13 so E0 T = (15 + 25 + 10)/13 = 50/13 = 3.846 points. Here the three terms in the sum are the expected number of visits to 1, 0, and 1. Having worked two examples, it is time to show that we have computed the right answer. In some cases we will want to guess and verify the answer. In those situations it is nice to know that the solution is unique. The next result proves this. Theorem 1.28. Consider a Markov chain with finite state space S . Let A ⇢ S and VA = inf {n 0 : Xn 2 A}. We suppose C = S A is finite, and that Px (VA < 1) > 0 for any x 2 C . Suppose g (a) = 0 for all a 2 A, and that for x 2 C we have X g (x) = 1 + p(x, y )g (y ) (1.25) y Then g (x) = Ex (VA ). Proof. It follows from Lemma 1.3 that Ex VA < 1 for all x 2 C . (1.25) implies that g (x) = 1 + Ex g (X1 ) when x 62 A. The Markov property implies g (x) = Ex (T ^ n) + Ex g (XT ^n n). We have to stop at time T because the equation is not valid for x 2 A. It follows from the...
View Full Document

Ask a homework question - tutors are online