Stochastic

# 51 19 exit times since getting a tails increases the

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: note that (I = (I + r + r2 + r3 + · · · ) r)(I + r + r2 + r3 + · · · ) (r + r2 + r3 + r4 · · · ) = I If T is the duration of the game then T = Ex T = (I P y r) To solve the problem now we note that 0 1 1 .4 0 1 .4A I r = @ .6 (I r) 0 .6 1 1 N (y ) so 1 (1.24) 1 0 19/13 10/13 = @15/13 25/13 9/13 15/13 1 4/13 10/13A 19/13 so E0 T = (15 + 25 + 10)/13 = 50/13 = 3.846 points. Here the three terms in the sum are the expected number of visits to 1, 0, and 1. Having worked two examples, it is time to show that we have computed the right answer. In some cases we will want to guess and verify the answer. In those situations it is nice to know that the solution is unique. The next result proves this. Theorem 1.28. Consider a Markov chain with ﬁnite state space S . Let A ⇢ S and VA = inf {n 0 : Xn 2 A}. We suppose C = S A is ﬁnite, and that Px (VA < 1) > 0 for any x 2 C . Suppose g (a) = 0 for all a 2 A, and that for x 2 C we have X g (x) = 1 + p(x, y )g (y ) (1.25) y Then g (x) = Ex (VA ). Proof. It follows from Lemma 1.3 that Ex VA < 1 for all x 2 C . (1.25) implies that g (x) = 1 + Ex g (X1 ) when x 62 A. The Markov property implies g (x) = Ex (T ^ n) + Ex g (XT ^n n). We have to stop at time T because the equation is not valid for x 2 A. It follows from the...
View Full Document

## This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell.

Ask a homework question - tutors are online