Unformatted text preview: note that (I = (I + r + r2 + r3 + · · · ) r)(I + r + r2 + r3 + · · · ) (r + r2 + r3 + r4 · · · ) = I If T is the duration of the game then T =
Ex T = (I P y r) To solve the problem now we note that
0
1
1
.4
0
1
.4A
I r = @ .6
(I r)
0
.6
1 1 N (y ) so
1 (1.24) 1 0 19/13 10/13
= @15/13 25/13
9/13 15/13 1
4/13
10/13A
19/13 so E0 T = (15 + 25 + 10)/13 = 50/13 = 3.846 points. Here the three terms in
the sum are the expected number of visits to 1, 0, and 1.
Having worked two examples, it is time to show that we have computed the
right answer. In some cases we will want to guess and verify the answer. In
those situations it is nice to know that the solution is unique. The next result
proves this.
Theorem 1.28. Consider a Markov chain with ﬁnite state space S . Let A ⇢ S
and VA = inf {n
0 : Xn 2 A}. We suppose C = S A is ﬁnite, and that
Px (VA < 1) > 0 for any x 2 C . Suppose g (a) = 0 for all a 2 A, and that for
x 2 C we have
X
g (x) = 1 +
p(x, y )g (y )
(1.25)
y Then g (x) = Ex (VA ).
Proof. It follows from Lemma 1.3 that Ex VA < 1 for all x 2 C . (1.25) implies
that g (x) = 1 + Ex g (X1 ) when x 62 A. The Markov property implies
g (x) = Ex (T ^ n) + Ex g (XT ^n n).
We have to stop at time T because the equation is not valid for x 2 A. It
follows from the...
View
Full
Document
This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell.
 Spring '10
 DURRETT
 The Land

Click to edit the document details