This preview shows page 1. Sign up to view the full content.
Unformatted text preview: 0 for all x 2 C , then h(x) = Px (Va < Vb ). Proof. Let T = Va ^ Vb . It follows from Lemma 1.3 that Px (T < 1) = 1 for
all x 2 C . (1.16) implies that h(x) = Ex h(X1 ) when x 6= a, b. The Markov
h(x) = Ex h(XT ^n ). We have to stop at time T because the equation is not assumed to be valid for
x = a, b. Since S is ﬁnite, Px (T < 1) = 1 for all x 2 C , h(a) = 1, and h(b) = 0,
it is not hard to prove that Ex h(XT ^n ) ! Px (Va < Vb ) which gives the desired
result. 46 CHAPTER 1. MARKOV CHAINS
1 0.9 0.8 0.7 0.6 0.5
0.5 0.6 0.7 0.8 0.9 1 Figure 1.3: Probability the server winning a tied game as a function of the
probability of winning a point.
Example 1.41. Matching pennies. Bob, who has 15 pennies, and Charlie,
who has 10 pennies, decide to play a game. They each ﬂip a coin. If the two
coins match, Bob gets the two pennies (for a proﬁt of 1). If the two coins are
di↵erent, then Charlie gets the two pennies. They quit when someone has all
of the pennies. What is the probability Bob will win the game?
View Full Document
This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell University (Engineering School).
- Spring '10
- The Land