17 to prove this note that by considering what

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 0 for all x 2 C , then h(x) = Px (Va < Vb ). Proof. Let T = Va ^ Vb . It follows from Lemma 1.3 that Px (T < 1) = 1 for all x 2 C . (1.16) implies that h(x) = Ex h(X1 ) when x 6= a, b. The Markov property implies h(x) = Ex h(XT ^n ). We have to stop at time T because the equation is not assumed to be valid for x = a, b. Since S is finite, Px (T < 1) = 1 for all x 2 C , h(a) = 1, and h(b) = 0, it is not hard to prove that Ex h(XT ^n ) ! Px (Va < Vb ) which gives the desired result. 46 CHAPTER 1. MARKOV CHAINS 1 0.9 0.8 0.7 0.6 0.5 0.5 0.6 0.7 0.8 0.9 1 Figure 1.3: Probability the server winning a tied game as a function of the probability of winning a point. Example 1.41. Matching pennies. Bob, who has 15 pennies, and Charlie, who has 10 pennies, decide to play a game. They each flip a coin. If the two coins match, Bob gets the two pennies (for a profit of 1). If the two coins are di↵erent, then Charlie gets the two pennies. They quit when someone has all of the pennies. What is the probability Bob will win the game? The a...
View Full Document

This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell University (Engineering School).

Ask a homework question - tutors are online