# 17 to prove this note that by considering what

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 0 for all x 2 C , then h(x) = Px (Va < Vb ). Proof. Let T = Va ^ Vb . It follows from Lemma 1.3 that Px (T < 1) = 1 for all x 2 C . (1.16) implies that h(x) = Ex h(X1 ) when x 6= a, b. The Markov property implies h(x) = Ex h(XT ^n ). We have to stop at time T because the equation is not assumed to be valid for x = a, b. Since S is ﬁnite, Px (T < 1) = 1 for all x 2 C , h(a) = 1, and h(b) = 0, it is not hard to prove that Ex h(XT ^n ) ! Px (Va < Vb ) which gives the desired result. 46 CHAPTER 1. MARKOV CHAINS 1 0.9 0.8 0.7 0.6 0.5 0.5 0.6 0.7 0.8 0.9 1 Figure 1.3: Probability the server winning a tied game as a function of the probability of winning a point. Example 1.41. Matching pennies. Bob, who has 15 pennies, and Charlie, who has 10 pennies, decide to play a game. They each ﬂip a coin. If the two coins match, Bob gets the two pennies (for a proﬁt of 1). If the two coins are di↵erent, then Charlie gets the two pennies. They quit when someone has all of the pennies. What is the probability Bob will win the game? The a...
View Full Document

## This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell University (Engineering School).

Ask a homework question - tutors are online