Stochastic

# To explain we begin by noting that ey ty 1 y and the

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: deﬁnition of the expected value that Ex (T ^ n) " Ex T . Since S is ﬁnite, Px (T < 1) = 1 for all x 2 C , g (a) = 0 for a 2 A, it is not hard to see that Ex g (XT ^n ) ! 0. Example 1.47. Waiting time for TT. Let TT T be the (random) number of times we need to ﬂip a coin before we have gotten Tails on two consecutive tosses. To compute the expected value of TT T we will introduce a Markov chain with states 0, 1, 2 = the number of Tails we have in a row. 51 1.9. EXIT TIMES Since getting a Tails increases the number of Tails we have in a row by 1, but getting a Heads sets the number of Tails we have in a row to 0, the transition matrix is 0 1 2 0 1/2 1/2 0 1 1 /2 0 1 /2 2 0 0 1 Since we are not interested in what happens after we reach 2 we have made 2 an absorbing state. If we let V2 = min{n 0 : Xn = 2} and g (x) = Ex V2 then one step reasoning gives g (0) = 1 + .5g (0) + .5g (1) g (1) = 1 + .5g (0) Plugging the second equation into the ﬁrst gives g (0) = 1.5 + .75g (0), so .25g (0) = 1.5 or g (0) = 6. To do this with the previous approach we note ✓ ◆ ✓ ◆ 1/2 1/2 42 I r= (I r) 1 = 1/2 1 22 so E0 V2 = 6. Example 1.48. Waiting time f...
View Full Document

## This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell.

Ask a homework question - tutors are online