Stochastic

To explain we begin by noting that ey ty 1 y and the

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: definition of the expected value that Ex (T ^ n) " Ex T . Since S is finite, Px (T < 1) = 1 for all x 2 C , g (a) = 0 for a 2 A, it is not hard to see that Ex g (XT ^n ) ! 0. Example 1.47. Waiting time for TT. Let TT T be the (random) number of times we need to flip a coin before we have gotten Tails on two consecutive tosses. To compute the expected value of TT T we will introduce a Markov chain with states 0, 1, 2 = the number of Tails we have in a row. 51 1.9. EXIT TIMES Since getting a Tails increases the number of Tails we have in a row by 1, but getting a Heads sets the number of Tails we have in a row to 0, the transition matrix is 0 1 2 0 1/2 1/2 0 1 1 /2 0 1 /2 2 0 0 1 Since we are not interested in what happens after we reach 2 we have made 2 an absorbing state. If we let V2 = min{n 0 : Xn = 2} and g (x) = Ex V2 then one step reasoning gives g (0) = 1 + .5g (0) + .5g (1) g (1) = 1 + .5g (0) Plugging the second equation into the first gives g (0) = 1.5 + .75g (0), so .25g (0) = 1.5 or g (0) = 6. To do this with the previous approach we note ✓ ◆ ✓ ◆ 1/2 1/2 42 I r= (I r) 1 = 1/2 1 22 so E0 V2 = 6. Example 1.48. Waiting time f...
View Full Document

This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell.

Ask a homework question - tutors are online