This preview shows page 1. Sign up to view the full content.
Unformatted text preview: deﬁnition of the expected value that Ex (T ^ n) " Ex T . Since
S is ﬁnite, Px (T < 1) = 1 for all x 2 C , g (a) = 0 for a 2 A, it is not hard to
see that Ex g (XT ^n ) ! 0.
Example 1.47. Waiting time for TT. Let TT T be the (random) number
of times we need to ﬂip a coin before we have gotten Tails on two consecutive
tosses. To compute the expected value of TT T we will introduce a Markov chain
with states 0, 1, 2 = the number of Tails we have in a row. 51 1.9. EXIT TIMES Since getting a Tails increases the number of Tails we have in a row by 1, but
getting a Heads sets the number of Tails we have in a row to 0, the transition
matrix is
0
1
2
0 1/2 1/2
0
1 1 /2
0
1 /2
2
0
0
1
Since we are not interested in what happens after we reach 2 we have made 2
an absorbing state. If we let V2 = min{n 0 : Xn = 2} and g (x) = Ex V2 then
one step reasoning gives
g (0) = 1 + .5g (0) + .5g (1)
g (1) = 1 + .5g (0)
Plugging the second equation into the ﬁrst gives g (0) = 1.5 + .75g (0), so
.25g (0) = 1.5 or g (0) = 6. To do this with the previous approach we note
✓
◆
✓
◆
1/2
1/2
42
I r=
(I r) 1 =
1/2
1
22
so E0 V2 = 6.
Example 1.48. Waiting time f...
View
Full
Document
This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell.
 Spring '10
 DURRETT
 The Land

Click to edit the document details