This preview shows page 1. Sign up to view the full content.
Unformatted text preview: or HT. Let THT be the (random) number of
times we need to ﬂip a coin before we have gotten a Heads followed by a Tails.
Consider Xn is Markov chain with transition probability:
HH
HT
TH
TT HH
1 /2
0
1 /2
0 If we eliminate the row and the
0
1/ 2
I r = @ 1/2
0 HT
1/2
0
1/2
0 TH
0
1 /2
0
1 /2 TT
0
1/2
0
1/2 column for HT then
1
0
0
1
0 A (I r)
1/2 1/2 01
2
1
1 = @2A
4 To compute the expected waiting time for our original problem, we note that
after the ﬁrst two tosses we have each of the four possibilities with probability
1/4 so
1
ETHT = 2 + (0 + 2 + 2 + 4) = 4
4
Why is ETT T = 6 while ETHT = 4? To explain we begin by noting that
Ey Ty = 1/⇡ (y ) and the stationary distribution assigns probability 1/4 to each
state. One can verify this and check that convergence to equilibrium is rapid
by noting that all the entries of p2 are equal to 1/4. Our identity implies that
EHT THT = 1
=4
⇡ (HT ) To get from this to what we wanted to calculate, note that if we start with a
H at time 1 and a T at time 0, then we have nothing that will help us in the 52 CHAPTER 1. MARKOV CHAINS future, so the expected waiting time for a HT when we start with nothing is
the same.
When we consider T T , our identity again gives
ET T T T...
View
Full
Document
This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell University (Engineering School).
 Spring '10
 DURRETT
 The Land

Click to edit the document details