In the exercise 159 we will consider waiting times

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: or HT. Let THT be the (random) number of times we need to flip a coin before we have gotten a Heads followed by a Tails. Consider Xn is Markov chain with transition probability: HH HT TH TT HH 1 /2 0 1 /2 0 If we eliminate the row and the 0 1/ 2 I r = @ 1/2 0 HT 1/2 0 1/2 0 TH 0 1 /2 0 1 /2 TT 0 1/2 0 1/2 column for HT then 1 0 0 1 0 A (I r) 1/2 1/2 01 2 1 1 = @2A 4 To compute the expected waiting time for our original problem, we note that after the first two tosses we have each of the four possibilities with probability 1/4 so 1 ETHT = 2 + (0 + 2 + 2 + 4) = 4 4 Why is ETT T = 6 while ETHT = 4? To explain we begin by noting that Ey Ty = 1/⇡ (y ) and the stationary distribution assigns probability 1/4 to each state. One can verify this and check that convergence to equilibrium is rapid by noting that all the entries of p2 are equal to 1/4. Our identity implies that EHT THT = 1 =4 ⇡ (HT ) To get from this to what we wanted to calculate, note that if we start with a H at time 1 and a T at time 0, then we have nothing that will help us in the 52 CHAPTER 1. MARKOV CHAINS future, so the expected waiting time for a HT when we start with nothing is the same. When we consider T T , our identity again gives ET T T T...
View Full Document

This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell University (Engineering School).

Ask a homework question - tutors are online