Problem 3 from lecture suppose coin 1 has a bias of p

This preview shows page 2 - 5 out of 6 pages.

Problem 3. (From lecture) Suppose coin 1 has a bias of p , and coin 2 has a bias of q 6 = p . I first choose coin 1 with with probability 1/2, or coin 2 with probability 1/2. Then, I repeatedly flip the chosen coin. (Note that you do not observe which coin I chose.) Let N t denote the total number of heads seen in the first t flips. Is N t a Markov chain?
2
Let’s compute P ( Y = 1 | X 1 , . . . , X t ) : P ( Y = 1 | X 1 , . . . , X t ) = P ( Y = 1 , X 1 , . . . , X t ) P ( X 1 , . . . , X t ) = P ( Y = 1) P ( X 1 , . . . , X t | Y = 1) P ( Y = 1) P ( X 1 , . . . , X t | Y = 1) + P ( Y = 2) P ( X 1 , . . . , X t | Y = 2) = P ( X 1 , . . . , X t | Y = 1) P ( X 1 , . . . , X t | Y = 1) + P ( X 1 , . . . , X t | Y = 2) = p N t (1 - p ) t - N t p N t (1 - p ) t - N t + q N t (1 - q ) t - N t , and note that it depends on X 1 , . . . , X t only through N t . This implies that (1) depends only on N t . Finally, this implies that P ( N t +1 | N t , X 1 , . . . , X t ) depends on X 1 , . . . , X t only through N t and hence N t is a Markov chain. Problem 4. Let X 0 , X 1 , X 2 , . . . be a time-homogeneous Markov chain on the state space { 0 , 1 , 2 , 3 } , and suppose a new process Y 0 , Y 1 , Y 2 , . . . is defined according to: Y n = 0 , if X n is even ; 1 , if X n is odd . Under exactly what conditions is the process Y n also a time-homogeneous Markov chain? Explain your answer.
3
Now observe that for (2) to hold true, it must be the case that we must be able to get rid of A in the expansion above. That implies that the term in [ · ] must be zero, implying that P ( Y n | X n - 1 = 0) = P ( Y n | X n - 1 = 2) p 00 + p 02 = p 20 + p 22 p 01 + p 03 = p 21 + p 23 . And similarly, for the case of Y n - 1 = 1 , we get p 10 + p 12 = p 30 + p 32 p 11 + p 13 = p 31 + p 33 .

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture