final_2007

# 0 are they class dic that the process eventually

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: .) Assume that X = 2. For each recurrent class, compute the probability (a) What are the recurrent classes? 0 Are they class. dic? that the process eventually enters this ap erio (b) For X0 = 2, cc) (5 pts.) Find limn→∞ P(Xn = 5 |that the Markov chain eventually enters each of the ( ompute the probabilities X0 = 2). recurrent classes. (c) Repeat (b) for X0 = 1, 3, 4, 5, 6. 2 (d) For all pairs of states (i, j ) compute limn→∞ P(Xn = j |X0 = i). (e) Find the expected value and variance of the number of transitions N up to and including the last transition out of state 2 given that the Markov chain starts out in state 2. (f ) Conditional on eventually entering the recurrent class 5,6, ﬁnd the expected value of the number of transitions M up to an including the transition into the recurrent class given that the Markov chain starts out in state 2. (d) (5 pts.) Given that X0 = 2, ﬁnd the expected time until a recurrent state is reached. (e) (5 pts.) Find the probability P(Xn−1 = 5 | Xn = 5), in the limit of large n. Solution: (a) The recurrent classes are {1} and {5, 6}. (b) Let ai denote the probability of absorption into State 1 starting from state i. Then, it is clear that a1 = 1 and a3 = 0. We also have a2 = 1 1 1 11 a1 + a2 + a3 = + a2 . 3 6 2 36 It follows that a2 = 2/5. This also implies that the probability of absorption in the class {5, 6} starting from state 2 is 3/5. (c) With probability 3/5 the chain enters the recurrent class {5, 6}. Once in the class, P (Xn = 5) will approach π5 of the Markov chain composed only of states 5 and 6, which we can determine with the aid of the birth-death equation π5 (3/4) = π6 (1/2), which yield π5 = 2/5. The ﬁnal answer is (3/5) · (2/5) = 6/25. (d) Let ti be the expected time to enter a recurrent class conditioned on being at state i. Then the equations are: t1 = t5 = t6 = t2 = t3 = t4 0 = 1 1 + t1 + 3 1 1 + t4 + 2 3 1 + t3 + 4 1 1 t2 + t3 6 2 1 t5 2 1 t6 4 which have the solution of t2 = 66/25. (e) lim P (Xn−1 = 5|Xn = 5) = lim n n P (Xn−1 = 5)P (Xn = 5|Xn−1 = 5) π5 (1/4) 1 = =. P (Xn = 5) π5 4 Problem 3: (13 points) The number of people that enter a pizzeri...
View Full Document

## This document was uploaded on 03/19/2014 for the course EECS 6.436 at MIT.

Ask a homework question - tutors are online