final_2007

0 are they class dic that the process eventually

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: .) Assume that X = 2. For each recurrent class, compute the probability (a) What are the recurrent classes? 0 Are they class. dic? that the process eventually enters this ap erio (b) For X0 = 2, cc) (5 pts.) Find limn→∞ P(Xn = 5 |that the Markov chain eventually enters each of the ( ompute the probabilities X0 = 2). recurrent classes. (c) Repeat (b) for X0 = 1, 3, 4, 5, 6. 2 (d) For all pairs of states (i, j ) compute limn→∞ P(Xn = j |X0 = i). (e) Find the expected value and variance of the number of transitions N up to and including the last transition out of state 2 given that the Markov chain starts out in state 2. (f ) Conditional on eventually entering the recurrent class 5,6, find the expected value of the number of transitions M up to an including the transition into the recurrent class given that the Markov chain starts out in state 2. (d) (5 pts.) Given that X0 = 2, find the expected time until a recurrent state is reached. (e) (5 pts.) Find the probability P(Xn−1 = 5 | Xn = 5), in the limit of large n. Solution: (a) The recurrent classes are {1} and {5, 6}. (b) Let ai denote the probability of absorption into State 1 starting from state i. Then, it is clear that a1 = 1 and a3 = 0. We also have a2 = 1 1 1 11 a1 + a2 + a3 = + a2 . 3 6 2 36 It follows that a2 = 2/5. This also implies that the probability of absorption in the class {5, 6} starting from state 2 is 3/5. (c) With probability 3/5 the chain enters the recurrent class {5, 6}. Once in the class, P (Xn = 5) will approach π5 of the Markov chain composed only of states 5 and 6, which we can determine with the aid of the birth-death equation π5 (3/4) = π6 (1/2), which yield π5 = 2/5. The final answer is (3/5) · (2/5) = 6/25. (d) Let ti be the expected time to enter a recurrent class conditioned on being at state i. Then the equations are: t1 = t5 = t6 = t2 = t3 = t4 0 = 1 1 + t1 + 3 1 1 + t4 + 2 3 1 + t3 + 4 1 1 t2 + t3 6 2 1 t5 2 1 t6 4 which have the solution of t2 = 66/25. (e) lim P (Xn−1 = 5|Xn = 5) = lim n n P (Xn−1 = 5)P (Xn = 5|Xn−1 = 5) π5 (1/4) 1 = =. P (Xn = 5) π5 4 Problem 3: (13 points) The number of people that enter a pizzeri...
View Full Document

Ask a homework question - tutors are online