Unformatted text preview: .) Assume that X = 2. For each recurrent class, compute the probability (a) What are the recurrent classes? 0 Are they class. dic?
that the process eventually enters this ap erio (b) For X0 = 2, cc) (5 pts.) Find limn→∞ P(Xn = 5 that the Markov chain eventually enters each of the
( ompute the probabilities X0 = 2).
recurrent classes.
(c) Repeat (b) for X0 = 1, 3, 4, 5, 6. 2 (d) For all pairs of states (i, j ) compute limn→∞ P(Xn = j X0 = i). (e) Find the expected value and variance of the number of transitions N up to and including
the last transition out of state 2 given that the Markov chain starts out in state 2.
(f ) Conditional on eventually entering the recurrent class 5,6, ﬁnd the expected value of the
number of transitions M up to an including the transition into the recurrent class given that
the Markov chain starts out in state 2. (d) (5 pts.) Given that X0 = 2, ﬁnd the expected time until a recurrent state is reached. (e) (5 pts.) Find the probability P(Xn−1 = 5  Xn = 5), in the limit of large n.
Solution:
(a) The recurrent classes are {1} and {5, 6}.
(b) Let ai denote the probability of absorption into State 1 starting from state i. Then,
it is clear that a1 = 1 and a3 = 0. We also have
a2 = 1
1
1
11
a1 + a2 + a3 = + a2 .
3
6
2
36 It follows that a2 = 2/5. This also implies that the probability of absorption in the
class {5, 6} starting from state 2 is 3/5.
(c) With probability 3/5 the chain enters the recurrent class {5, 6}. Once in the class,
P (Xn = 5) will approach π5 of the Markov chain composed only of states 5 and
6, which we can determine with the aid of the birthdeath equation π5 (3/4) =
π6 (1/2), which yield π5 = 2/5. The ﬁnal answer is (3/5) · (2/5) = 6/25.
(d) Let ti be the expected time to enter a recurrent class conditioned on being at state i.
Then the equations are: t1 = t5 = t6 = t2 = t3 = t4 0 = 1
1 + t1 +
3
1
1 + t4 +
2
3
1 + t3 +
4 1
1
t2 + t3
6
2
1
t5
2
1
t6
4 which have the solution of t2 = 66/25.
(e)
lim P (Xn−1 = 5Xn = 5) = lim
n n P (Xn−1 = 5)P (Xn = 5Xn−1 = 5)
π5 (1/4)
1
=
=.
P (Xn = 5) π5
4 Problem 3: (13 points) The number of people that enter a pizzeri...
View
Full
Document
This document was uploaded on 03/19/2014 for the course EECS 6.436 at MIT.
 Fall '08
 DavidGarnarnik

Click to edit the document details