{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

6711-09-h3-solu

# 6711-09-h3-solu - IEOR 6711 HMWK 3 Solutions Professor...

This preview shows pages 1–2. Sign up to view the full content.

IEOR 6711, HMWK 3 Solutions, Professor Sigman 1. Consider a positive recurrent Markov chain with limiting stationary distribution π . We know that π i is, by definition, the long-run proportion of time that the chain moves into state i ; π i = lim n →∞ 1 n n k =1 I { X k = i } . It is also a rate: The long-run rate (number of times per unit time) that the chain moves into state i . (a) Argue that π i is also the long-run proportion of time (rate) that the chain moves out of state i . SOLUTION: Every time the chain visits state i , it must leave state i (at some time soon after) so as to be able to return to it again (which it must by recurrence). Thus there is a one-to-one correspondence between visits into state i and visits out of state i . In fact the number of visits into state i during the first n units of time (denote by N + i ( n )) is equal to the number of visits out of state i (denote by N - i ( n )), plus or minus 1: N + i ( n ) = N - i ( n ) ± 1 Dividing by n and taking the limit as n → ∞ thus yields the two limits (rates) as identical. (This basic fact has nothing to do with Markov chains, it is just a basic fact about functions/paths.) So, “the rate into a state must equal the rate out of that state” (b) Argue that π i P i,j is the long-run rate that the chain moves from state i into state j . SOLUTION: π is the rate out of state i , and independent of the past , whenever the chain is in state i it moves next to state j with probability P i,j by the Markov property. (c) Argue that X i π i P i,j is the long-run rate that the chain moves into state j . SOLUTION: Summing up (b) over all states i yields, for each fixed j ∈ S , that the long-run rate into state j is X i π i P i,j . (d) Use the above to conclude that, for each j , π j = X i π i P i,j ; that is, π = πP . SOLUTION: It says that “The rate out of state j equals the rate into state j , for each j ” which we know must be true from (a). 2. George has r 3 umbrellas distributed between home and office as follows: When de- parting home at the beginning of a day, if it is raining, then he takes an umbrella (if there is one) with him to the office. Similarly, when departing the office at the end of a day, if it is raining, then he takes an umbrella (if there is one) with him to home. Assume that independent of the past there is a fixed probability 0 p

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}