This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: IEOR 6711, HMWK 3, Professor Sigman 1. Consider a positive recurrent Markov chain with limiting stationary distribution π . We know that π i is, by definition, the longrun proportion of time that the chain moves into state i ; π i = lim n →∞ 1 n ∑ n k =1 I { X k = i } . It is also a rate: The longrun rate (number of times per unit time) that the chain moves into state i . (a) Argue that π i is also the longrun proportion of time (rate) that the chain moves out of state i . (b) Argue that π i P i,j is the longrun rate that the chain moves from state i into state j . (c) Argue that X i π i P i,j is the longrun rate that the chain moves into state j . (d) Use the above to conclude that, for each j , π j = X i π i P i,j ; that is, π = πP . 2. George has r ≥ 3 umbrellas distributed between home and office as follows: When de parting home at the beginning of a day, if it is raining, then he takes an umbrella (if there is one) with him to the office. Similarly, when departing the office at the end of a day, if it is raining, then he takes an umbrella (if there is one) with him to home. Assume that independent of the past there is a fixed probability 0 ≤ p ≤ 1 that it rains any time he...
View
Full Document
 Fall '09
 all
 Probability theory, Markov chain, Xn

Click to edit the document details