This preview shows pages 1–2. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: IEOR 6711, HMWK 3, Professor Sigman 1. Consider a positive recurrent Markov chain with limiting stationary distribution π . We know that π i is, by definition, the longrun proportion of time that the chain moves into state i ; π i = lim n →∞ 1 n ∑ n k =1 I { X k = i } . It is also a rate: The longrun rate (number of times per unit time) that the chain moves into state i . (a) Argue that π i is also the longrun proportion of time (rate) that the chain moves out of state i . (b) Argue that π i P i,j is the longrun rate that the chain moves from state i into state j . (c) Argue that X i π i P i,j is the longrun rate that the chain moves into state j . (d) Use the above to conclude that, for each j , π j = X i π i P i,j ; that is, π = πP . 2. George has r ≥ 3 umbrellas distributed between home and office as follows: When de parting home at the beginning of a day, if it is raining, then he takes an umbrella (if there is one) with him to the office. Similarly, when departing the office at the end of a day, if it is raining, then he takes an umbrella (if there is one) with him to home. Assume that independent of the past there is a fixed probability 0 ≤ p ≤ 1 that it rains any time he...
View
Full
Document
This note was uploaded on 01/05/2010 for the course ECE 01 taught by Professor All during the Fall '09 term at Aarhus Universitet.
 Fall '09
 all

Click to edit the document details