6711-09-h3 - IEOR 6711, HMWK 3, Professor Sigman 1....

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: IEOR 6711, HMWK 3, Professor Sigman 1. Consider a positive recurrent Markov chain with limiting stationary distribution π . We know that π i is, by definition, the long-run proportion of time that the chain moves into state i ; π i = lim n →∞ 1 n ∑ n k =1 I { X k = i } . It is also a rate: The long-run rate (number of times per unit time) that the chain moves into state i . (a) Argue that π i is also the long-run proportion of time (rate) that the chain moves out of state i . (b) Argue that π i P i,j is the long-run rate that the chain moves from state i into state j . (c) Argue that X i π i P i,j is the long-run rate that the chain moves into state j . (d) Use the above to conclude that, for each j , π j = X i π i P i,j ; that is, π = πP . 2. George has r ≥ 3 umbrellas distributed between home and office as follows: When de- parting home at the beginning of a day, if it is raining, then he takes an umbrella (if there is one) with him to the office. Similarly, when departing the office at the end of a day, if it is raining, then he takes an umbrella (if there is one) with him to home. Assume that independent of the past there is a fixed probability 0 ≤ p ≤ 1 that it rains any time he...
View Full Document

This note was uploaded on 01/05/2010 for the course ECE 01 taught by Professor All during the Fall '09 term at Aarhus Universitet.

Page1 / 2

6711-09-h3 - IEOR 6711, HMWK 3, Professor Sigman 1....

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online