midtermsol_2005

# Midtermsol_2005 - MS&E 221 CA Mark Peters Midterm Solutions Problem 1(a TRUE If the class were not closed then some state would be transient since

This preview shows pages 1–2. Sign up to view the full content.

Midterm Solutions CA: Mark Peters February 15, 2005 Problem 1 (a) TRUE If the class were not closed, then some state would be transient since there would be some positive probability of leaving the class and never returning from that state. If one state is transient, then all communicating states would be transient. Hence, the class would be transient. (See Fact 4 from Lecture 4). (b) FALSE For an example, take the random walk on the integers with a positive drift. Let the class be all the integers - an inﬁnite class. If P ( X n +1 = i + 1 | X n = i ) > 0 . 5 , then each state is transient. Thus, the class is transient and closed. (c) FALSE Our deﬁnition for reversibility requires that π i P i,j = π j P j,i . There is no requirement that π i = π j . Thus, P i,j doesn’t necessarily need to equal P j,i . For example, consider a Markov Chain with the following transition matrix. P = ± 1 - α α β 1 - β where α 6 = β . We can verify that the invariant distribution is π 0 = β α + β 1 = α α + β . Now, this chain will pass the requirement for reversibility but P i,j 6 = P j,i . (d) TRUE Since the Markov chain is irreducible and has an invariant distribution, it is positive recurrent. Since the MC is positive recurrent, we know that E [ T i ] < for all i . Since π i = 1 E [ T i ] , then we know that π i > 0 for all i . (e) FALSE In addition to positive recurrence, we require irreducibility for there to be a unique invariant distribution. Consider the two state chain, where

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

## This note was uploaded on 02/03/2011 for the course MS&E 221 taught by Professor Ramesh during the Winter '11 term at Stanford.

### Page1 / 5

Midtermsol_2005 - MS&E 221 CA Mark Peters Midterm Solutions Problem 1(a TRUE If the class were not closed then some state would be transient since

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online