This preview shows pages 1–3. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Notes for Section Feb 812 ORIE 3510 Review accessibility, communication, decomposition of state space, irreducibil ity transience, recurrence, positive recurrence, periodicity, ergodicity as solidarity (class) properties limiting probabilities, stationary probabilities, longrun proportions Problem 14 P 1 : { 1 , 2 , 3 } irreducible recurrent P 2 : { 1 , 2 , 3 , 4 } irreducible recurrent P 3 : { 1 , 3 } , { 4 , 5 } recurrent closed; { 2 } transient P 4 : { 1 , 2 } , { 3 } recurrent closed; { 4 } , { 5 } transient Problem 15 Suppose the state space of a Markov chain has M states and i j . Then P ( n ) ij > 0 for some n , so we can find a path leading from i to j with positive probability, i.e. P ( X n = j,X n 1 = i n 1 ,...,X 1 = i 1  X = i ) > for some sequence of states i 1 ,...,i n 1 . We can write this probability as P ( X n = j  X n 1 = i n 1 ) ...P ( X 1 = i 1  X = i ) = P i,i 1 P i 1 ,i 2 ...P i n 1 ,j 1 by conditioning and applying the Markov property.by conditioning and applying the Markov property....
View Full
Document
 Spring '09
 RESNIK

Click to edit the document details