This preview shows pages 1–5. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: 3/17/10 State i is recurrent if, starting in state i , the process returns to state i with probability 1, i.e., if the ultimate return to state i is a certain event State i is transient if, starting in state i , the process returns to state i with probability less than 1 i.e., if it is possible (probability greater than 0) that the process never returns to state i 2 1 0.6 0.4 1 2 1 0.6 0.4 1 State 1 and state 2 are recurrent State 2 is recurrent, but state 1 is transient Recurrent and Transient States 11 IOE 316: Notes 3 3/17/10 Recurrent and Transient States If the probability fi = 1, then state i is a recurrent state If the probability fi < 1, then state i is a transient state Thus, for a transient state, there is a nonzero probability (1 fi ) that the process will never return to that state, while, for a recurrent state, returning is a certain event 22 Let fi ( n ) = probability that, starting in state i , the process first returns to state i in n steps and = probability that, starting in state i , the process eventually returns to state i ( n = 1, 2, 3, ....... ) = P {process will some time return to state i  X 0 = i } = = 1 ) ( n n i i f f More formally : 3/17/10 33 2 1 3 2 1 4 3 State 1 and state 2 are transient, state 3 and state 4 are recurrent State 1 is transient, state 2 and state 3 are recurrent Recurrent and Transient States : Examples 33 32 23 22 13 12 P P P P P P 44 43 34 33 23 21 12 11 P P P P P P P P 3/17/10 44 2 1 3 2 1 4 3 State 1 and state 2 are transient, state 3 and state 4 are recurrent...
View
Full
Document
This note was uploaded on 03/17/2010 for the course IOE 316 taught by Professor Dolinskaya during the Winter '08 term at University of Michigan.
 Winter '08
 Dolinskaya

Click to edit the document details