IOE+316+Notes+3 - 3/17/10 State i is recurrent if, starting...

Info iconThis preview shows pages 1–5. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 3/17/10 State i is recurrent if, starting in state i , the process returns to state i with probability 1, i.e., if the ultimate return to state i is a certain event State i is transient if, starting in state i , the process returns to state i with probability less than 1 i.e., if it is possible (probability greater than 0) that the process never returns to state i 2 1 0.6 0.4 1 2 1 0.6 0.4 1 State 1 and state 2 are recurrent State 2 is recurrent, but state 1 is transient Recurrent and Transient States 11 IOE 316: Notes 3 3/17/10 Recurrent and Transient States If the probability fi = 1, then state i is a recurrent state If the probability fi < 1, then state i is a transient state Thus, for a transient state, there is a non-zero probability (1 fi ) that the process will never return to that state, while, for a recurrent state, returning is a certain event 22 Let fi ( n ) = probability that, starting in state i , the process first returns to state i in n steps and = probability that, starting in state i , the process eventually returns to state i ( n = 1, 2, 3, ....... ) = P {process will some time return to state i | X 0 = i } = = 1 ) ( n n i i f f More formally : 3/17/10 33 2 1 3 2 1 4 3 State 1 and state 2 are transient, state 3 and state 4 are recurrent State 1 is transient, state 2 and state 3 are recurrent Recurrent and Transient States : Examples 33 32 23 22 13 12 P P P P P P 44 43 34 33 23 21 12 11 P P P P P P P P 3/17/10 44 2 1 3 2 1 4 3 State 1 and state 2 are transient, state 3 and state 4 are recurrent...
View Full Document

This note was uploaded on 03/17/2010 for the course IOE 316 taught by Professor Dolinskaya during the Winter '08 term at University of Michigan.

Page1 / 13

IOE+316+Notes+3 - 3/17/10 State i is recurrent if, starting...

This preview shows document pages 1 - 5. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online