{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

IOE+316+Notes+3

# IOE+316+Notes+3 - Recurrent and Transient States IOE 316...

This preview shows pages 1–5. Sign up to view the full content.

3/17/10 State i is recurrent if, starting in state i , the process returns to state i with probability 1, i.e., if the ultimate return to state i is a certain event State i is transient if, starting in state i , the process returns to state i with probability less than 1 i.e., if it is possible (probability greater than 0) that the process never returns to state i 2 1 0.6 0.4 1 2 1 0.6 0.4 1 State 1 and state 2 are recurrent State 2 is recurrent, but state 1 is transient Recurrent and Transient States 11 IOE 316: Notes 3

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
3/17/10 Recurrent and Transient States If the probability fi = 1, then state i is a recurrent state If the probability fi < 1, then state i is a transient state Thus, for a transient state, there is a non-zero probability (1 – fi ) that the process will never return to that state, while, for a recurrent state, returning is a certain event 22 Let fi ( n ) = probability that, starting in state i , the process first returns to state i in n steps and = probability that, starting in state i , the process eventually returns to state i ( n = 1, 2, 3, ....... ) = P {process will some time return to state i | X 0 = i } = = 1 ) ( n n i i f f More formally :
3/17/10 33 2 1 3 2 1 4 3 State 1 and state 2 are transient, state 3 and state 4 are recurrent State 1 is transient, state 2 and state 3 are recurrent Recurrent and Transient States : Examples 33 32 23 22 13 12 0 0 0 P P P P P P 44 43 34 33 23 21 12 11 0 0 0 0 0 0 0 0 P P P P P P P P

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
3/17/10 44 2 1 3 2 1 4 3 State 1 and state 2 are transient, state 3 and state 4 are recurrent State 1 is transient,
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 13

IOE+316+Notes+3 - Recurrent and Transient States IOE 316...

This preview shows document pages 1 - 5. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online