Unformatted text preview: Stat 333 Renewal Theory In Discrete-Time Markov Chains Let X ,X 1 ,X 2 ,... be a Markov chain with state space S and transition matrix P = ( P ij ) i,j ∈ S Let i ∈ S and j ∈ S be two states where i = j . The event λ ii = “return to i ” is a renewal event. This follows from the Markov property. The event λ ij = “visit j , starting from i ” is a delayed renewal event with associated renewal event “return to j”. For convenience, in Markov chains we simplify terminology and say, for example, that state i is recurrent if “return to i ” is recurrent. Thus a recurrent state is one which, if it has occurred once, occurs over and over infinitely often in the sequence. We similarly define state i to be transient if “return to i ” is transient. Transient states occur at most a finite number of times and then disappear forever. Analogous definitions hold for states to be null recurrent or positive recurrent. Alternatively we may “start from scratch” as follows: Let T...
View Full Document
This note was uploaded on 07/12/2010 for the course STAT 333 taught by Professor Chisholm during the Winter '08 term at Waterloo.
- Winter '08
- Markov Chains