This preview shows page 1. Sign up to view the full content.
Unformatted text preview: Stat 333 Renewal Theory In Discrete-Time Markov Chains Let X ,X 1 ,X 2 ,... be a Markov chain with state space S and transition matrix P = ( P ij ) i,j S Let i S and j S be two states where i = j . The event ii = return to i is a renewal event. This follows from the Markov property. The event ij = visit j , starting from i is a delayed renewal event with associated renewal event return to j. For convenience, in Markov chains we simplify terminology and say, for example, that state i is recurrent if return to i is recurrent. Thus a recurrent state is one which, if it has occurred once, occurs over and over infinitely often in the sequence. We similarly define state i to be transient if return to i is transient. Transient states occur at most a finite number of times and then disappear forever. Analogous definitions hold for states to be null recurrent or positive recurrent. Alternatively we may start from scratch as follows: Let T...
View Full Document
- Winter '08
- Markov Chains