RenewalTheoryAndMarkovChains

# RenewalTheoryAndMarkovChains - Stat 333 Renewal Theory In...

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Stat 333 Renewal Theory In Discrete-Time Markov Chains Let X ,X 1 ,X 2 ,... be a Markov chain with state space S and transition matrix P = ( P ij ) i,j ∈ S Let i ∈ S and j ∈ S be two states where i = j . The event λ ii = “return to i ” is a renewal event. This follows from the Markov property. The event λ ij = “visit j , starting from i ” is a delayed renewal event with associated renewal event “return to j”. For convenience, in Markov chains we simplify terminology and say, for example, that state i is recurrent if “return to i ” is recurrent. Thus a recurrent state is one which, if it has occurred once, occurs over and over infinitely often in the sequence. We similarly define state i to be transient if “return to i ” is transient. Transient states occur at most a finite number of times and then disappear forever. Analogous definitions hold for states to be null recurrent or positive recurrent. Alternatively we may “start from scratch” as follows: Let T...
View Full Document

## This note was uploaded on 07/12/2010 for the course STAT 333 taught by Professor Chisholm during the Winter '08 term at Waterloo.

Ask a homework question - tutors are online