Stochastic

# Theorems 15 and 17 allow us to decompose the state

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: he results above for branching processes we see that if we denote the mean P number of children by µ = k kak , then If µ < 1, 0 is positive recurrent If µ = 1, 0 is null recurrent If µ > 1, 0 is transient To bring out the parallels between the three examples, note that when µ > 1 or p > 1/2 the process drifts away from 0 and is transient. When µ < 1 or p < 1/2 the process drifts toward 0 and is positive recurrent. When µ = 1 or p = 1/2, there is no drift. The process eventually hits 0 but not in ﬁnite expected time, so 0 is null recurrent. 1.11 Chapter Summary A Markov chain with transition probability p is deﬁned by the property that given the present state the rest of the past is irrelevant for predicting the future: P (Xn+1 = y |Xn = x, Xn 1 = xn 1 , . . . , X0 = x0 ) = p(x, y ) 60 CHAPTER 1. MARKOV CHAINS The m step transition probability pm (i, j ) = P (Xn+m = y |Xn = x) is the mth power of the matrix p. Recurrence and transience The ﬁrst thing we need to determine about a Markov chain is which states are recurrent and which are transient. To do this we...
View Full Document

## This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell.

Ask a homework question - tutors are online