# To check that it is aperiodic note that p0 0 0 implies

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: q ) plays to get to N . The second term represents the contribution to the expected value from paths that end at 0, but it is hard to explain why the term has exactly this form. 1.10 Inﬁnite State Spaces* In this section we consider chains with an inﬁnite state space. The major new complication is that recurrence is not enough to guarantee the existence of a stationary distribution. 54 CHAPTER 1. MARKOV CHAINS Example 1.51. Reﬂecting random walk. Imagine a particle that moves on {0, 1, 2, . . .} according to the following rules. It takes a step to the right with probability p. It attempts to take a step to the left with probability 1 p, but if it is at 0 and tries to jump to the left, it stays at 0, since there is no 1 to jump to. In symbols, p(i, i + 1) = p p(i, i 1) = 1 p p(0, 0) = 1 when i when i 0 1 p This is a birth and death chain, so we can solve for the stationary distribution using the detailed balance equations: p⇡ (i) = (1 p)⇡ (i + 1) Rewriting this as ⇡ (i + 1) = ⇡ (i) · p/(1 ⇡ (i) =...
View Full Document

Ask a homework question - tutors are online