This preview shows page 1. Sign up to view the full content.
Unformatted text preview: q ) plays to get to N . The second term represents the contribution
to the expected value from paths that end at 0, but it is hard to explain why
the term has exactly this form. 1.10 Inﬁnite State Spaces* In this section we consider chains with an inﬁnite state space. The major new
complication is that recurrence is not enough to guarantee the existence of a
stationary distribution. 54 CHAPTER 1. MARKOV CHAINS Example 1.51. Reﬂecting random walk. Imagine a particle that moves on
{0, 1, 2, . . .} according to the following rules. It takes a step to the right with
probability p. It attempts to take a step to the left with probability 1 p, but
if it is at 0 and tries to jump to the left, it stays at 0, since there is no 1 to
jump to. In symbols,
p(i, i + 1) = p
p(i, i 1) = 1 p
p(0, 0) = 1 when i
when i 0
1 p This is a birth and death chain, so we can solve for the stationary distribution
using the detailed balance equations:
p⇡ (i) = (1 p)⇡ (i + 1) Rewriting this as ⇡ (i + 1) = ⇡ (i) · p/(1
⇡ (i) =...
View
Full
Document
 Spring '10
 DURRETT
 The Land

Click to edit the document details