Unformatted text preview: be the time of the
ﬁrst visit to y and let hN (x) = Px (VN < V0 ). By considering what happens on
the ﬁrst step, we can write
hN (x) = px hN (x + 1) + rx hN (x) + qx hN (x 1) Set hN (1) = cN and solve this equation to conclude that 0 is recurrent if and
P1 Qy 1
Q0
only if y=1 x=1 qx /px = 1 where by convention x=1 = 1. 1.71. To see what the conditions in the last problem say we will now consider
↵
some concrete examples. Let px = 1/2, qx = e cx /2, rx = 1/2 qx for
x
1 and p0 = 1. For large x, qx ⇡ (1 cx ↵ )/2, but the exponential
formulation keeps the probabilities nonnegative and makes the problem easier
to solve. Show that the chain is recurrent if ↵ > 1 or if ↵ = 1 and c 1 but is
transient otherwise.
1.72. Consider the Markov chain with state space {0, 1, 2, . . .} and transition
probability
✓
◆
1
1
1
for m 0
p(m, m + 1) =
2
m+2
✓
◆
1
1
p(m, m 1) =
1+
for m 1
2
m+2 and p(0, 0) = 1 p(0, 1) = 3/4. Find the stationary distribution ⇡ . 1.73. Consider the Markov chain with state space {1, 2, . . .} and transition
probability
p(m, m + 1) = m/(2m + 2) f...
View
Full
Document
This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell.
 Spring '10
 DURRETT
 The Land

Click to edit the document details