Stochastic

# Chapter 2 poisson processes 21 exponential

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: be the time of the ﬁrst visit to y and let hN (x) = Px (VN < V0 ). By considering what happens on the ﬁrst step, we can write hN (x) = px hN (x + 1) + rx hN (x) + qx hN (x 1) Set hN (1) = cN and solve this equation to conclude that 0 is recurrent if and P1 Qy 1 Q0 only if y=1 x=1 qx /px = 1 where by convention x=1 = 1. 1.71. To see what the conditions in the last problem say we will now consider ↵ some concrete examples. Let px = 1/2, qx = e cx /2, rx = 1/2 qx for x 1 and p0 = 1. For large x, qx ⇡ (1 cx ↵ )/2, but the exponential formulation keeps the probabilities nonnegative and makes the problem easier to solve. Show that the chain is recurrent if ↵ > 1 or if ↵ = 1 and c 1 but is transient otherwise. 1.72. Consider the Markov chain with state space {0, 1, 2, . . .} and transition probability ✓ ◆ 1 1 1 for m 0 p(m, m + 1) = 2 m+2 ✓ ◆ 1 1 p(m, m 1) = 1+ for m 1 2 m+2 and p(0, 0) = 1 p(0, 1) = 3/4. Find the stationary distribution ⇡ . 1.73. Consider the Markov chain with state space {1, 2, . . .} and transition probability p(m, m + 1) = m/(2m + 2) f...
View Full Document

## This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell.

Ask a homework question - tutors are online