Unformatted text preview: Theorem 1.27.
5.17. Expectations of hitting times. Consider a Markov chain state space S .
Let A ⇢ S and suppose that C = S A is a ﬁnite set. Let VA = min{n 0 :
Xn 2 A} be the time of the ﬁrst visit to A. Suppose that g (x) = 0 for x 2 A,
while for x 2 C we have
X
g (x) = 1 +
p(x, y )g (y )
y (a) Show that g (XVA ^n ) + (VA ^ n) is a martingale. (b) Conclude that if
Px (VA < 1) > 0 for all x 2 C then g (x) = Ex VA , giving a proof of Theorem
1.28.
5.18. Lyapunov functions. Let Xn be an irreducible Markov chain with state
space {0, 1, 2, . . .} and let
0 be a function with limx!1 (x) = 1, and
Ex (X1 ) (x) when x K . Then Xn is recurrent. This abstract result is
often useful for proving recurrence in many chains that come up in applications
and in many cases it is enough to consider (x) = x.
5.19. GI/G/1 queue. Let ⇠1 , ⇠2 , . . . be independent with distribution F and
Let ⌘1 , ⌘2 , . . . be independent with distribution G. Deﬁne a Markov chain by
Xn+1 = (Xn + ⇠n ⌘n+1 )+ where y + = max{y, 0}. Here Xn is the workload in the queue at the time of
arrival of the nth customer, not counting the service time of the nth customer,
⌘n . The amount of wo...
View
Full
Document
This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell.
 Spring '10
 DURRETT
 The Land

Click to edit the document details