This preview shows pages 1–2. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: ORIE3510 Introduction to Engineering Stochastic Processes Spring 2010 Section 6 Problem 1 Let { X n } be a DTMC with TPM P . Let A S , and suppose we want to compute P ( X enters A by time m ) = P ( X k A , for some k = 1 ,...,m  X = i ) =: . To determine , we define an auxiliary MC { W n } consisting of all states except those in A , plus an additional state, call it A . Further, we make the state A absorbing. Let N = min { n : X n A } and let N = if X n / A for all n . Hence, N is the first hitting time of set A for the chain { X n } . Define W n = X n , if n < N A, if n N . Hence X and W follow the same path up until the point where X enters A , at which time W goes to A and remains there forever. The TPM for W is given by Q ij = P ij , if i / A ,j / A Q iA = X j A P ij , if i / A Q AA = 1 . Finally, because X n will have entered A my time m if and only if W m = A , we have = P ( X k A , for some k = 1 ,...,m  X = i ) = P ( W m = A  X = i ) = P ( W m = A  W = i ) = Q m iA . Next, suppose we want to compute := P ( X m = j,X k / A ,k = 1 ,...,m 1  X = i ) , i,j / A . Noting that [ X m = j,X k / A ,k = 1 ,...,m 1] = [ W m = j ] , it follows that = Q m ij . If i / A , but j A , we condition on the penultimate step, and get = X r/ A P ( X m = j,X m 1 = r,X k / A ,k = 1 ,...,m 2  X = i ) = X r/ A P ( X m = j  X m 1 = r,X k / A ,k = 1 ,...,m 2 ,X = i ) P ( X m 1 = r,X k / A ,k = 1 ,...,m 2  X = i ) = X r/ A P rj P ( X m 1 = r,X k / A ,k = 1 ,...,m 2  X = i ) = X r/...
View
Full
Document
This note was uploaded on 10/29/2011 for the course ORIE 3510 taught by Professor Resnik during the Spring '09 term at Cornell University (Engineering School).
 Spring '09
 RESNIK

Click to edit the document details