Midterm Review Problems - ORIE3510 Introduction to...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ORIE3510 Introduction to Engineering Stochastic Processes Spring 2010 Section 6 Problem 1 Let { X n } be a DTMC with TPM P . Let A S , and suppose we want to compute P ( X enters A by time m ) = P ( X k A , for some k = 1 ,...,m | X = i ) =: . To determine , we define an auxiliary MC { W n } consisting of all states except those in A , plus an additional state, call it A . Further, we make the state A absorbing. Let N = min { n : X n A } and let N = if X n / A for all n . Hence, N is the first hitting time of set A for the chain { X n } . Define W n = X n , if n < N A, if n N . Hence X and W follow the same path up until the point where X enters A , at which time W goes to A and remains there forever. The TPM for W is given by Q ij = P ij , if i / A ,j / A Q iA = X j A P ij , if i / A Q AA = 1 . Finally, because X n will have entered A my time m if and only if W m = A , we have = P ( X k A , for some k = 1 ,...,m | X = i ) = P ( W m = A | X = i ) = P ( W m = A | W = i ) = Q m iA . Next, suppose we want to compute := P ( X m = j,X k / A ,k = 1 ,...,m- 1 | X = i ) , i,j / A . Noting that [ X m = j,X k / A ,k = 1 ,...,m- 1] = [ W m = j ] , it follows that = Q m ij . If i / A , but j A , we condition on the penultimate step, and get = X r/ A P ( X m = j,X m- 1 = r,X k / A ,k = 1 ,...,m- 2 | X = i ) = X r/ A P ( X m = j | X m- 1 = r,X k / A ,k = 1 ,...,m- 2 ,X = i ) P ( X m- 1 = r,X k / A ,k = 1 ,...,m- 2 | X = i ) = X r/ A P rj P ( X m- 1 = r,X k / A ,k = 1 ,...,m- 2 | X = i ) = X r/...
View Full Document

This note was uploaded on 10/29/2011 for the course ORIE 3510 taught by Professor Resnik during the Spring '09 term at Cornell University (Engineering School).

Page1 / 4

Midterm Review Problems - ORIE3510 Introduction to...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online