{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

Midterm Review Problems

# Midterm Review Problems - ORIE3510 Introduction to...

This preview shows pages 1–3. Sign up to view the full content.

ORIE3510 Introduction to Engineering Stochastic Processes Spring 2010 Section 6 Problem 1 Let { X n } be a DTMC with TPM P . Let A S , and suppose we want to compute P ( X enters A by time m ) = P ( X k A , for some k = 1 , . . . , m | X 0 = i ) =: β. To determine β , we define an auxiliary MC { W n } consisting of all states except those in A , plus an additional state, call it A . Further, we make the state A absorbing. Let N = min { n : X n A } and let N = if X n / A for all n . Hence, N is the first hitting time of set A for the chain { X n } . Define W n = X n , if n < N A, if n N . Hence X and W follow the same path up until the point where X enters A , at which time W goes to A and remains there forever. The TPM for W is given by Q ij = P ij , if i / A , j / A Q iA = X j A P ij , if i / A Q AA = 1 . Finally, because X n will have entered A my time m if and only if W m = A , we have β = P ( X k A , for some k = 1 , . . . , m | X 0 = i ) = P ( W m = A | X 0 = i ) = P ( W m = A | W 0 = i ) = Q m iA . Next, suppose we want to compute α := P ( X m = j, X k / A , k = 1 , . . . , m - 1 | X 0 = i ) , i, j / A . Noting that [ X m = j, X k / A , k = 1 , . . . , m - 1] = [ W m = j ] , it follows that α = Q m ij . If i / A , but j A , we condition on the penultimate step, and get α = X r/ A P ( X m = j, X m - 1 = r, X k / A , k = 1 , . . . , m - 2 | X 0 = i ) = X r/ A P ( X m = j | X m - 1 = r, X k / A , k = 1 , . . . , m - 2 , X 0 = i ) × P ( X m - 1 = r, X k / A , k = 1 , . . . , m - 2 | X 0 = i ) = X r/ A P rj P ( X m - 1 = r, X k / A , k = 1 , . . . , m - 2 | X 0 = i ) = X r/ A P rj Q m - 1 ir . Similarly, if i A we can condition on the first transition to get P ( X m = j, X k / A , k = 1 , . . . , m - 1 | X 0 - i ) = X r/ A Q m - 1 rj P i r, j / A .

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Problem 2 Consider an irreducible, positive recurrent DTMC on a state space consisting of n
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 4

Midterm Review Problems - ORIE3510 Introduction to...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online