{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

HW_2_Solution

# HW_2_Solution - IEOR 4701 Assignment 2 Solution Summer 2011...

This preview shows pages 1–3. Sign up to view the full content.

IEOR 4701 Assignment 2 Solution Summer 2011 Question 1 : Suppose that ( X n : n 0) is a finite state-space Markov chain. Suppose that B is a given subset of the state-space and define T B = min { n 0 : X n B } . We assume that P i ( T B < ) = 1 for all i / B . Find systems of linear equations that are satisfied for the expectations in a) and b) together with associated boundary conditions. Express your solution using suitably defined vectors and matrices. a) g ( i ) = E i T B - 1 j =0 f ( X j ) for some function f ( · ). b) e g ( i ) = E i exp - T B - 1 k =0 r ( X k ) f ( X T B ) , where r ( · ) is some given non-negative function and we set by convention that a sum over an empty set is interpreted as zero. So, for instance, if j = 0 then j - 1 k =0 r ( X k ) = 0. c) Now, define τ B = min { n 1 : X n B } (note that τ B 1, whereas T B 0). Also define e h ( i ) = E i exp - τ B - 1 X k =0 r ( X k ) ! f ( X τ B ) ! . Set up similar systems of linear equations as you did in a) and b). Do you have clear boundary conditions this time? Can your solutions in a) and b) be of any help to solve for the expectations in c) – explain how? Solution. a) First notice that for i B we have that T B = 0 and hence g ( i ) = 0 . If i / B, we can set up a recursion and use it to solve the problem. We have that g ( i ) = E i [ T B - 1 j =0 f ( X j )] = f ( i ) + E [ T B - 1 j =1 f ( X j )] . Using the transition proba- bilities we have that g ( i ) = f ( i ) + k S p ( i, k ) g ( k ) = f ( i ) + k B p ( i, k ) g ( k ) + k / B p ( i, k ) g ( k ) (just split up the sum into two parts). Now we have that k B p ( i, k ) g ( k ) = 0 and hence we get g ( i ) = f ( i ) + k / B p ( i, k ) g ( k ) . To finish, define if i / B the vector g as the vector with components g ( i ) and simi- larly for the vector f . Furthermore, if we define Q as the subset of the transition matrix P with p ( i, j ) for i, j / B we have that g = f + Q g . b) We proceed as before. If we start at a point i B then immediately T B = 0 and hence he have the conditions ˜ g ( i ) = f ( i ) . For i / B, we have the recursion ˜ g ( i ) = E i [ e T B - 1 j =0 r ( X j ) f ( X T B )] = e - r ( i ) E i [ e - T B - 1 j =1 r ( X j ) f ( X T B )] . This last expression is equal to e - r ( i ) k S p ( i, k g ( k ) = e - r ( i ) k B p ( i, k g ( k )+ e - r ( i ) k / B p ( i, k g ( k ) . By the fact that ˜ g ( i ) = f ( i ) if we have i B we get that the last expression is actually e - r ( i ) X k B p ( i, k ) f ( k ) + e - r ( i ) X k / B p ( i, k g ( k ) . 1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
IEOR 4701 Assignment 2 Solution Summer 2011 Hence, for i / B, we set ˜ g as the vector whose components are ˜ g ( i ) , f as the vector whose components are f ( i ) and r as the vector with components e - r ( i ) .
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}