This preview shows pages 1–2. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: IEOR 4701 Assignment 2 Solution Summer 2011 Question 1 : Suppose that ( X n : n 0) is a finite statespace Markov chain. Suppose that B is a given subset of the statespace and define T B = min { n 0 : X n B } . We assume that P i ( T B < ) = 1 for all i / B . Find systems of linear equations that are satisfied for the expectations in a) and b) together with associated boundary conditions. Express your solution using suitably defined vectors and matrices. a) g ( i ) = E i T B 1 j =0 f ( X j ) for some function f ( ). b) e g ( i ) = E i exp T B 1 k =0 r ( X k ) f ( X T B ) , where r ( ) is some given nonnegative function and we set by convention that a sum over an empty set is interpreted as zero. So, for instance, if j = 0 then j 1 k =0 r ( X k ) = 0. c) Now, define B = min { n 1 : X n B } (note that B 1, whereas T B 0). Also define e h ( i ) = E i exp B 1 X k =0 r ( X k ) ! f ( X B ) ! . Set up similar systems of linear equations as you did in a) and b). Do you have clear boundary conditions this time? Can your solutions in a) and b) be of any help to solve for the expectations in c) explain how? Solution. a) First notice that for i B we have that T B = 0 and hence g ( i ) = 0 . If i / B, we can set up a recursion and use it to solve the problem. We have that g ( i ) = E i [ T B 1 j =0 f ( X j )] = f ( i )+ E [ T B 1 j =1 f ( X j )] . Using the transition proba bilities we have that g ( i ) = f ( i )+ k S p ( i,k ) g ( k ) = f ( i )+ k B p ( i,k ) g ( k )+ k/ B p ( i,k ) g ( k ) (just split up the sum into two parts). Now we have that k B p ( i,k ) g ( k ) = 0 and hence we get g ( i ) = f ( i ) + k/ B p ( i,k ) g ( k ) . To finish, define if i / B the vector g as the vector with components g ( i ) and simi larly for the vector f . Furthermore, if we define Q as the subset of the transition matrix P with p ( i,j ) for i,j / B we have that g = f + Q g . b) We proceed as before. If we start at a point i B then immediately T B = 0 and hence he have the conditions g ( i ) = f ( i ) . For i / B, we have the recursion g ( i ) = E i [ e T B 1 j =0 r ( X j ) f ( X T B )] = e r ( i ) E i [ e T B 1 j =1 r ( X j ) f ( X T B )] . This last expression is equal to e r ( i ) k S p ( i,k ) g ( k ) = e r ( i ) k B p ( i,k ) g ( k )+ e r ( i ) k/ B p ( i,k ) g ( k ) . By the fact that g ( i ) = f ( i ) if we have i B we get that the last expression is actually e r ( i ) X k B p ( i,k ) f ( k ) + e r ( i ) X k/ B p ( i,k ) g ( k ) ....
View
Full
Document
This note was uploaded on 10/30/2011 for the course IEOR 4701 taught by Professor Karlsigma during the Summer '10 term at Columbia.
 Summer '10
 KarlSigma

Click to edit the document details