HW_2_Solution

HW_2_Solution - IEOR 4701 Assignment 2 Solution Summer 2011...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: IEOR 4701 Assignment 2 Solution Summer 2011 Question 1 : Suppose that ( X n : n 0) is a finite state-space Markov chain. Suppose that B is a given subset of the state-space and define T B = min { n 0 : X n B } . We assume that P i ( T B < ) = 1 for all i / B . Find systems of linear equations that are satisfied for the expectations in a) and b) together with associated boundary conditions. Express your solution using suitably defined vectors and matrices. a) g ( i ) = E i T B- 1 j =0 f ( X j ) for some function f ( ). b) e g ( i ) = E i exp- T B- 1 k =0 r ( X k ) f ( X T B ) , where r ( ) is some given non-negative function and we set by convention that a sum over an empty set is interpreted as zero. So, for instance, if j = 0 then j- 1 k =0 r ( X k ) = 0. c) Now, define B = min { n 1 : X n B } (note that B 1, whereas T B 0). Also define e h ( i ) = E i exp- B- 1 X k =0 r ( X k ) ! f ( X B ) ! . Set up similar systems of linear equations as you did in a) and b). Do you have clear boundary conditions this time? Can your solutions in a) and b) be of any help to solve for the expectations in c) explain how? Solution. a) First notice that for i B we have that T B = 0 and hence g ( i ) = 0 . If i / B, we can set up a recursion and use it to solve the problem. We have that g ( i ) = E i [ T B- 1 j =0 f ( X j )] = f ( i )+ E [ T B- 1 j =1 f ( X j )] . Using the transition proba- bilities we have that g ( i ) = f ( i )+ k S p ( i,k ) g ( k ) = f ( i )+ k B p ( i,k ) g ( k )+ k/ B p ( i,k ) g ( k ) (just split up the sum into two parts). Now we have that k B p ( i,k ) g ( k ) = 0 and hence we get g ( i ) = f ( i ) + k/ B p ( i,k ) g ( k ) . To finish, define if i / B the vector g as the vector with components g ( i ) and simi- larly for the vector f . Furthermore, if we define Q as the subset of the transition matrix P with p ( i,j ) for i,j / B we have that g = f + Q g . b) We proceed as before. If we start at a point i B then immediately T B = 0 and hence he have the conditions g ( i ) = f ( i ) . For i / B, we have the recursion g ( i ) = E i [ e T B- 1 j =0 r ( X j ) f ( X T B )] = e- r ( i ) E i [ e- T B- 1 j =1 r ( X j ) f ( X T B )] . This last expression is equal to e- r ( i ) k S p ( i,k ) g ( k ) = e- r ( i ) k B p ( i,k ) g ( k )+ e- r ( i ) k/ B p ( i,k ) g ( k ) . By the fact that g ( i ) = f ( i ) if we have i B we get that the last expression is actually e- r ( i ) X k B p ( i,k ) f ( k ) + e- r ( i ) X k/ B p ( i,k ) g ( k ) ....
View Full Document

This note was uploaded on 10/30/2011 for the course IEOR 4701 taught by Professor Karlsigma during the Summer '10 term at Columbia.

Page1 / 5

HW_2_Solution - IEOR 4701 Assignment 2 Solution Summer 2011...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online