4106-08-mid-prac-solu

4106-08-mid-prac-solu - IEOR 4106 Midterm practice problems...

Info icon This preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
IEOR 4106 Midterm practice problems: Spring, 2008 1. Consider the rat in the open maze (state space S = { 0 , 1 , 2 , 3 , 4 } ). Given that the rat starts in cell 1, what is the expected total number of visits to cell 2 before the rat escapes? SOLUTION : For i and j any transient states (from 1 , 2 , 3 , 4), let s i,j denote the expected total number of visits to state j , given X 0 = i . S = ( s i,j ) is a 4 × 4 matrix. We are to find the value of s 1 , 2 . If we condition on the first move, X 1 = 2 or X 1 = 3, we obtain by the Markov property, s 1 , 2 = s 2 , 2 P 1 , 2 + s 3 , 2 P 1 , 3 = (1 / 2)( s 2 , 2 + s 3 , 2 ) . We can continue this type of conditioning to obtain a set of 4 linear equations for the 4 unknowns s i, 2 , i = 1 , 2 , 3 , 4: s 1 , 2 = s 2 , 2 P 1 , 2 + s 3 , 2 P 1 , 3 = (1 / 2)( s 2 , 2 + s 3 , 2 ) s 2 , 2 = 1 + s 1 , 2 P 2 , 1 + s 4 , 2 P 2 , 4 = 1 + (1 / 2)( s 1 , 2 + s 4 , 2 ) s 3 , 2 = s 1 , 2 P 3 , 1 + s 4 , 2 P 3 , 4 = (1 / 2)( s 1 , 2 + s 4 , 2 ) s 4 , 2 = s 2 , 2 P 4 , 2 + s 3 , 2 P 4 , 3 = (1 / 3)( s 2 , 2 + s 3 , 2 ) . Solving (left to reader) leads to the solution ( s 1 , 2 , s 2 , 2 , s 3 , 2 , s 4 , 2 ) = (3 , 3 . 5 , 2 . 5 , 2); s 1 , 2 = 3. More elegantly: Letting P T denote the 4 × 4 transition matrix for (only) states 1 , 2 , 3 , 4 (the transient states), P T = 0 1 2 1 2 0 1 2 0 0 1 2 1 2 0 0 1 2 0 1 3 1 3 0 , we see that in fact the above equations, when expanded to include all 16 of the s i,j , in matrix form become: S = I + SP T , where I is the identity matrix. Rewriting this as S - SP T = I, or S ( I - P T ) = I , we thus can solve for S all a once: S = ( I - P T ) - 1 .
Image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern