Answer a a state a b is feasible if it belongs to c a

This preview shows page 2 - 4 out of 10 pages.

Answer: (a) A state ( a, b ) is feasible if it belongs to C = { ( a, b ) : a, b ∈ Z , 0 a, b 4 , ( a, b ) ̸ = (4 , 4) } . If ( a, b ) C and a, b 3 then P ( a,b )( c,d ) = p c = a + 1 , d = b 1 p c = a, d = b + 1 0 otherwise If ( a, b ) C and a = 4 or b = 4 then P ( a,b )( c,d ) = 1 if ( c, d ) = ( a, b ), and 0 otherwise.
(b) Let p ( a,b ) be the probability of finishing series in 5 games from being in state ( a, b ) currently.
2
Copyright © Peter Glynn All rights reserved. MS&E 221 Spring 2020 (c) Yes. Note that if we ever reach a state where both teams have won at least 2 matches, we can never end up wining in a total of 5 matches. Using this fact, we can use the boundary condition that p (2 , 3) = p (3 , 2) = 0, and remove all states ( a, b ) C with a + b > 5. (If you observe closely, you can remove even more states.) (d) Let T ( a,b ) be the expected length of series that remains, starting in state ( a, b ). If you are at an absorbing state already, the remaining length of series is 0; hence, ( a, b ) C for which a = 4 or b = 4, T ( a,b ) = 0. For any other state ( a, b ) C , it follows from first transition analysis, T ( a,b ) = 1 + p × T ( a +1 ,b ) + (1 p ) × T ( a,b +1) . Solve this system of equations and find T (0 , 0) . (e) A way to accommodate this change while preserving the Markov property is by using a state of the form ( a, b, f ), where a and b holds their old meaning, while f represents the team which won the last match and can take only two values. The basic transition structure remains the same as before, with the exact transition probabilities now dictated by f , and that f of the ending state depends on which team won in that transition. Question 1.3 (Coffee Shop) : A 24-hour coffee shop operates in 4 shifts: morning, afternoon, evening and night . The workload of the coffeeshop in every shift is either 1 (low), 2 (medium) or 3(high) . The transition probabilities of workload for the coffee shop from each shift to the next are as follows: P morning = 0 . 04 0 . 41 0 . 55 0 . 36 0 . 21 0 . 43 0 . 16 0 . 28 0 . 56 , P afternoon = 0 . 26 0 . 37 0 . 37 0 . 09 0 . 21 0 . 70 0 . 23 0 . 28 0 . 49 , P evening = 0 . 38 0 . 37 0 . 25 0 . 49 0 . 06 0 . 45 0 . 16 0 . 22 0 . 62 P night = 0 . 15 0 . 13 0 . 72 0 . 62 0 . 02 0 . 36 0 . 10 0 . 63 0 . 27 Specifically, P morning means that the transition probabilities of workload when the current shift is morning. Define X = ( X n , n 0) as the coffee shop workload at subsequent time shifts. Assume X is a Markov chain and X 0 is an afternoon shift. Also define Y m = X 4 m for all m N ∪ { 0 } . (a) Find the transition probabilities of X . Is X a time homogeneous Markov chain? (b) Explain why the process Y = ( Y n , n 0) a Markov chain. Is it time homogeneous? Find the transition probabilities. (c) In the afternoon of January 7, 2019 the workload of the coffee shop is high. What is the probability that the workload of the coffee shop will be high in the evening of January 8, 2019?

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture