3.1 Markov Chains - Introduction - Exercises and Solutions.pdf - THE CHINESE UNIVERSITY OF HONG KONG Department of Statistics STAT3007 Introduction to

# 3.1 Markov Chains - Introduction - Exercises and Solutions.pdf

• Homework Help
• 3

This preview shows page 1 - 2 out of 3 pages.

THE CHINESE UNIVERSITY OF HONG KONG Department of Statistics STAT3007: Introduction to Stochastic Processes Markov Chains - Introduction - Exercises Solutions 1. (Problem 3.2.3 in Pinsky and Karlin) The first item appears at time zero. The fourth item appears at time 3. So we want to find P ( X 3 = defecive | X 0 = defective) := P ( X 3 = 1 | X 0 = 1) = p (3) 11 . This by Chapman-Kolmogorov we need to find the (1 , 1) entry in P 3 : P 3 = 0 . 9737 0 . 0263 0 . 3152 0 . 6848 so P ( X 3 = 1 | X 0 = 1) = P 3 11 = 0 . 6848. 2. (Exercise 3.3.1 in Pinsky and Karlin) This is the inventory model from Slides 13 to 20 of the Markov Chains - Introduction notes. Recall the ( s, S ) policy meant the chain { X n } represented the quantity of hand at the end of period n just prior to re-stocking and X n +1 = X n - ξ n +1 if s < X n S = S - ξ n +1 if X n s where ξ n +1 is the demand on day n + 1. The state space is {- 1 , 0 , 1 , 2 , 3 } , as we could start the day with 1 units and have 2 units demanded (meaning State -1 is possible). We work out the transition probabilities one-by-one. Say we are in State -1 now. Then tomorrow morning S = 3 units will arrive and by the end of the day we will have 3 left if ξ 1 = 0 (w.p 0.4); 2 left if ξ 1 = 1 (w.p. 0.3); 1 left if ξ 1 = 2 (w.p. 0.3). Say we are in State 0 now. Then tomorrow morning S

#### You've reached the end of your free preview.

Want to read all 3 pages?

• Fall '14

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern