View the step-by-step solution to:


Smith gambles constantly at four different casinos, and when he leaves one casino he goes to another which is

chosen from the other three with equal probabilities 1 . He owns just one umbrella, which he can store at the casinos. Each trip between casinos, he takes his umbrella if it is raining and if he has it at his starting casino, and he never takes the umbrella if it is not raining. It rains independently each trip with probability p.

(a) Let Xn be the number of umbrellas at his current location at the start of the nth trip. This defines a Markov Chain. Write down its transition matrix. 

(b) Determine the stationary distribution of the Markov Chain. 

(c) In the long run, what fraction of trips does Smith get wet? 

My answer and question is below. Please verify my answer.



Top Answer

Sign up to view the full answer

Why Join Course Hero?

Course Hero has all the homework and study help you need to succeed! We’ve got course-specific notes, study guides, and practice tests along with expert tutors.


Educational Resources
  • -

    Study Documents

    Find the best study resources around, tagged to your specific courses. Share your own to gain free Course Hero access.

    Browse Documents
  • -

    Question & Answers

    Get one-on-one homework help from our expert tutors—available online 24/7. Ask your own questions or browse existing Q&A threads. Satisfaction guaranteed!

    Ask a Question
Ask a homework question - tutors are online