Smith gambles constantly at four different casinos, and when he leaves one casino he goes to another which is
chosen from the other three with equal probabilities 1 . He owns just one umbrella, which he can store at the casinos. Each trip between casinos, he takes his umbrella if it is raining and if he has it at his starting casino, and he never takes the umbrella if it is not raining. It rains independently each trip with probability p.
(a) Let Xn be the number of umbrellas at his current location at the start of the nth trip. This defines a Markov Chain. Write down its transition matrix.
(b) Determine the stationary distribution of the Markov Chain.
(c) In the long run, what fraction of trips does Smith get wet?
My answer and question is below. Please verify my answer.