Stochastic

# Xn is a markov chain a compute its transition

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: he beginning of each day, a piece of equipment is inspected to determine its working condition, which is classiﬁed as state 1 = new, 2, 3, or 4 = broken. We assume the state is a Markov chain with the following transition matrix: 1 2 3 4 1 .95 .05 0 0 20 .9 .1 0 30 0 .875 .125 (a) Suppose that a broken machine requires three days to ﬁx. To incorporate this into the Markov chain we add states 5 and 6 and suppose that p(4, 5) = 1, p(5, 6) = 1, and p(6, 1) = 1. Find the fraction of time that the machine is working. (b) Suppose now that we have the option of performing preventative 70 CHAPTER 1. MARKOV CHAINS maintenance when the machine is in state 3, and that this maintenance takes one day and returns the machine to state 1. This changes the transition probability to 1 23 1 .95 .05 0 20 .9 .1 31 0 0 Find the fraction of time the machine is working under this new policy. 1.44. Landscape dynamics. To make a crude model of a forest we might introduce states 0 = grass, 1 = bushes, 2 = small trees, 3 = large trees, and write down a trans...
View Full Document

Ask a homework question - tutors are online