DTMC_ExamplesI - Some worked Numerical Examples on...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
Some worked Numerical Examples on Discrete-Time Markov Chains for ORIE 361 Instructor: Mark E. Lewis 1 Modelling Example This is a simple problem to get you into modeling DTMC’s. Consider a rat wandering in a maze. The maze has six rooms labelled F, 2 , 3 , 4 , 5 and S as in Figure 1. If a room has k doors, the probability that the rat Figure 1: The maze. selects a particular door is 1 /k . However, if the rat reaches room F , which contains the food, or room S , which gives it an electric shock, then it is kept there, and the experiment stops. The transition matrix can be computed from the above description and figure, P = 1 0 0 0 0 0 1 / 2 0 0 1 / 2 0 0 1 / 3 0 0 1 / 3 1 / 3 0 0 1 / 3 1 / 3 0 0 1 / 3 0 0 1 / 2 0 0 1 / 2 0 0 0 0 0 1 where the states are labelled in the order { F, 2 , 3 , 4 , 5 , S } . It may seem a bit difficult to see the sets of recurrent and transient states, so a transition diagram is in order. From this it should be clearer that
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

Page1 / 2

DTMC_ExamplesI - Some worked Numerical Examples on...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online