{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

DTMC_ExamplesI

# DTMC_ExamplesI - Some worked Numerical Examples on...

This preview shows pages 1–2. Sign up to view the full content.

Some worked Numerical Examples on Discrete-Time Markov Chains for ORIE 361 Instructor: Mark E. Lewis 1 Modelling Example This is a simple problem to get you into modeling DTMC’s. Consider a rat wandering in a maze. The maze has six rooms labelled F, 2 , 3 , 4 , 5 and S as in Figure 1. If a room has k doors, the probability that the rat Figure 1: The maze. selects a particular door is 1 /k . However, if the rat reaches room F , which contains the food, or room S , which gives it an electric shock, then it is kept there, and the experiment stops. The transition matrix can be computed from the above description and figure, P = 1 0 0 0 0 0 1 / 2 0 0 1 / 2 0 0 1 / 3 0 0 1 / 3 1 / 3 0 0 1 / 3 1 / 3 0 0 1 / 3 0 0 1 / 2 0 0 1 / 2 0 0 0 0 0 1 where the states are labelled in the order { F, 2 , 3 , 4 , 5 , S } . It may seem a bit difficult to see the sets of recurrent and transient states, so a transition diagram is in order. From this it should be clearer that

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 2

DTMC_ExamplesI - Some worked Numerical Examples on...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online