lect0303

lect0303 - IEOR 4106 Introduction to Operations Research...

This preview shows pages 1–3. Sign up to view the full content.

IEOR 4106: Introduction to Operations Research: Stochastic Models Spring 2009, Professor Whitt Class Lecture Notes: Tuesday, March 3. Continuous-Time Markov Chains, Ross Chapter 6 Problems for Discussion and Solutions 1. Pooh Bear and the Three Honey Trees. A bear of little brain named Pooh is fond of honey. Bees producing honey are located in three trees: tree A , tree B and tree C . Tending to be somewhat forgetful, Pooh goes back and forth among these three honey trees randomly (in a Markovian manner) as follows: From A , Pooh goes next to B or C with probability 1 / 2 each; from B , Pooh goes next to A with probability 3 / 4, and to C with probability 1 / 4; from C , Pooh always goes next to A . Pooh stays a random time at each tree. (Assume that the travel times can be ignored.) Pooh stays at each tree an exponential length of time, with the mean being 5 hours at tree A or B , but with mean 4 hours at tree C . (a) Construct a CTMC enabling you to ﬁnd the limiting proportion of time that Pooh spends at each honey tree. (b) What is the average number of trips per day Pooh makes from tree B to tree A ? ANSWERS: (a) Find the limiting fraction of time that Pooh spends at each tree. ———————————————————————- These problems are part of a longer set of lecture notes, which have been posted on the web page. The focus here is on diﬀerent ways to model. We are thus focusing on Section 3 of the notes. For this problem formulation, it is natural to use the SMP (semi-Markov process) formu- lation of a CTMC (continuous-time Markov chain), involving the embedded DTMC and the mean holding times in each state. (See Sections 6.2 and 7.6 of Ross.) We thus deﬁne the embedded transition matrix P directly and the mean holding times 1 i directly. Note that this problem is formulated directly in terms of the DTMC, describing the random motion at successive transitions, so it is natural to use this initial modelling approach. Here the transition matrix for the DTMC is P = A B C 0 1 / 2 1 / 2 3 / 4 0 1 / 4 1 0 0 . In the displayed transition matrix P , we have only labelled the rows. The columns are assumed to be labelled in the same order. In general, the steady-state probability is α i = π i (1 i ) j π j (1 j )

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
In this case, the steady state probability vector of the discrete-time Markov chain is obtained by solving π = πP , yielding π = ( 8 17 , 4 17 , 5 17 ) . Then the ﬁnal steady-state distribution, accounting for the random holding times is α = ( 1 2 , 1 4 , 1 4 ) . You could alternatively work with the inﬁnitesimal transition rate matrix Q . If we want to deﬁne the inﬁnitesimal transition matrix Q (Ross uses lower case q ), then we can do so by setting Q i,j = ν i P i,j for i 6 = j .
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

Page1 / 6

lect0303 - IEOR 4106 Introduction to Operations Research...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online