lec0301 - IEOR 4106: Introduction to Operations Research:...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
IEOR 4106: Introduction to Operations Research: Stochastic Models Spring 2011, Professor Whitt Class Lecture Notes: Tuesday, March 1. Continuous-Time Markov Chains, Ross Chapter 6 Problems for Discussion and Solutions 1. Pooh Bear and the Three Honey Trees. A bear of little brain named Pooh is fond of honey. Bees producing honey are located in three trees: tree A , tree B and tree C . Tending to be somewhat forgetful, Pooh goes back and forth among these three honey trees randomly (in a Markovian manner) as follows: From A , Pooh goes next to B or C with probability 1 / 2 each; from B , Pooh goes next to A with probability 3 / 4, and to C with probability 1 / 4; from C , Pooh always goes next to A . Pooh stays a random time at each tree. (Assume that the travel times can be ignored.) Pooh stays at each tree an exponential length of time, with the mean being 5 hours at tree A or B , but with mean 4 hours at tree C . (a) Construct a CTMC enabling you to find the limiting proportion of time that Pooh spends at each honey tree. (b) What is the average number of trips per day Pooh makes from tree B to tree A ? ANSWERS: (a) Find the limiting fraction of time that Pooh spends at each tree. ———————————————————————- These problems are part of a longer set of lecture notes, which have been posted on the web page. The focus here is on different ways to model. We are thus focusing on Section 3 of the notes. For this problem formulation, it is natural to use the SMP (semi-Markov process) formu- lation of a CTMC (continuous-time Markov chain), involving the embedded DTMC and the mean holding times in each state. (See Sections 6.2 and 7.6 of Ross.) We thus define the embedded transition matrix P directly and the mean holding times 1 i directly. Note that this problem is formulated directly in terms of the DTMC, describing the random motion at successive transitions, so it is natural to use this initial modelling approach. Here the transition matrix for the DTMC is P = A B C 0 1 / 2 1 / 2 3 / 4 0 1 / 4 1 0 0 . In the displayed transition matrix P , we have only labelled the rows. The columns are assumed to be labelled in the same order. In general, the steady-state probability is α i = π i (1 i ) j π j (1 j )
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
In this case, the steady state probability vector of the discrete-time Markov chain is obtained by solving π = πP , yielding π = ( 8 17 , 4 17 , 5 17 ) . Then the final steady-state distribution, accounting for the random holding times is α = ( 1 2 , 1 4 , 1 4 ) . You could alternatively work with the infinitesimal transition rate matrix Q . If we want to define the infinitesimal transition matrix Q (Ross uses lower case q ), then we can do so by setting Q i,j = ν i P i,j for i 6 = j . As usual, the diagonal elements
Background image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

Page1 / 6

lec0301 - IEOR 4106: Introduction to Operations Research:...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online