CONTINUOUS-TIME
MARKOV CHAINS
IE 425 Handout # 3
Jose A. Ventura
Overview
Definitions
Properties:
Markov
Stationary
Transition Probability Functions
Chapman-Kolmogorov Equations
Classification of States
Steady-State Probabilities
Transition Rates
Stat

IE 425: HOMEWORK # 9 SOLUTIONS
Problem # 1
(a) Classify the queue system according to Kendalls notation: M/M/2/4
(b) Set up the transition rate diagram including the transition rates.
= 6 customers/hour,
= 4 customers/hour/server.
=6
=6
1
0
=6
=6
=4
4
3

IE 425: Homework # 11
Problem # 1
A product is consumed at the rate of 30 units per day. The holding cost per unit per day is $0.05, and the
setup cost is $100. Suppose that no shortage is allowed, and the unit purchasing cost is $10 for any quantity
less

IE 425: HOMEWORK # 6 SOLUTIONS
Problem # 1
Consider the discrete time Markov chain represented by the following state transition diagram:
2/5
4/5
1
1/10
1
2
1/2
4
2/5
5
1/2
1/5
1/10
3
1
(a) Identify the classes and states in each class, and classify the c

APPLICATIONS OF
QUEUEING THEORY
IE 425 Handout # 6
Overview
Problem 1: Server Selection in a Fast Food
Restaurant
Problem 2: Crew Size Selection in the Loading
Dock of a Warehouse
Problem 3: Selection of a Commercial Copier in
a Printing Company
Findi

IE 425: Homework # 13
Problem # 1: Hillier and Lieberman (10th Edition): 18.6-2 (page 869)
One of the largest selling items in J.C. Wards Department Store is a new refrigerator that is highly energy
efficient. About 40 of these refrigerators are being sol

IE 425: HOMEWORK # 13 SOLUTIONS
Problem # 1: Hillier and Lieberman (10th Edition): 18.6-2 (page 869)
(a) Solve by hand for R and Q.
Problem # 2: Hillier and Lieberman (10th Edition): 18.7-2 (page 871)
Problem # 3: Hillier and Lieberman (10th Edition): 18.

QUEUEING THEORY II
IE 425 Handout # 5
Overview
Probability Distributions of the Time Spent in
the System and the Time Spent in Queue
M/M/1
M/M/s (Example 1)
Example 2: Pooled Servers vs. Separate Servers
Infinite Queues in Series
Steady-State Probab

IE 425: HOMEWORK # 5 SOLUTIONS
Problem # 1: Hillier and Lieberman (9th Edition): 16.4-1 (pages 754-755)
(a) Since all of the states of this Markov chain communicate, this Markov chain is irreducible. Thus,
there is only one class, cfw_ 0, 1, 2, 3 . Each s

IE 425: Homework # 9
Problem # 1
Consider a barbershop with two barbers, two chairs for the customers receiving service (a haircut), and two
additional chairs for the customers that are waiting. The state of the system is the number of customers in
the sh

IE 425: Homework # 5
Problem # 1: Hillier and Lieberman (9th Edition): 16.4-1 (pages 754-755)
Given the following (one-step) transition probability matrices of Markov chains, determine the classes of
each Markov chain and whether the states in each class

IE 425: Homework # 6
Problem # 1
Consider the discrete time Markov chain represented by the following state transition diagram:
2/5
4/5
1
1
1/10
2
1/2
4
2/5
5
1/2
1/5
3
1/10
1
(a) Identify the classes and states in each class, and classify the classes and

IE 425: Homework # 10
Problem # 1
In a manufacturing system, jobs arrive at a particular CNC work center according to a Poisson process at
a mean rate of two per hour, and the operation time has an exponential distribution with a mean of 15
min. Enough in

IE 425: Homework # 3
Problem # 1: Hillier and Lieberman (9th Edition): 16.2-2 (page 754)
Assume that the probability of rain tomorrow is 0.5 if it is raining today, and assume that the probability of
being clear (no rain) tomorrow is 0.9 if it is clear to

IE 425: HOMEWORK # 12 SOLUTIONS
Problem # 1
(a) Economic order quantity and optimal cycle time:
Time unit = 1 week,
Unit purchasing cost: c 60 cents/pad,
Demand rate: a 200 pads/week,
Setup cost: K 5,000 cents/order,
Unit holding cost: h 3 cents/pad/week,

IE 425: HOMEWORK # 7 SOLUTIONS
Problem # 1
Consider an M/E2/4/20/LIFO queuing system.
(a) What is the inter-arrival time distribution? Exponential.
(b) What is the distribution of the number of customers entering the system in one time unit?
Poisson.
(c)

DYNAMIC PROGRAMMING
IE 425 Handout # 1
Overview
Introduction
Deterministic Dynamic Programming Models
Terminology
Principle of Optimality, Markov Property, and Curse of
Dimensionality
Example 1: Allocation of Study Time to Maximize Total Grade Points

IE 425: HOMEWORK # 2 SOLUTIONS
Problem # 1: Hillier and Lieberman (10th Edition): similar to 11.3-8 (pp. 471-472)
In order to solve this problem we must define the followings:
1. Objective of the problem : to determine the number of parallel units to inst

QUEUEING THEORY I
IE 425 Handout # 4
Overview
Introduction
Basic Structure and Parameters
System Outputs in Steady-State Condition and their
Relationship
Kendalls Notation
Behavior or Attitude of Customers
Birth-Death Process
Poisson Process and Ex

IE 425: HOMEWORK # 1 SOLUTIONS
Problem # 1: Hillier and Lieberman (10th Edition): 11.3-4 (pp. 470)
The campaign manager is not able to complete the DYNAMIC PROGRAMMING PROCEDURE and
needs your consulting services.
(a)
(b)
(c)
(d)
Complete the table in sta

IE 425: Homework # 12
Problem # 1 (final exam, fall 2004)
A college bookstore buys pads of paper for 60 cents each and sells them to students for 80 cents each.
Students use 200 pads per week, and it is reasonable to assume that the pads are continuously

INVENTORY THEORY I:
Deterministic Economic Order Quantity
(EOQ) Models
IE 425 Handout # 7
Overview
Introduction
Costs to be considered in an Inventory
Problem
Types of Inventory Models
Deterministic Inventory Models
Basic Economic Order Quantity (EOQ)

IE 425: HOMEWORK # 11 SOLUTIONS
Problem # 1
a) Economic order quantity (lot size):
K = $100, a = 30 units/day, c0 = $10/unit, c1 = $8/unit, M1 = 500 units, h = $0.05/unit/day.
Therefore,
Q
2Ka
h
2 100 30
346.41 units ,
0.05
TC (346.41)
2 a K h c 0 a 2 3

IE 425: HOMEWORK # 3 SOLUTIONS
Problem # 1: Hillier and Lieberman (9th Edition): 16.2-2 (page 754)
(a) A stochastic process
X t is said to possess the Markov Property if
P X t 1 j : X0 k 0 , X1 k1, . , X t 1 k t 1, X t i P X t 1 j : X t i ,
for all t 0,

IE 425: Homework # 4
Problem # 1
Each American family can be classified as living in an urban, suburban, or rural location. In a given year,
15 % of the urban families move to suburban locations and 5 % to rural locations.
5 % of the suburban families mov

IE 425: HOMEWORK # 10 SOLUTIONS
Problem # 1
(a) Classify the queueing system according to Kendalls notation: M/M/1/4.
(b) Set up the transition rate diagram including the transition rates.
= 2 jobs/hour,
= 4 jobs/hour.
=2
=2
1
0
=4
(c) P0
1
1
N 1
1 (

IE 425: Homework # 8
Problem # 1
Customers arrive to a facility according to a Poisson distribution with the rate of two customers per hour.
Find the following:
(a) The average number of customers arriving in an 8-hour period.
(b) The probability that at

IE 425: Homework # 2
Problem # 1: Hillier and Lieberman (10th Edition): similar to 11.3-8 (pp. 471-472)
Consider an electric system consisting of four components each of which must work for the system to
function. The reliability of the system can be impr

IE 425: Homework # 7
Problem # 1
Consider an M/E2/4/20/LIFO queueing system.
(a) What is the interarrival time distribution?
(b) What is the distribution of the number of customers entering the system in one time unit?
(c) What is the service time distrib