Stochastic Processes The Markov Property Markov Chains Examples
Introductory Engineering Stochastic Processes, ORIE 3510
Instructor: Mark E. Lewis, Associate Professor
School of Operations Research and Information Engineering Cornell University
Disclaimer
ORIE 3510
Introduction to Engineering Stochastic Processes
Spring 2014
Solutions to Homework 1
1. (a) Poisson (b) Exponential (c) Geometric (d) Bernoulli (e) Binomial (f) Normal
2. We are given that E[X] = 2 and V ar(X) = 16. Thus
a) E[6 2X] = 6 2E[X] = 2
Introduction to Engineering Stochastic Processes I
ORIE 3510

Fall 2015
ORIE 3510/5510
1
Jim Dai
Spring 2015
April 13, 2015
1. A call center has two agents and three phone lines. The processing times of calls are
iid exponentially distributed with mean 2 minutes. The arrival process is Poisson
with rate = 1 call per minute. T
Introduction to Engineering Stochastic Processes I
ORIE 3510

Fall 2015
ORIE 3510
Jim Dai and Mark E. Lewis
Introduction to Engineering Stochastic Processes I
Spring 2015
Homework 14
(Due on Monday at 2pm, April 27, 2015)
1. Recall Problem 2 in Homework 13: a call center has two agents and four phone lines. The processing
tim
Introduction to Engineering Stochastic Processes I
ORIE 3510

Fall 2015
ORIE 3510
J. Dai and M. Lewis
Introduction to Engineering Stochastic Processes I
Spring 2015
Homework 15
(Due on Wednesday, 10am, May 6, 2015)
1. Consider an M/M/n/0 system that models a customer call center without additional waiting space.
This system i
Introduction to Engineering Stochastic Processes I
ORIE 3510

Fall 2015
ORIE 3510
J. Dai and M. Lewis
Introduction to Engineering Stochastic Processes I
Spring 2015
Homework 15
(Due on Wednesday, 10am, May 6, 2015)
1. Consider an M/M/n/0 system that models a customer call center without additional waiting space.
This system i
ORIE 3510 Homework 5 Solutions
Instructor: Mark E. Lewis
due 2PM, Wednesday February 29, 2012 (ORIE Hallway drop box)
(4n+6m)
1. The chain is irreducible. Since p1,1
solidarity property, d(8) = 2.
> 0, n, m N, the period of state 1 is d(1) = 2. By
2. Coun
Introduction to Engineering Stochastic Processes I
ORIE 3510

Fall 2015
ORIE 3510
J. Dai and M. Lewis
Introduction to Engineering Stochastic Processes I
Spring 2015
Homework 13
April 20, 2015
Due: Monday, April 20th
1. (a) The rate diagram is
(b) Let vi be the rate at state i cfw_A, B, C. Then the total rates are
vA = 12,
vB
ORIE3510
Introduction to Engineering Stochastic Processes
Section 4
Spring 2010
Review Stationary distribution interpretations Computation of Steadystate costs/rewards Transient state analysis (expected number of visits to transient states & absorption p
ORIE 3510
J. G. Dai
Introduction to Engineering Stochastic Processes I
Spring 2014
Test 1 (March 6, 2014)
This is a closed book test. No calculator is allowed. There are a total of 4
problems. The full score is 100.
1. (25 points) A warranty department ma
Introduction to Engineering Stochastic Processes I
ORIE 3510

Fall 2015
ORIE 3510
Jim Dai and Mark E. Lewis
Introduction to Engineering Stochastic Processes I
Spring 2015
Homework 13
(Due on Monday at 10am, April 20, 2015)
1. Consider a CTMC X = cfw_X(t), t 0 on S = cfw_A, B, C with generator G given by
12 4
8
6 1
A= 5
2
0 2
Spring 2009 OR3510/5510 Problem Set 7; due March 30 as usual
Reading: We are into Section 6.5. x/y=page x, problem y in Ross. (1) 409/13 (2) 409/14 (3) A salesman flies around between Atlanta, Boston, and Chicago as follows. She stays in each city for an
Spring 2009 OR3510/5510 Problem Set 3
Due Monday Feb 16 at 10am. You may insert in the homework box between Rhodes and Upson or give it to me in PHL 101 at the beginning of class by 10:10am. If you intend to give it to me, please make sure to arrive in go
ORIE 3510 Homework 3 Solutions
Instructor: Mark E. Lewis
due 2PM, Wednesday February 15, 2012 (ORIE Hallway drop box)
1. (a) cfw_Xn is not a Markov chain. To see this, it suces to check that
P (X4 = 1X3 = 0, X2 = 1) = P (X4 = 1X3 = 0, X2 = 1).
Indeed,
ORIE 3510/5510 Introduction to Engineering Stochastic Processes I
Spring 2013
Section4
1. A pensioner receives 2 (thousand dollars) at the beginning of each month. The amount of money he
needs to spend during a month is independent of the amount he has a
Spring 2011 OR3510/5510
Problem Set 7
When this is duebreak in the usual routine: Because of the coming spring
break, this problem set is due Tuesday March 29. The recitation on Monday is converted to an
oce hour; it will be held in the usual recitation r
ORIE 3510
J. Dai
Introduction to Engineering Stochastic Processes I
Spring 2014
Homework 8
(Due on Friday, March 14, 2014)
1. Let X be a Markov chain with state space
0.3
0
0
P =
0
0.1
0
cfw_a, b, c, d, e, f and transition matrix given by
0.5 0
0
0 0 .2
Spring 2011 OR3510/5510
Problem Set 3
Reminder: Due Monday February 21. Problem sets should be written neatly on 8 1/2 X 11
inch paper without fuzzy left margin because you tore it out of a notebook. Staple multiple
sheets. Deposit the papers in the homew
ORIE 3510/5510 Introduction to Engineering Stochastic Processes I
Spring 2013
ASSIGNMENT 6. Given: February 25, 2013. Due: March 4, 2013.
1. A taxi driver provides service in two zones of a city. Fares picked up in zone A will have destinations
in zone A
Spring 2011 OR3510/5510
Problem Set 5
Reading: You should be browsing in Chapter 5 after memorizing Chapter 4.
Because of the upcoming prelim, do not hand this in. However, you are responsible for the
material.
1. The lifetime of a radio is exponentially
ORIE 3510/5510 Introduction to Engineering Stochastic Processes I
Spring 2013
ASSIGNMENT 11. Given: Wednesday, April 10, 2013. Due: Monday, April 22, 2013.
1. Consider two machines that are maintained by a single repairman. Machine i functions for an expo
MEMORANDUM OF UNDERSTANDING
We, the undersigned agree that the following terms will modify the existing working agreement between the Adam Baxter Company, Deloitte, and Local 190 of the AFUICIO:
(Additional sheets may be attached)
All negotiators sign bel
Mathematical Programming:
An Overview
1
Management science is characterized by a scientic approach to managerial decision making. It attempts
to apply mathematical methods and the capabilities of modern computers to the difcult and unstructured
problems c
ORIE 3510/5510 Introduction to Engineering Stochastic Processes I
Spring 2013
ASSIGNMENT 10. Given: Monday, April 1, 2013. Due: Monday, April 8, 2013.
1. In good years, storms occur according to a Poisson process with rate 3 per unit time, while in other
Stochastic for Manufacturing and Service Systems
Lectures by Jim Dai, TeX by Hyunwoo Park
(These lecture notes were initially taken in Spring 2011 by Hyunwoo Park)
Spring 2011
Abstract
A note to ORIE 3510 students on 2/10/2014: these lecture notes were ta
ORIE3510 Introduction to Engineering Stochastic Processes I Spring 2014
Recitation 8: 2428 March 2014
Poisson Process
1. Customers arrive at a store according to a Poisson process with rate = 2 per hour.
(a) What is the probability that there is at most 1
ORIE3510 Introduction to Engineering Stochastic Processes I Spring 2014
Recitation 1: 2731 January 2014
1
Newsvendor Problem:
Notations:
D = demand of the period (random variable)
y = quantity of order
Cv = buying price of one item from the supplier
Cp =
ORIE3510 Introduction to Engineering Stochastic Processes I Spring 2014
Recitation 3: 1014 February 2014
1
Queueing Theory
Threepart code of queueing models: M/G/1. We will use the following letters:
G = General distribution
M = Exponential distribution