Stochastic Processes The Markov Property Markov Chains Examples
Introductory Engineering Stochastic Processes, ORIE 3510
Instructor: Mark E. Lewis, Associate Professor
School of Operations Research and Information Engineering Cornell University
Disclaimer
ORIE3510
Introduction to Engineering Stochastic Processes
Section 4
Spring 2010
Review Stationary distribution interpretations Computation of Steadystate costs/rewards Transient state analysis (expected number of visits to transient states & absorption p
Spring 2009 OR3510/5510 Problem Set 3
Due Monday Feb 16 at 10am. You may insert in the homework box between Rhodes and Upson or give it to me in PHL 101 at the beginning of class by 10:10am. If you intend to give it to me, please make sure to arrive in go
ORIE 3510 Homework 3 Solutions
Instructor: Mark E. Lewis
due 2PM, Wednesday February 15, 2012 (ORIE Hallway drop box)
1. (a) cfw_Xn is not a Markov chain. To see this, it suces to check that
P (X4 = 1X3 = 0, X2 = 1) = P (X4 = 1X3 = 0, X2 = 1).
Indeed,
ORIE 3510/5510 Introduction to Engineering Stochastic Processes I
Spring 2013
ASSIGNMENT 6. Given: February 25, 2013. Due: March 4, 2013.
1. A taxi driver provides service in two zones of a city. Fares picked up in zone A will have destinations
in zone A
Spring 2011 OR3510/5510
Problem Set 7
When this is duebreak in the usual routine: Because of the coming spring
break, this problem set is due Tuesday March 29. The recitation on Monday is converted to an
oce hour; it will be held in the usual recitation r
Spring 2011 OR3510/5510
Problem Set 5
Reading: You should be browsing in Chapter 5 after memorizing Chapter 4.
Because of the upcoming prelim, do not hand this in. However, you are responsible for the
material.
1. The lifetime of a radio is exponentially
Mathematical Programming:
An Overview
1
Management science is characterized by a scientic approach to managerial decision making. It attempts
to apply mathematical methods and the capabilities of modern computers to the difcult and unstructured
problems c
MEMORANDUM OF UNDERSTANDING
We, the undersigned agree that the following terms will modify the existing working agreement between the Adam Baxter Company, Deloitte, and Local 190 of the AFUICIO:
(Additional sheets may be attached)
All negotiators sign bel
ORIE 3510/5510 Introduction to Engineering Stochastic Processes I
Spring 2013
ASSIGNMENT 10. Given: Monday, April 1, 2013. Due: Monday, April 8, 2013.
1. In good years, storms occur according to a Poisson process with rate 3 per unit time, while in other
ORIE 3510
Introduction to Engineering Stochastic Processes
Spring 2014
J. Dai
Solutions to Homework 4
1. Data: p = 20, cv = 5, sv = 1, and the distribution of demand is:
D=d
P cfw_D = d
10
11
12
13
14
1
10
2
10
4
10
2
10
1
10
(a) E(D) = 12.
(b) Since E[
ORIE 3510
Stochastic Introduction to Engineering Stochastic Processes I
Spring 2014
J. Dai
Solutions to Homework 9
1. (a) For the Poisson process N , we have E[N (t)] = t. Then expected number of passengers
that arrive in the rst 2 hours is given by E[N (
ORIE 3510
Introduction to Engineering Stochastic Processes
Spring 2014
Solutions to Homework 6
1. (a) Observe that the state space of Xn is
S = cfw_3, 4, 5, 6,
since if the inventory drops below 3 we order up to 6. So at the beginning of a day the
minimum
ISyE 3232
Stochastic Manufacturing and Service Systems
Homework 10 Solutions
Spring 2009
Homework 10 Solutions: ORIE 3510
J. Dai
Solutions to Homework 12
1.
(j) Note that arrival processes for males and females follow indeppendent poisson distributions wi
ORIE 3510
J. Dai
Introduction to Engineering Stochastic Processes I
Spring 2014
Homework 10
(Due on Friday, March 28, 2014)
1. (This problem continues Problem 1 in Homework 9.) Suppose passengers arrive at a subway station
between 10am 5pm following a Po
ORIE 3510
Introduction to Engineering Stochastic Processes
Spring 2014
J. Dai
Solutions to Homework 3
1.
a) min(D, 6) has the pmf given by
1/10
9/10
P (min(D, 6) = k) =
0
if k = 5
if k = 6
otherwise
Hence,
E[min(D, 6)] =
1
9
5+
6 = 5.9.
10
10
b) (6 D)+
ORIE 3510
Introduction to Engineering Stochastic Processes I
Spring 2014
J. Dai
Solutions to Homework 7
1. (a) The state space for the process cfw_Xn , n 0 is given by S = cfw_10, 20. The transition
probability matrix is then given by:
.8
.1
P =
.2
.9
(b)
ORIE 3510
Introduction to Engineering Stochastic Processes
Spring 2014
J. Dai
Solutions to Homework 8
1. (a) The state diagram is as the following
0.8
0.5
0.3
0.5
e
a
b
0.1
0.1
1
0.2
c
0.5
0.3
f
d
0.7
1
(b) The recurrent states are: b, c, d, f .
(c) The i
ORIE 3510
Introduction to Engineering Stochastic Processes
Spring 2014
Solutions to Homework 1
1. (a) Poisson (b) Exponential (c) Geometric (d) Bernoulli (e) Binomial (f) Normal
2. We are given that E[X] = 2 and V ar(X) = 16. Thus
a) E[6 2X] = 6 2E[X] = 2
Spring 2014
ORIE 3510
J. G. Dai
Homework 2
(Due: Monday, February 17 by noon)
Read The Goal to page 264. Turn in a onepage description of the wisdom of Jonah in bulletized
format. Keep a copy of the description for yourself during class on Wednesday, Feb
Introduction to Engineering Stochastic Processes I
ORIE 3510

Spring 2015
ORIE 3510
J. Dai and M. Lewis
Introduction to Engineering Stochastic Processes I
Spring 2015
Homework 3
January 27, 2015
Due: Wednesday, February 4
1. Let D be a discrete random variable with the following
1/10
1/10
2/5
P[D = k] =
3/10
1/10
0
p.m.f.
i
Stochastic for Manufacturing and Service Systems
Lectures by Jim Dai, TeX by Hyunwoo Park
(These lecture notes were initially taken in Spring 2011 by Hyunwoo Park)
Spring 2011
Abstract
A note to ORIE 3510 students on 2/10/2014: these lecture notes were ta
ORIE 3510
J. G. Dai
Introduction to Engineering Stochastic Processes I
Spring 2014
Test 1 (March 6, 2014)
This is a closed book test. No calculator is allowed. There are a total of 4
problems. The full score is 100.
1. (25 points) A warranty department ma
Introduction to Engineering Stochastic Processes I
ORIE 3510

Spring 2015
ORIE 3510/5510
J. G. Dai
A sample writeup
Feb 11, 2015
The Wisdom of the Goal
Drew Ungerman
for an operations course at Stanford
taught by Professor Kumar
The Goal outlines new global principles in manufacturing and illustrates how cash ow can be maximiz
Introduction to Engineering Stochastic Processes I
ORIE 3510

Spring 2015
ORIE 3510
J. Dai and M. Lewis
Introduction to Engineering Stochastic Processes I
Spring 2015
Homework 4
February 4, 2015
Due: Wednesday, February 11
1. Suppose we are selling lemonade during a football game. The lemonade sells for $20 per gallon but
only
ORIE 3510
J. G. Dai
Introduction to Engineering Stochastic Processes I
Spring 2014
Test 2 (April 17, 2014)
Please print your name. This is a closed book test. No calculator is allowed.
There are a total of 4 problems. The full score is 100. Please do not
ORIE3510 Introduction to Engineering Stochastic Processes I Spring 2014
Recitation 9: 711 April 2014
1. Consider a system with c identical (parallel) servers and a single queue. Customers arrive into the
system according to a Poisson process with rate . I
ORIE3510 Introduction to Engineering Stochastic Processes I Spring 2014
Recitation 8: 2428 March 2014
Poisson Process
1. Customers arrive at a store according to a Poisson process with rate = 2 per hour.
(a) What is the probability that there is at most 1
Introduction to Engineering Stochastic Processes I
ORIE 3510

Spring 2015
Homework 4 Solutions
ORIE 3510
Problem 1
(a) cfw_0, 2 and cfw_1, 3 are recurrent; cfw_5 is transient.
(b) cfw_1, 4, 9, cfw_3, 8, and cfw_5 are recurrent; cfw_2, cfw_6 and cfw_7 are transient.
Problem 2
First suppose that the condition holds, i.e. there is