Stochastic Processes The Markov Property Markov Chains Examples
Introductory Engineering Stochastic Processes, ORIE 3510
Instructor: Mark E. Lewis, Associate Professor
School of Operations Research and Information Engineering Cornell University
Disclaimer
ORIE 3510
Introduction to Engineering Stochastic Processes
Spring 2014
Solutions to Homework 1
1. (a) Poisson (b) Exponential (c) Geometric (d) Bernoulli (e) Binomial (f) Normal
2. We are given that E[X] = 2 and V ar(X) = 16. Thus
a) E[6 2X] = 6 2E[X] = 2
ORIE 3510 Homework 5 Solutions
Instructor: Mark E. Lewis
due 2PM, Wednesday February 29, 2012 (ORIE Hallway drop box)
(4n+6m)
1. The chain is irreducible. Since p1,1
solidarity property, d(8) = 2.
> 0, n, m N, the period of state 1 is d(1) = 2. By
2. Coun
ORIE 3510
J. G. Dai
Introduction to Engineering Stochastic Processes I
Spring 2014
Test 1 (March 6, 2014)
This is a closed book test. No calculator is allowed. There are a total of 4
problems. The full score is 100.
1. (25 points) A warranty department ma
Spring 2009 OR3510/5510 Problem Set 7; due March 30 as usual
Reading: We are into Section 6.5. x/y=page x, problem y in Ross. (1) 409/13 (2) 409/14 (3) A salesman flies around between Atlanta, Boston, and Chicago as follows. She stays in each city for an
ORIE3510
Introduction to Engineering Stochastic Processes
Section 4
Spring 2010
Review Stationary distribution interpretations Computation of Steadystate costs/rewards Transient state analysis (expected number of visits to transient states & absorption p
ORIE 3510/5510 Introduction to Engineering Stochastic Processes I
Spring 2013
Section4
1. A pensioner receives 2 (thousand dollars) at the beginning of each month. The amount of money he
needs to spend during a month is independent of the amount he has a
Spring 2009 OR3510/5510 Problem Set 3
Due Monday Feb 16 at 10am. You may insert in the homework box between Rhodes and Upson or give it to me in PHL 101 at the beginning of class by 10:10am. If you intend to give it to me, please make sure to arrive in go
ORIE 3510 Homework 3 Solutions
Instructor: Mark E. Lewis
due 2PM, Wednesday February 15, 2012 (ORIE Hallway drop box)
1. (a) cfw_Xn is not a Markov chain. To see this, it suces to check that
P (X4 = 1X3 = 0, X2 = 1) = P (X4 = 1X3 = 0, X2 = 1).
Indeed,
Spring 2011 OR3510/5510
Problem Set 7
When this is duebreak in the usual routine: Because of the coming spring
break, this problem set is due Tuesday March 29. The recitation on Monday is converted to an
oce hour; it will be held in the usual recitation r
ORIE 3510
J. Dai
Introduction to Engineering Stochastic Processes I
Spring 2014
Homework 8
(Due on Friday, March 14, 2014)
1. Let X be a Markov chain with state space
0.3
0
0
P =
0
0.1
0
cfw_a, b, c, d, e, f and transition matrix given by
0.5 0
0
0 0 .2
ORIE 3510/5510 Introduction to Engineering Stochastic Processes I
Spring 2013
ASSIGNMENT 6. Given: February 25, 2013. Due: March 4, 2013.
1. A taxi driver provides service in two zones of a city. Fares picked up in zone A will have destinations
in zone A
Spring 2011 OR3510/5510
Problem Set 3
Reminder: Due Monday February 21. Problem sets should be written neatly on 8 1/2 X 11
inch paper without fuzzy left margin because you tore it out of a notebook. Staple multiple
sheets. Deposit the papers in the homew
Spring 2011 OR3510/5510
Problem Set 5
Reading: You should be browsing in Chapter 5 after memorizing Chapter 4.
Because of the upcoming prelim, do not hand this in. However, you are responsible for the
material.
1. The lifetime of a radio is exponentially
ORIE 3510/5510 Introduction to Engineering Stochastic Processes I
Spring 2013
ASSIGNMENT 11. Given: Wednesday, April 10, 2013. Due: Monday, April 22, 2013.
1. Consider two machines that are maintained by a single repairman. Machine i functions for an expo
MEMORANDUM OF UNDERSTANDING
We, the undersigned agree that the following terms will modify the existing working agreement between the Adam Baxter Company, Deloitte, and Local 190 of the AFUICIO:
(Additional sheets may be attached)
All negotiators sign bel
Mathematical Programming:
An Overview
1
Management science is characterized by a scientic approach to managerial decision making. It attempts
to apply mathematical methods and the capabilities of modern computers to the difcult and unstructured
problems c
ORIE 3510/5510 Introduction to Engineering Stochastic Processes I
Spring 2013
ASSIGNMENT 10. Given: Monday, April 1, 2013. Due: Monday, April 8, 2013.
1. In good years, storms occur according to a Poisson process with rate 3 per unit time, while in other
Introduction to Engineering Stochastic Processes I
ORIE 3510

Spring 2014
ORIE 3510
Introduction to Engineering Stochastic Processes I
Anton Braverman and J. Dai
Spring 2017
Homework 5
February 15, 2017
Due: Friday, February 24
1. Let X = cfw_Xn : n = 0, 1, 2, . . . be a DTMC on state space cfw_1, 2, 3 with transition
probabili
ORIE 3510/5510 Introduction to Engineering Stochastic Processes I
Spring 2013
Section2 (Probability review contd, Discrete Time Markov Chains).
1. An unbiased die is successively rolled. Let X and Y denote, respectively the number of rolls necessary
to o
ORIE 3510
J. G. Dai
Introduction to Engineering Stochastic Processes I
Spring 2014
Test 2 (April 17, 2014)
Please print your name. This is a closed book test. No calculator is allowed.
There are a total of 4 problems. The full score is 100. Please do not
OR 3510/5510, Spring 13, Section 8
Section 8
1. A store must decide how much of a certain commodity to order so as to
meet next months demand, where that demand is assumed to have an
exponential distribution with rate . If the commodity costs the store c
ORIE 3510/5510, Spring 2013
Review Topics:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
Dene the sample space for an experiment.
Dene a random variable.
Dene an event.
Use probabilistic tools and identities (e.g. Bayes Rule, law of t
ORIE3510 Introduction to Engineering Stochastic Processes I Spring 2014
Recitation 9: 711 April 2014
1. Consider a system with c identical (parallel) servers and a single queue. Customers arrive into the
system according to a Poisson process with rate . I
ORIE3510 Introduction to Engineering Stochastic Processes I Spring 2014
Recitation 8: 2428 March 2014
Poisson Process
1. Customers arrive at a store according to a Poisson process with rate = 2 per hour.
(a) What is the probability that there is at most 1
ORIE3510 Introduction to Engineering Stochastic Processes I Spring 2014
Recitation 5: 35 March 2014
1
DTMC  Continued
Let X = cfw_Xn , n = 0, 1, . . . be a DTMC on state space S with transition matrix P .
(Stationary Distribution) = (i , i S) is said
ORIE3510 Introduction to Engineering Stochastic Processes I Spring 2014
Recitation 3: 1014 February 2014
1
Queueing Theory
Threepart code of queueing models: M/G/1. We will use the following letters:
G = General distribution
M = Exponential distribution
Introduction to Engineering Stochastic Processes I
ORIE 3510

Spring 2014
The Psychology of Waiting
Lines
By David H. Maister
Introduction
In one of a series of memorable
advertisements for which it has become
justly famous, Federal Express (the
overnight package delivery service)
noted that: "Waiting is frustrating,
demoralizi
Introduction to Engineering Stochastic Processes I
ORIE 3510

Spring 2014
ORIE 3510
Introduction to Engineering Stochastic Processes I
Anton Braverman and J. Dai
Spring 2017
Homework 7
March 5, 2017
Due: Friday, March 10
1. Consider the following transition matrix:
0 0.5 0 0.5
0.6 0 0.4 0
P =
0 0.7 0 0.3
0.8 0 0.2 0
(0.1)
(a)
Introduction to Engineering Stochastic Processes I
ORIE 3510

Spring 2014
ORIE 3510
Introduction to Engineering Stochastic Processes I
Anton Braverman and J. Dai
Spring 2017
Homework 6
February 24, 2017
Due: Friday, March 3
1. Consider Problem 3 in Assignment 4 with (s, S) = (2, 4).
(a) Does the DTMC have a stationary distribut
Introduction to Engineering Stochastic Processes I
ORIE 3510

Spring 2014
ORIE 3510
Introduction to Engineering Stochastic Processes I
Anton Braverman and J. Dai
Spring 2017
Homework 4
February 8, 2017
Due: Wednesday, February 15
1. Next months production at a manufacturing company will use a certain solvent
for part of its pro