IEOR 3106: Fall 2012, Professor Whitt
Time Reversibility and Queueing Networks, Thursday, November 1
In this class we will go over Section 6 in the long CTMC notes. See Sections 6.6 and 8.4 in
Ross for additional discussion.
I. A Cafeteria Example
An exam
IEOR 3106: Introduction to Operations Research: Stochastic Models
Fall 2012, Professor Whitt, October 23
1. Money Withdrawn from an ATM Machine
Customers arrive at an automated teller machine (ATM) at the times of a Poisson process
with a rate of = 10 per
IEOR 3106: Introduction to Operations Research: Stochastic Models
Professor Whitt, October 18, 2012
The OK Barbershop
Two enterprizing students - Hong Ouyang and Vishnu Kalugotla - have decided to earn a
little money in their spare time by opening a barbe
IEOR 3106: Introduction to Operations Research: Stochastic Models
Class Lecture Notes: Tuesday, October 16, 2012
Continuous-Time Markov Chains, Ross Chapter 6
1. Pooh Bear and the Three Honey Trees.
These notes are extracted from the longer set of notes o
IEOR 3106: Fall 2012, Professor Whitt
Poisson Entertainment, Thursday, October 11
1. Gone Fishing
To seek a change of pace from her intense Columbia experience, Chelsea Crowder has
decided to go shing. It is conceivable that this, like everything else, co
IEOR 3106: Introduction to Operations Research: Stochastic Models
Professor Whitt, Fall, 2012
Problem for Discussion, Tuesday, October 9
Trip to the Post Oce
Five students from IEOR 3106 Siddhant Bhatt (B), Ratnam Jain (J), Ethan Kochav
(K), Juan Carlos M
IEOR 3106: Introduction to Operations Research: Stochastic Models
Professor Whitt
Reversibility
1. Four Problems
(1) The Knight Errant (The Random Knight)
A knight is placed alone on one of the corner squares of a chessboard (having 8 8 = 64
squares). Wha
IEOR 3106: Professor Whitt
Lecture Notes, Tuesday, September 25, 2012
More Markov chains
1. Classication of States
See Section 4.3.
Concepts:
1. State j is accessible from state i if it is possible to get to j from i in some nite number
of steps. (notatio
IEOR 3106: Professor Whitt
Lecture Notes, Thursday, September 20, 2012
More on Markov Chains
1. Liberating Markov Mouse: An Absorbing Markov Chain
It is important to see that there are two basic kinds of markov chains: (i) irreducible
Markov chains and (i
IEOR 3106: Professor Whitt
Lecture Notes
Introduction to Markov Chains
1. Markov Mouse: The Closed Maze
We start by considering how to model a mouse moving around in a maze. The maze is a
closed space containing nine rooms. The space is arranged in a thre
IEOR 3106: Introduction to Operations Research: Stochastic Models
Fall 2012, Professor Whitt, Thursday, September 13.
Sums of Independent Random Variables and Moment Generating Functions
0. The Spider and the Fly
1. 2.5. Sums of Independent Random Variabl
IEOR 3106: Introduction to OR: Stochastic Models
Fall 2012, Professor Whitt
Class Lecture Notes: Tuesday, September 11.
The Central Limit Theorem and Stock Prices
1. The Central Limit Theorem (CLT)
See Section 2.7 of Ross.
(a) Time on My Hands:
Suppose th
IEOR 3106: Introduction to Operations Research: Stochastic Models
Fall 2012, Professor Whitt
Class Lecture Notes: Thursday, September 6.
Random Variables, Distributions and Expectation
0. Probability Art Gallery
Identify the pictures (on the blackboard) o
IEOR 3106: Introduction to Operations Research: Stochastic Models
Fall 2012, Professor Whitt
Class Lecture Notes: Tuesday, September 4
Conditional Probability
We discussed several problems in Chapter 1 of Ross. I emphasized several key points:
1. Probabil
CONTINUOUS-TIME MARKOV CHAINS
by Ward Whitt
Department of Industrial Engineering and Operations Research Columbia University New York, NY 10027-6699 Email: ww2040@columbia.edu URL: www.columbia.edu/ww2040
December 19, 2006 c Ward Whitt
1. Introduction
We
A Concise Summary
Everything you need to know about exponential and Poisson
Exponential Distribution
Assume that X exp(), by which we mean that X has an exponential distribution with
rate . Then X has mean 1/; i.e., EX = 1/. Also the variance is V ar(X )
Formula Sheet
IEOR 3106: Second Midterm Exam, Chapters 5-6, November 8, 2012
1. exponential distribution
Let X have an exponential distribution with mean E [X ] = 1/. Then the pdf and cdf are
fX (x) ex ,
x
x 0,
and FX (x)
0
fX (s) ds = 1 ex ,
x 0.
2. min
IEOR 3106: Introduction to Operations Research: Stochastic Models
SOLUTIONS for First Midterm Exam, October 4, 2012
There are 3 problems, each with multiple parts.
This exam is open book, but only the Ross textbook.
You need to show your work. Briey expla
IEOR 3106: First Midterm Exam, Chapters 1-4, October 4, 2012
There are 3 problems, each with multiple parts.
This exam is open book, but only the Ross textbook.
You need to show your work. Briey explain your reasoning.
Honor Code: Students are expected to
IEOR 3106: Introduction to Operations Research: Stochastic Models
Fall 2012, Professor Whitt
Homework Assignment 11: Tuesday, November 27. THE LAST ONE!
Due on Tuesday, December 4. (Thursday, December 6, is the last class.)
Chapter 10: Brownian Motion
In
IEOR 3106: Introduction to Operations Research: Stochastic Models
Fall 2012, Professor Whitt
Homework Assignment 10: Tuesday, November 20.
Due on Thursday, November 29. (so not to be due immediately after
Thanksgiving)
Chapter 7: More Renewal Theory
In Ro
IEOR 3106: Introduction to Operations Research: Stochastic Models
Fall 2012, Professor Whitt
Homework Assignment 9: Tuesday, November 13.
Due on Tuesday, November 20.
Chapter 7: Renewal Theory and its Applications
In Ross, read Sections 7.1-7.3 up to, but
IEOR 3106: Introduction to Operations Research: Stochastic Models
Fall 2012, Professor Whitt
Homework Assignment 8: Tuesday, October 30.
To be discussed in recitation on Sunday, November 4. Not to be turned in,
because no class on Tuesday, November 6, bec
IEOR 3106: Introduction to Operations Research: Stochastic Models
Fall 2012, Professor Whitt
Homework Assignment 7: Tuesday, October 23
Due on Tuesday, October 30.
More of Chapter 5: Read the rest of Section 5.3, skipping Examples 5.17, 5.21 and Subsectio
IEOR 3106: Introduction to Operations Research: Stochastic Models
Fall 2012, Professor Whitt
Homework Assignment 6: Tuesday, October 16
Due on Tuesday, October 23.
Chapter 6: Continuous-Time Markov Chains
Read Sections 6.1-6.5 in Ross. Do the following ex
IEOR 3106: Introduction to Operations Research: Stochastic Models
Fall 2012, Professor Whitt
Homework Assignment 5: Tuesday, October 9
Due on Tuesday, October 16.
Read Chapter 5: Sections 5.1-5.3, up to (but not including) Example 5.16 on page 322 of
10th
IEOR 3106: Introduction to Operations Research: Stochastic Models
Fall 2012, Professor Whitt
Homework Assignment 4: Tuesday, September 25.
Due on Tuesday, October 2. To be discussed at the recitation on Sunday,
September 30.
Note that the rst midterm exam
IEOR 3106: Introduction to Operations Research: Stochastic Models
Fall 2012, Professor Whitt
Numerical Part of Homework Assignment 3: Tuesday, September 18
Markov Chains
Due on Tuesday, October 2, an extra week is given, To be discussed at the
recitation
IEOR 3106: Introduction to Operations Research: Stochastic Models
Fall 2012, Professor Whitt
Homework Assignment 3: Tuesday, September 18
Introduction to Discrete-Time Markov Chains
Due on Tuesday, September 25 before class; to be discussed at the
recitat