IEMS 315 Final Review Session 2
2008 Fall
Exercise 1. Consider a single server system where customers arrive according to a Poisson
process at a rate of 20 per hour, and require exponentially distributed service times with mean 2
minutes. Suppose it costs

IEMS 315: Recitation 8 - CTMC
1
Warm Up
1. A process cfw_X(t), t 0 is a continuous-time stochastic process on a countable (nite or innite) state
space E . Then X(t) is a CTMC if for all s, t 0:
P (X(s + t) = j|X(), 0 s)
= P (X(t + s) = j|X(s), j E.
The tr

IEMS 315: Stochastic Models and Simulation
Solutions to Homework Assignment 6
The numbers of the problems refer to the 10th edition of Ross.
(5.37) A machine works for an exponentially distributed time with rate and then fails. A
repair crew checks the ma

IEMS 315: Recitation 3 - Fundamental Limit Theorems & DTMC
1
Warm Up
1. What is the strong law of large numbers (SLLN)? The weak law of large numbers (WLLN)?
SLLN considers almost sure convergence of the average of IID r.v.'s to their mean: Let cfw_Xn :
n

IEMS 315: Recitation 7 - More Poisson Process
1
Warm Up
1. What is the conditional distribution of arrival times for a Poisson Process?
(a) Recall that an inverse relationship holds for every sample path of the Poisson Process: N (t) n
if and only if Sn t

G/G/n
n
M/M/n
n
G/G/n/m
n
m
G/G/n/m + G
M/M/n/m + G
n
m
L= W
j
j
j
j
j
j
j
j / = j
1
:=
)n , n
n = (1
/
0.
L = E[N ]
L
=
1
X
)n
n(1
n=0
L
=
(1
)
1
X
nn
n=0
L
P1
n
n=0 n =
=
=
=
L=
1
12
1
8
2
3
2/3
1/3
P1
n
n=1 n
=
(1
1
=
)
(1
P1
)2
d n
n=1 d
=
1
d
= d

IEMS 315: Recitation 1 - Probability
April 5, 2015
1
Warm Up
1. What is Probability?
(a) Many events can't be predicted with total certainty. So, when we say how likely something is to
happen, we use the idea of probability
(b) Probability satises three a

IEMS 315: Recitation 5 - More on DTMC in Finite and Innite
State Spaces
1
Warm Up
1. What does a stationary chain imply?
(a) The probability distribution of the chain does not change with time. The MC is in steady state
2. How do we get = P and
i
i = 1? (

IEMS 315: Recitation 4 - DTMC, Limiting Probabilities, Innite
State Spaces
1
Problems
1. An organization has N employees where N is a large number. Each employee has one of three possible job classications and changes classications (independently) accordi

IEMS 315: Recitation 6 - Exponential and Poisson Process
1
Warm Up
1. A stochastic process cfw_N (t) : t 0 is a counting process if:
(a) N (t) 0 for all t 0
(b) N (t) is integer valued
(c) If s < t, then N (s) N (t)
2. A counting process cfw_N (t) : t 0 i

IEMS 315: Stochastic Models and Simulation
Solutions to Homework Assignment 8
The numbers of the problems refer to the 10th edition of Ross.
(6.8) Consider two machines, both of which have an exponential lifetime with mean 1/.
There is a single repairman

IEMS 315: Stochastic Models and Simulation
Spring 2015, Prof. Ohad Perry
Lecture 13
Page 1 of 4
Lecture 13: Absorbing Markov Chains
Recall the gamblers ruin example from the previous class, where a gambler is playing a game
until he either looses all his

IEMS 315: Recitation 2 - Random Variables, Moments, Joint
Distributions
April 13, 2015
1
Warm Up
1. What is the pdf of a uniform random variable, U , that is uniformly distributed over the interval (, )?
U U nif (, )
f (u) =
F () =
where for a set A, 1

IEMS 315: Stochastic Models and Simulation
Spring 2015, Prof. Ohad Perry
Lecture 10
Page 1 of 5
Lecture 10: DTMC - Introduction to Asymptotic Analysis
As we saw in the previous classes, using the one-step transition matrix P and the vector of initial
prob

IEMS 315: Stochastic Models and Simulation
Spring 2015, Prof. Ohad Perry
Lecture 1
Page 1 of 4
Lecture 1: Probability Basics
Probability is a branch of mathematics and therefore requires special attention to axioms,
denitions and theorems.
So what is prob

IEMS 315: Stochastic Models and Simulation
Spring 2015, Prof. Ohad Perry
Lecture 11
Page 1 of 5
Lecture 11: Stationarity and Limiting Probabilities
As usual, we let Pi,j denote the one-step probabilities of a Markov chain on a state space E, and P
denote

IEMS 315: Stochastic Models and Simulation
Spring 2015, Prof. Ohad Perry
Lecture 12
Page 1 of 4
Lecture 12: More on DTMC on Innite State Spaces
Recall the random-walk example from last lecture:
If cfw_Sn : n
0 is a random walk on the integers with transit

IEMS 315: Stochastic Models and Simulation
Spring 2015, Prof. Ohad Perry
Lecture 19
Page 1 of 4
Lecture 19: Generalizations of the Poisson Process
Although the Poisson process is reasonable to assume in many applications, it is not always appropriate. We

IEMS 315: Stochastic Models and Simulation
Spring 2015, Prof. Ohad Perry
Lecture 9
Page 1 of 4
Lecture 9: More on DTMC
Recap: A discrete-time Markov chain (DTMC) is a discrete-time stochastic process on a
countable state-space E such that, for all i, j, i

IEMS 315: Stochastic Models and Simulation
Spring 2015, Prof. Ohad Perry
Lecture 14
Page 1 of 4
Lecture 14: The Exponential Distribution
Discrete-time Markov chains have a notable limitation: There is no real notion of time. In particular, we considered t

IEMS 315: Stochastic Models and Simulation
Spring 2015, Prof. Ohad Perry
Lecture 15
Page 1 of 2
Lecture 15: Exploiting the Properties of the Exponential Distribution
Recall from last lecture:
If X1 , . . . , Xn are independent exponential r.v.s with respe

IEMS 315: Stochastic Models and Simulation
Spring 2015, Prof. Ohad Perry
Lecture 17
Page 1 of 4
Lecture 17: Poisson Process - Three Points of View
Recall from last lecture:
A counting process with unit jumps, having stationary and independent increments i

IEMS 315: Stochastic Models and Simulation
Spring 2015, Prof. Ohad Perry
Lecture 4
Page 1 of 4
Lecture 4: Random Variables and Moments
Special Random Variables
Recall that
A (real-valued) r.v. is a function mapping a sample-space into the real line R.
A

IEMS 315: Stochastic Models and Simulation
Fall 2013, Prof. Ohad Perry
Solutions to Homework Assignment 7
Page 1 of 3
Solutions to Homework Assignment 7
The numbers of the problems refer to the 10th edition of Ross.
(6.1) A population of organisms consist

IEMS 315: Stochastic Models and Simulation
Spring 2015, Prof. Ohad Perry
Lecture 8
Page 1 of 3
Lecture 8: Stochastic Processes and DTMC
A stochastic process cfw_Xt : t 2 T is a collection of random variables: for each t 2 T , Xt is
a r.v. with T being a

\TEPH‘“!£ yum“ Isms 35 Hymnal! 6 ,
1 r , L _ _
| *5” “N" " “PG"! mudA'IIno 1 Mean! _ 33’1- 411/2137 :17-6'2,_g_,.»-»-#~””'/]
I 9‘ T v‘b‘mq ”I when; 33LI5_,!?13_L-»/ﬂf~ ”(Mn—r
I 9‘ T 7' Us I Tm IWﬁLTVIxF—FME I
- - “Wm- 9'- ' IVA} b dro magma; .:
I fwwmmi I

IEMS 315 Winter 2015 Homework 6
Prof. Irina Dolinskaya
Due 9:00 am on Tuesday, February 24th, 2015
To turn in the assignment you can submit it electronically on canvas, or slide it under my office door
(M235).
The solution will be posted shortly after the