IEMS 315: Recitation 4
1. Warm Up
1. Classification of States: recurrent vs. transient, irreducible vs. reducible - some examples.
2. What are the three properties of an ergodic Markov Chain?
2. Problems
1. An organization has N employees where N is a lar

IEMS 315: Recitation 10
1. Warm Up
1. Basic Queueing Notation:
(a) First letter for arrival process; second letter for service distribution; third letter for number
of homogeneous servers.
(b) G/G/n: general arrival, general service times, n statistically

IEMS 315: Recitation 9
1. Warm Up
1. For an irreducible CTMC on a state space E, two properties hold:
(a) If state j is transient or null recurrent, then
lim Pi,j (t) = 0,
t
iE
(b) The CTMC is ergodic if and only if there exists a unique solution to the t

IEMS 315: Recitation 5
1. Warm Up
(a) What does a stationary chain imply?
The probability distribution of the chain does not change with time. The MC is in steady
state.
(b) How do we get = P and
P
i
i = 1? (Infinite State Space)
j = limn P (Xn = j) and

IEMS 315: Recitation 1
1. Warm Up
(a) What is Probability?
Many events cannot be predicted with complete certainty. So, when we say how likely
something is to happen, we use the idea of probability.
Probability satisfies three axioms: Positivity, P (E)

IEMS 315: Recitation 6
1. Warm Up
(a) A stochastic process cfw_N (t) : t 0 is a counting process if:
a. N (t) 0 for all t 0.
b. N (t) is integer-valued.
c. If s < t, then N (s) N (t),
(b) a counting process cfw_N (t) : t 0 is a Poisson process with rate i

IEMS 315: Recitation 8
1. Warm Up
1. A stochastic process cfw_X(t), t 0 is a continuous-time stochastic process on a countable (finite
or infinite) space E. Then X(t) is a CTMC if for all s, t > 0,
P (X(s + t) = j|X(u), 0 u s) = P (X(t + s) = j|X(s), j E

IEMS 315: Recitation 2
1. Warm Up
(a) What is the pdf of a uniform random variable, U, that is uniformly distributed over the interval
(, )?
U U (, )
f (u) =
1
1(,) (u)
F (u) =
u
,
< u < .
(b) What is the expected value of the r.v. X if:
P
X is disc

IEMS 315: Recitation 3
1. Warm Up
(a) What is the strong law of large numbers (SLLN)? The weak law of large numbers (WllN)?
SLLN considers almost sure convergence of the average of IID r.v.s to thri mean: Let
cfw_Xn : n 1 be a sequence of IID r.v.s havin

IEMS 315: Recitation 7
1. Warm Up
1. What is the conditional distribution of arrival times for a Poisson process?
(a) Recall the inverse relation: N (t) n Sn t hence P (Sn t) = P (N (t) n).
(b) If we know that there is an event in [0, t], then for s t and