Lecture 03: Properties of Poisson Process
1
Conditional Distribution of Arrivals
Proposition 1.1. Let cfw_N (t), t > 0 be a Poisson process with cfw_Ai PR+ : i [n] a set of finite
disjoint intervals with B = i[n] Ai , and cfw_ki N : i [n] and k = i[n] ki

Lecture 05: Renewal Theory
1
Renewal Theory
One of the characterization for the Poisson process is of it being a counting process with iid
exponential inter-arrival times. Now we shall relax the exponential part.
Definition 1.1. A counting process cfw_N (

Lecture 09: Equilibrium Renewal Processes and Renewal
Reward Processes
1
Renewal theory Contd. Delayed Renewal processes
1.1
Example:
(Optional not covered in class)
Consider two coins and suppose that each time is coin flipped, it lands tail with some un

Lecture 10: Discrete Time Markov Chains
1
Introduction
Definition 1.1 (DTMC). A stochastic process cfw_Xn , n N0 , where Xn S, where S is
assumed at most countable, is called a DTMC (Discrete Time Markov Chain) if
(n)
P [Xn+1 = j|Xn = i, Xn1 = in1 , , X0

Lecture 06: Key Renewal Theorem and Applications
1
Key Renewal Theorem and Applications
Definition 1.1 (Lattice Random Variable). A non-negative random variable X is said to
be lattice if there exists d 0 such that
X
Prcfw_X = nd = 1.
nN
For a lattice X,

Lecture 04: Compound Poisson Processes
1
Queueing Theory
Consider the scenario of a bus stop or a movie ticket counter. Each person arrives to the queue at
a random time and has to wait another random amount of time before he is serviced. A natural
questi

Lecture 01: Introduction to Stochastic Processes
1
Probability Review
Definition 1.1. A probability space (, F, P ) consists of set of all possible outcomes denoted
by and called a sample space, a collection of subsets F of sample space, and a non-negativ

Lecture 14 : Continuous Time Markov Chains
1
Markov Process
Definition 1.1. For a countable set I a continuous time stochastic process cfw_X(t) I, t > 0
is a Markov process if
Prcfw_X(t + s) = j|X(u), u [0, s] = Prcfw_X(t + s) = j|X(s), for all s, t > 0 a

Lecture 24: Brownian Motion
1
Introduction
Definition 1.1. A continuous time stochastic process cfw_X(t), t > 0 is called standard Brownian motion process if the following hold.
i) X(0) = 0.
ii) Process has stationary and independent increments.
iii) For

Lecture 21: Exchangeability
1
Random Walk
Definition 1.1. Let cfw_Xi : i N be iid random variables with finite E[|X1 |]. Let
Sn =
n
X
Xi , n N0 .
k=1
Then the process cfw_Sn : n N0 is called a random walk.
Definition 1.2. A random walk is called a simple

Lecture 13: Foster-Lyapunov Theorem
1
Fosters Theorem
Theorem 1.1 (Foster,1950). Let cfw_Xn n0 be a irreducible DTMC on N0 if there exist a
function L : N0 R+ with E[L(X0 )] < , such that for some K > k 0, and > 0:
1. |cfw_x N0 : L(x) k| <
2. E[L(Xn )|Xn

Lecture 02: Poisson Process
1
Simple point processes
Definition 1.1. A stochastic process cfw_N (t), t > 0 is a point process if
1. N (0) = 0, and
2. for each , the map t 7 N (t) is non-decreasing, integer valued, and right continuous.
Definition 1.2. A s

Lecture 18: Queueing Networks
1
Migration Processes
Corollary 1.1. Consider an M/M/s queue with Poisson() arrivals and each server having
exponential service time exp() service. If > s, then the output process in steady state is
Poisson().
Proof. Let X(t)

Lecture 12 : Convergence of DTMCs and Coupling theorem
1
Total Variation Distance
Definition 1.1. Given two probability distributions p and q defined on a countable space I,
their total variation distance is defined as
dT V (p, q) ,
1
kp qk1 .
2
Lemma 1.2

Lecture 22: Random Walks
1
Duality in Random Walks
Essentially, if X is an exchangeable sequence of random variables, then (X1 , X2 , , Xn ) has the
same joint distribution as (Xn , Xn1 , , X1 ). In particular, an iid sequence of random variables
is excha

Lecture 15 : Limiting Probabilities and Uniformization
1
Limiting Probabilities
We denote by cfw_Sn : n N0 the jump times of CTMC and the probability transition of the
embedded Markov chain is denoted by P = cfw_pij : i 6= j I.
Definition 1.1. If the emb

Lecture 08: Branching Processes and Delayed Renewal
Process
1
Age-dependent Branching Process
Suppose an organism lives upto a time period of X F and produces N P number of offspring.
Let X(t) denote the number of organisms alive at time t. The stochastic

Lecture 16: Reversibility
1
Reversibility
Definition 1.1. A stochastic process X(t) is reversible if (X(ti ) : i [n]) has the same
distribution as (X( ti ) : i [n]) for all ti , I, i [n].
Lemma 1.2. A reversible process is stationary.
Proof. Since X(t) is

Lecture 19 : Martingales
1
Martingales
Definition 1.1. A stochastic process cfw_Zn , n N is said to be a martingale if
1. E[|Zn |] < ,
for all n.
2. E[Zn+1 |Z1 , Z2 , . . . Zn ] = Zn .
If the equality in second condition is replaced by or , then the proce

Lecture 20: Polyas Urn Scheme
The gambling interpretation of the stochastic integral suggests that it is natural
to let the amount bet at time n depend on the outcomes of the first n 1 flips but
not on the flip we are betting on, or on later flips. The ne

Lecture 11 : Discrete Time Markov Chains
1
Discrete Time Markov Chains Contd
Theorem 1.1. An irreducible, aperiodic Markov Chain with countable state space I is of one of
the following types:
i) All the states are either transient or null recurrent. That

Lecture 23: Martingale Concentration Inequalities
1
Introduction
Lemma 1.1. If cfw_Xn : n N is a submartingale and N is a stopping time such that Prcfw_N
n = 1 then
EX1 EXN EXn .
Proof. It follows from optional stopping theorem that since N is bounded, E

Lecture 07: Inspection Paradox and Limiting Mean Excess
Time
1
The Inspection Paradox
Define XN (t)+1 = A(t) + Y (t) as the length of the renewal interval containing t, in other words,
the length of current renewal interval. Inspection paradox says that P