Lecture 03: Properties of Poisson Process
Conditional Distribution of Arrivals
Proposition 1.1. Let cfw_N (t), t > 0 be a Poisson process with cfw_Ai PR+ : i [n] a set of finite
disjoint intervals with B = i[n] Ai , and cfw_ki N : i [n] and k = i[n] ki
Lecture 05: Renewal Theory
One of the characterization for the Poisson process is of it being a counting process with iid
exponential inter-arrival times. Now we shall relax the exponential part.
Definition 1.1. A counting process cfw_N (
Lecture 09: Equilibrium Renewal Processes and Renewal
Renewal theory Contd. Delayed Renewal processes
(Optional not covered in class)
Consider two coins and suppose that each time is coin flipped, it lands tail with some un
Lecture 10: Discrete Time Markov Chains
Definition 1.1 (DTMC). A stochastic process cfw_Xn , n N0 , where Xn S, where S is
assumed at most countable, is called a DTMC (Discrete Time Markov Chain) if
P [Xn+1 = j|Xn = i, Xn1 = in1 , , X0
Lecture 06: Key Renewal Theorem and Applications
Key Renewal Theorem and Applications
Definition 1.1 (Lattice Random Variable). A non-negative random variable X is said to
be lattice if there exists d 0 such that
Prcfw_X = nd = 1.
For a lattice X,
Lecture 04: Compound Poisson Processes
Consider the scenario of a bus stop or a movie ticket counter. Each person arrives to the queue at
a random time and has to wait another random amount of time before he is serviced. A natural
Lecture 01: Introduction to Stochastic Processes
Definition 1.1. A probability space (, F, P ) consists of set of all possible outcomes denoted
by and called a sample space, a collection of subsets F of sample space, and a non-negativ
Lecture 14 : Continuous Time Markov Chains
Definition 1.1. For a countable set I a continuous time stochastic process cfw_X(t) I, t > 0
is a Markov process if
Prcfw_X(t + s) = j|X(u), u [0, s] = Prcfw_X(t + s) = j|X(s), for all s, t > 0 a
Lecture 24: Brownian Motion
Definition 1.1. A continuous time stochastic process cfw_X(t), t > 0 is called standard Brownian motion process if the following hold.
i) X(0) = 0.
ii) Process has stationary and independent increments.
Lecture 21: Exchangeability
Definition 1.1. Let cfw_Xi : i N be iid random variables with finite E[|X1 |]. Let
Xi , n N0 .
Then the process cfw_Sn : n N0 is called a random walk.
Definition 1.2. A random walk is called a simple
Lecture 13: Foster-Lyapunov Theorem
Theorem 1.1 (Foster,1950). Let cfw_Xn n0 be a irreducible DTMC on N0 if there exist a
function L : N0 R+ with E[L(X0 )] < , such that for some K > k 0, and > 0:
1. |cfw_x N0 : L(x) k| <
2. E[L(Xn )|Xn
Lecture 02: Poisson Process
Simple point processes
Definition 1.1. A stochastic process cfw_N (t), t > 0 is a point process if
1. N (0) = 0, and
2. for each , the map t 7 N (t) is non-decreasing, integer valued, and right continuous.
Definition 1.2. A s
Lecture 18: Queueing Networks
Corollary 1.1. Consider an M/M/s queue with Poisson() arrivals and each server having
exponential service time exp() service. If > s, then the output process in steady state is
Proof. Let X(t)
Lecture 12 : Convergence of DTMCs and Coupling theorem
Total Variation Distance
Definition 1.1. Given two probability distributions p and q defined on a countable space I,
their total variation distance is defined as
dT V (p, q) ,
kp qk1 .
Lecture 22: Random Walks
Duality in Random Walks
Essentially, if X is an exchangeable sequence of random variables, then (X1 , X2 , , Xn ) has the
same joint distribution as (Xn , Xn1 , , X1 ). In particular, an iid sequence of random variables
Lecture 15 : Limiting Probabilities and Uniformization
We denote by cfw_Sn : n N0 the jump times of CTMC and the probability transition of the
embedded Markov chain is denoted by P = cfw_pij : i 6= j I.
Definition 1.1. If the emb
Lecture 08: Branching Processes and Delayed Renewal
Age-dependent Branching Process
Suppose an organism lives upto a time period of X F and produces N P number of offspring.
Let X(t) denote the number of organisms alive at time t. The stochastic
Lecture 16: Reversibility
Definition 1.1. A stochastic process X(t) is reversible if (X(ti ) : i [n]) has the same
distribution as (X( ti ) : i [n]) for all ti , I, i [n].
Lemma 1.2. A reversible process is stationary.
Proof. Since X(t) is
Lecture 19 : Martingales
Definition 1.1. A stochastic process cfw_Zn , n N is said to be a martingale if
1. E[|Zn |] < ,
for all n.
2. E[Zn+1 |Z1 , Z2 , . . . Zn ] = Zn .
If the equality in second condition is replaced by or , then the proce
Lecture 20: Polyas Urn Scheme
The gambling interpretation of the stochastic integral suggests that it is natural
to let the amount bet at time n depend on the outcomes of the first n 1 flips but
not on the flip we are betting on, or on later flips. The ne
Lecture 11 : Discrete Time Markov Chains
Discrete Time Markov Chains Contd
Theorem 1.1. An irreducible, aperiodic Markov Chain with countable state space I is of one of
the following types:
i) All the states are either transient or null recurrent. That
Lecture 23: Martingale Concentration Inequalities
Lemma 1.1. If cfw_Xn : n N is a submartingale and N is a stopping time such that Prcfw_N
n = 1 then
EX1 EXN EXn .
Proof. It follows from optional stopping theorem that since N is bounded, E
Lecture 07: Inspection Paradox and Limiting Mean Excess
The Inspection Paradox
Define XN (t)+1 = A(t) + Y (t) as the length of the renewal interval containing t, in other words,
the length of current renewal interval. Inspection paradox says that P