Lecture 1: Introduction to Stochastic Processes
1
Probability Triples
We dene a probability triple or (probability) measure space or probability space to be a triple (, F, P ),
where:
the sample space is any non-empty set (e.g. = [0, 1]);
the -algebra (

Lecture 13: Discrete Renewal Theory
1
Introduction
Let be a possible event of a stochastic process cfw_Xt , and let T denote the waiting time until rst
occurs in the process, i.e. T = n means rst occurs on time n, while T = means that never
(k,k+1)
occurs

Lecture 16: Generalizations of Poisson Process
1
The Non-homogeneous Poisson process
Denition 1. The counting process cfw_N (t), t 0 is said to be a non-homogeneous Poisson process
with intensity function (t), t 0, if
a. N (0) = 0.
b. cfw_N (t), t 0 has i

Lecture 14: Poisson Processes
1
Introduction
A stochastic process cfw_N (t), t 0 is said to be a counting process if N (t) represents the total number
of events that occur by time t. A special and important counting process is the Poisson process.
Denitio

Biostatistics 682
10
103
Some Latent Variable Models
We saw last lecture how latent variables can be used to model the logistic model. A completely analogous latent variable representation holds for probit modeling. A similar model
holds for ordinal regre

Lecture 15: Poisson Processes II
1
Thinning (and Superposition)
Consider a Poisson process cfw_N (t), t 0 having rate . Suppose that each event can be classied as a
type i, i = 1, 2, . . . , m event with probability pi , i pi = 1, independently of all oth

Lecture 18: Continuous-time Markov Process
In this lecture, we will consider Markov chains whose index set T is no longer the non-negative integers,
but is now the set of non-negative real numbers. That is T = R+ cfw_0.
1
Continuous-time Markov chains
Den

Lecture 19: Continuous-time Markov Process (II)
1
Waiting Times of Birth and Death Process
Recall that a birth and death process can be dened by the following conditions:
1. Pr(X(t + h) X(t) = 1 | X(t) = i) = i h + o(h)
2. Pr(X(t + h) X(t) = 1 | X(t) = i)

Lecture 21: Introduction to Martingales
Stochastic processes are characterized by the dependence relationships among their variables. The
martingale property expresses a relation that occurs in many contexts. It is used to calculate absorption
probabiliti

Lecture 10: Reversible Markov Chains
Before we study reversibility, there is one more topic of interest:
1
The Mean Time Spent in Transient States
Consider a nite state Markov chain with transition matrix P and suppose that the states are numbered
so that

Lecture 11: Markov Chain Monte Carlo
Let X be a discrete random vector whose set of possible values is xj , j 1 such that Pr(X = xj ) is
the p.m.f. of X. Suppose we are interested in computing the expected value of some function of X. In
particular,
= E(

Lecture 2: Conditional Expectation and Probability
Before we adventure into stochastic processes we review conditional expectation and conditional probability, which are both fundamental techniques used in analyzing stochastic processes. Furthermore,
both

Lecture 4: Convolution and Generating Functions
In this lecture we will learn about the convolution operator on sequences and generating functions. The
operation of convolution and generating functions are intimately tied together and we will use them to

Lecture 5: Galton-Watson Processes
The simple branching process, also known as Galton-Watson process, was introduced by Francis Galton
in 1889 as a simple mathematical model for the propagation of family names. They were reinvented
by Leo Szilard in the l

Lecture 3: Computing Conditional Probability and Expectation
This lecture provides an overview on methods in computing conditional probability and expectations.
1
Computation by (Elementary) Denition
1.1
The Discrete Case
Let S denote the sample space and

Lecture 6: The Simple Random Walk
Denition 1. A random walk on the integers with step distribution F and initial state x Z is a
sequence cfw_Sn of random variables whose increments are independent, identically distributed random
variables i with common d

Lecture 8: Classication of States
1
The Strong Markov Property
n
Let fij denote the probability that, starting in state i, the process will rst enter state j at time n > 0.
0
1
Dene fij = 0 and note that fij = pij . Let Ti = mincfw_n 1 : Xn = i be the tim

Lecture 7: Introduction to Discrete Time Markov Chains
1
Denition and Examples
Let cfw_Xn denote a stochastic process whose index set is the non-negative integers and whose state
n=0
space is nite or countable. We say that the process is in state i at ti

Lecture 9: Stationary Distributions
1
Additional Properties of States
For the simple random walk the n-step transition probability pn = 0 when n is odd. We only have
00
pn > 0 when n is a multiple of 2. This phenomenon is called periodicity.
00
Denition 1

Lecture 20: Brownian Motion
Consider a stochastic process cfw_X(t); t 0. When we studied continuous time Markov processes we
assumed that the state space S was nite or countable innite. The last stochastic process we consider
in this course generalizes th