Western University (Ontario)  Also known as University of Western Ontario
Markov chains with application
STATISTICS 4654

Fall 2016
Textbook Sections 4.5, 4.6
4.5 The Gamblers Ruin Problem
Consider a modification of the Random Walk:
The state i represents the present wealth of an
individual, who gambles $1 at each play of a game
Western University (Ontario)  Also known as University of Western Ontario
Markov chains with application
STATISTICS 4654

Fall 2016
Textbook Section 4.8
4.8 Timereversible Markov Chains
One can look at a ergodic Markov chain that has been
operating forever in reversed time.
It is the case that this reversedtime process is also
Western University (Ontario)  Also known as University of Western Ontario
Markov chains with application
STATISTICS 4654

Fall 2016
Textbook Sections 4.7
4.7 Branching Processes
A branching process is one model to study whether a
family line survives or eventually dies out.
One starts with a single individual, who produces j 0
o
Western University (Ontario)  Also known as University of Western Ontario
Markov chains with application
STATISTICS 4654

Fall 2016
Textbook Section 4.1
A Stochastic Process:
A stochastic process is defined in your text as
a collection of random variables (p. 84).
A better definition is that a stochastic process is this:
A stoch
Western University (Ontario)  Also known as University of Western Ontario
Markov chains with application
STATISTICS 4654

Fall 2016
Textbook Sections 4.3, 4.4
The reason to classify states
As we have seen through many examples, it is possible
for there to be a number of possible outcomes when
one investigates as n gets large.
It
Western University (Ontario)  Also known as University of Western Ontario
Markov chains with application
STATISTICS 4654

Fall 2016
Textbook Section 4.3
Example 4.18 The Random Walk
Consider a Markov chain defined on all the integers:
An individual starts at at an arbirtrary integer (which
we might as well consider to be = 0).
Western University (Ontario)  Also known as University of Western Ontario
Markov chains with application
STATISTICS 4654

Fall 2016
Textbook Section 4.2
The ChapmanKolmogorov equations
Let () = + = = for all 0 where
is a Markov chain. Then for all m, > 0; , 0,
(+) =
()
()
=0
We will prove this result in class.
If we look at
Western University (Ontario)  Also known as University of Western Ontario
Markov chains with application
STATISTICS 4654

Fall 2016
Section 4.9
A direct way to craft the
probabilities , via the , s
Our decision to make this a timereversible Markov
chain provides us with the means to determine the
, s. Since time reversibility m
Western University (Ontario)  Also known as University of Western Ontario
Markov chains with application
STATISTICS 4654

Fall 2016
Section 4.9
Background
Suppose that you are interested not so much in terms
of a particular random variable X, but rather some
function h(X) of that random variable.
Suppose further that it is the e