The Stationary Distribution of a Markov Chain
Alessandro Panconesi
DI, La Sapienza of Rome
May 15, 2005
In this note I present a concise proof of the existence and uniqueness of the limit
distribution of an ergodic markov chain.
This nice proof was described to me by
David Gilat of Hebrew University during a hot summer afternoon in Perugia.
A finite
n
×
n
transition probability matrix
P
:= [
p
ij
] is a stochastic matrix where
p
ij
is the
transition probability
of going from state
i
to state
j
.
We can think of
a directed graph whose edges are weighted with transition probabilities. Note that
since
P
is stochastic, the sum of the probabilities of the arcs outgoing from a vertex
i
sum up to 1, i.e.
∑
j
p
ij
= 1.
We are interested in keeping track of the random walk of a pebble. The pebble
is initially placed on the graph according to an initial distribution
X
0
and then it
proceeds by traversing the edges
ij
’s with their associated probabilities
p
ij
’s.
X
0
is
a probability distribution,
X
i
0
being the probability of placing the pebble on vertex
i
initially.
After one step the position of the pebble is given by the probability
distribution
X
1
=
X
0
P
and in general after
t
steps we have
X
t
=
X
t

1
P
=
X
0
P
t
.
The infinite sequence
X
0
, X
1
, X
2
, . . .
is called a
markov chain
. In what follows we shall
use the term markov chain somewhat loosely, sometimes referring to the sequence
proper, sometimes to the transition matrix
P
or the underlying graph. The meaning
will be clear from the context.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Fall '08
 Mckinley,S
 Cone, Markov chain

Click to edit the document details