Unformatted text preview: , then the previous Proposition would imply that j = 0 for all j ,
which would contradict the assumption that is a probability distribution, and so must
sum to 1.
The previous Corollary says that for an irreducible Markov chain, the existence of a
stationary distribution implies recurrence. However, we know that the converse is not
true. That is, there are irreducible, recurrent Markov chains that do not have stationary
distributions. For example, we have seen that the simple symmetric random walk on
the integers in one dimension is irreducible and recurrent but does not have a stationary
distribution. This random walk is recurrent all right, but in a sense it is just barely
recurrent." That is, by recurrence we have P0 fT0 1g = 1, for example, but we also have
E 0 T0 = 1. The name for this kind of recurrence is null recurrence : the state i is null
recurrent if it is recurrent and E i Ti = 1. Otherwise, a recurrent state is called positive
recurrent : the state i is positive recurrent if E i Ti 1. A positive recurrent state i is not
just barely recurrent, it is recurrent by a comfortable marginwhen started at i, we have
not only that Ti is nite almost surely, but also that Ti has nite expectation.
Positive recurrence is in a sense the right notion to relate to the existence of a stationary
distibution. For now let me state just the facts, ma'am; these will be justi ed later. Positive
recurrence is also a class property, so that if a chain is irreducible, the chain is either
transient, null recurrent, or positive recurrent. It turns out that an irreducible chain has
a stationary distribution if and only if it is positive recurrent. That is, strengthening
recurrence" to positive recurrence" gives the converse to Corollary 1.47. 1.7 An aside on coupling
Coupling is a powerful technique in probability. It has a distinctly probabilistic avor. That
is, using the coupling idea entails thinking probabilistically, as opposed to simply applying
analysis or algebra or some other area of mathematics. Many people like to prove assertions
using coupling and feel happy when they have done soa probabilisitic assertion deserves
a probabilistic proof, and a good coupling proof can make obvious what might otherwise
Stochastic Processes J. Chang, March 30, 1999 1.7. AN ASIDE ON COUPLING Page 123 be a mysterious statement. For example, we will prove the Basic Limit Theorem of Markov
chains using coupling. As I have said before, we could do it using matrix theory, but the
probabilist tends to nd the coupling proof much more appealing, and I hope you do too.
It is a little hard to give a crisp de nition of coupling, and di erent people vary in how
they use the word and what they feel it applies to. Let's start by discussing a very simple
example of coupling, and then say something about what the common ideas are.
1.48 Example Connectivity of a random graph . A graph is said to be connected if for each pair of distinct nodes i and j there is a path from i to j that consists of edges of
the graph. Consider a random graph on a g...
View
Full
Document
This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell.
 Spring '10
 DURRETT
 Multiplication, Markov Chains

Click to edit the document details