Markov

# Otherwise a recurrent state is called positive

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: , then the previous Proposition would imply that j  = 0 for all j , which would contradict the assumption that  is a probability distribution, and so must sum to 1. The previous Corollary says that for an irreducible Markov chain, the existence of a stationary distribution implies recurrence. However, we know that the converse is not true. That is, there are irreducible, recurrent Markov chains that do not have stationary distributions. For example, we have seen that the simple symmetric random walk on the integers in one dimension is irreducible and recurrent but does not have a stationary distribution. This random walk is recurrent all right, but in a sense it is just barely recurrent." That is, by recurrence we have P0 fT0 1g = 1, for example, but we also have E 0 T0  = 1. The name for this kind of recurrence is null recurrence : the state i is null recurrent if it is recurrent and E i Ti  = 1. Otherwise, a recurrent state is called positive recurrent : the state i is positive recurrent if E i Ti  1. A positive recurrent state i is not just barely recurrent, it is recurrent by a comfortable margin|when started at i, we have not only that Ti is nite almost surely, but also that Ti has nite expectation. Positive recurrence is in a sense the right notion to relate to the existence of a stationary distibution. For now let me state just the facts, ma'am; these will be justi ed later. Positive recurrence is also a class property, so that if a chain is irreducible, the chain is either transient, null recurrent, or positive recurrent. It turns out that an irreducible chain has a stationary distribution if and only if it is positive recurrent. That is, strengthening recurrence" to positive recurrence" gives the converse to Corollary 1.47. 1.7 An aside on coupling Coupling is a powerful technique in probability. It has a distinctly probabilistic avor. That is, using the coupling idea entails thinking probabilistically, as opposed to simply applying analysis or algebra or some other area of mathematics. Many people like to prove assertions using coupling and feel happy when they have done so|a probabilisitic assertion deserves a probabilistic proof, and a good coupling proof can make obvious what might otherwise Stochastic Processes J. Chang, March 30, 1999 1.7. AN ASIDE ON COUPLING Page 1-23 be a mysterious statement. For example, we will prove the Basic Limit Theorem of Markov chains using coupling. As I have said before, we could do it using matrix theory, but the probabilist tends to nd the coupling proof much more appealing, and I hope you do too. It is a little hard to give a crisp de nition of coupling, and di erent people vary in how they use the word and what they feel it applies to. Let's start by discussing a very simple example of coupling, and then say something about what the common ideas are. 1.48 Example Connectivity of a random graph . A graph is said to be connected if for each pair of distinct nodes i and j there is a path from i to j that consists of edges of the graph. Consider a random graph on a g...
View Full Document

## This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell.

Ask a homework question - tutors are online