This contradiction proves the desired result 14

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: is finite, using Lemma 1.12 1 X XX 1> Ex N (y ) = pn (x, y ) y 2C 1 X = X y 2C n=1 1 X pn (x, y ) = n=1 y 2C n=1 1=1 where in the next to last equality we have used that C is closed. This contradiction proves the desired result. 1.4 Stationary Distributions In the next section we will see that if we impose an additional assumption called aperiodicity an irreducible finite state Markov chain converges to a stationary distribution pn (x, y ) ! ⇡ (y ) To prepare for that this section introduces stationary distributions and shows how to compute them. Our first step is to consider What happens in a Markov chain when the initial state is random? Breaking things down according to the value of the initial state and using the definition of conditional probability X P (Xn = j ) = P (X0 = i, Xn = j ) i = X i P (X0 = i)P (Xn = j |X0 = i) If we introduce q (i) = P (X0 = i), then the last equation can be written as X P (Xn = j ) = q (i)pn (i, j ) (1.7) i In words, we multiply the transition matrix on the left by the vecto...
View Full Document

This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell University (Engineering School).

Ask a homework question - tutors are online