Stochastic

# Here we consider 5 to be adjacent to 0 so if we are

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 2 . . . , L}. The chain goes to the right or left at each step with probability 1/2, subject to the rules that if it tries to go to the left from 0 or to the right from L it stays put. For example, when L = 4 the transition probability is 0 1 2 3 4 0 0.5 0.5 0 0 0 1 0.5 0 0.5 0 0 2 0 0.5 0 0.5 0 30 0 0.5 0 0.5 40 0 0 0.5 0.5 It is clear in the example L = 4 that each column adds up to 1. With a little thought one sees that this is true for any L, so the stationary distribution is uniform, ⇡ (i) = 1/(L + 1). 30 CHAPTER 1. MARKOV CHAINS Example 1.26. Tiny Board Game. Consider a circular board game with only six spaces {0, 1, 2, 3, 4, 5}. On each turn we roll a die with 1 on three sides, 2 on two sides, and 3 on one side to decide how far to move. Here we consider 5 to be adjacent to 0, so if we are there and we roll a 2 then the result is 5 + 2 mod 6 = 1, where i + k mod 6 is the remainder when i + k is divided by 6. In this case the transition probability is 0 1 2 3 4 5 0 0 0 0 1/6 1/3 1/2 1 2 3 4 5 1/3 1/3 1/6 0 0 0 1/2 1/3 1/6 0 0 0 1/2 1/3 1/6 0 0 0 1 / 2 1 /3 1/6 0 0 0 1 /2 1/3 1/6 0 0 0 It is clear...
View Full Document

## This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell.

Ask a homework question - tutors are online