33 random walks on graphs a graph is described by

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ain row 1, we note that the state will only decrease by 1 if one machine breaks and the repairman fails to repair the one he is working on, an event of probability (.1)(.5), while the state can only increase by 1 if he succeeds and there is no new failure, an event of probability .5(.9). Similar reasoning shows p(2, 1) = (.2)(.5) and p(2, 3) = .5(.8). To find the stationary distribution we use the recursive formula (1.12) to 34 CHAPTER 1. MARKOV CHAINS conclude that if ⇡ (0) = c then p0 0.5 =c· = 10c q1 0.05 p1 0.45 ⇡ (2) = ⇡ (1) · = 10c · = 45c q2 0.1 0.4 p2 = 45c · ⇡ (3) = ⇡ (2) · = 60c q3 0.3 ⇡ (1) = ⇡ (0) · The sum of the ⇡ ’s is 116c, so if we let c = 1/116 then we get ⇡ (3) = 60 , 116 ⇡ (2) = 45 , 116 ⇡ (1) = 10 , 116 ⇡ (0) = 1 116 There are many other Markov chains that are not birth and death chains but have stationary distributions that satisfy the detailed balance condition. A large number of possibilities are provided by Example 1.33. Random walks on graphs. A graph...
View Full Document

This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell University (Engineering School).

Ask a homework question - tutors are online