This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: and 2 if blue. The transition probability matrix is P = 1 / 5 4 / 5 2 / 7 3 / 7 2 / 7 3 / 9 4 / 9 2 / 9 The limiting probabilities are obtained from π = 1 / 5 π + 2 / 7 π 1 + 1 / 3 π 2 π 1 = 3 / 7 π 1 + 4 / 9 π 2 π + π 1 + π 2 = 1 which can be solved to get π = 25 / 89, π 1 = 28 / 89, π 2 = 36 / 89. 26. Let the state be the ordering of the deck of n cards, so there are n ! states. The transition probabilities are P ( i 1 , i 2 , · · · , i n ) , ( i j , i 1 , i 2 , · · · , i j1 i j +1 , · · · , i n ) = 1 n This Markov Chain is doubly stochastic, so in the limit all n ! states are equally likely. 2...
View
Full Document
 Fall '08
 Lim
 Operations Research, Probability theory, Stochastic process, Markov chain, Andrey Markov, Markov decision process, πi Pij

Click to edit the document details