9 - Substituting into the normalization equation 1 + (1...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
Substituting into the normalization equation π 1 + ` i =1 π (2 ,i ) = 1, we obtain 1= 1+ b ` X i =1 (1 r ) i 1 ! π 1 = b ( 1 (1 r ) ` ) r ! π 1 , or π 1 = r r + b ( 1 (1 r ) ` ) . Using the equation π (2 ,i ) =(1 r ) i 1 1 , we can also obtain explicit formulas for the π (2 ,i ) . Solution to Problem 7.11. We use a Markov chain model with 3 states, H , M , and E , where the state reflects the difficulty of the most recent exam. We are given the transition probabilities # r HH r HM r HE r MH r MM r ME r EH r EM r EE $ = # 0 . 5 . 5 . 25 . 5 . 25 . 25 . 25 . 5 $ . It is easy to see that our Markov chain has a single recurrent class, which is aperiodic. The balance equations take the form π 1 = 1 4 ( π 2 + π 3 ) , π 2 = 1 2 ( π 1 + π 2 )+ 1 4 π 3 , π 3 = 1 2 ( π 1 + π 3 1 4 π 2 , and solving these with the constraint i π i =1g ives π 1 = 1 5 2 = π 3 = 2 5 . Solution to Problem 7.12. (a) This is a generalization of Example 7.6. We may proceed as in that example and introduce a Markov chain with states 0 , 1 ,...,n , where state i indicates that there i available rods at Alvin’s present location. However, that Markov chain has a somewhat complex structure, and for this reason, we will proceed diFerently. We consider a Markov chain with states 0 , 1 , where state i indicates that Alvin is oF the island and has i rods available. Thus, a transition in this Markov chain reflects two trips (going to the island and returning). It is seen that this is a birth-death process. This is because if there are i rods oF the island, then at the end of the round trip, the number of rods can only be i 1, i or i +1. We now determine the transition probabilities. When i> 0, the transition prob- ability p i,i 1 is the probability that the weather is good on the way to the island, but is bad on the way back, so that p i,i 1 = p (1 p ). When 0 <i<n , the transition prob- ability p i,i +1 is the probability that the weather is bad on the way to the island, but is 81
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
good on the way back, so that p i,i +1 = p (1 p ). For i = 0, the transition probability p i,i +1 = p 0 , 1 is just the probability that the weather is good on the way back, so that p 0 , 1 = p . The transition probabilities p ii are then easily determined because the sum of the transition probabilities out of state i must be equal to 1. To summarize, we have p ii = ± (1 p ) 2 + p 2 , for i> 0, 1 p, for i =0 , 1 p + p 2 , for i = n , p i,i +1 , = n (1 p ) p, for 0 <i<n , p, for i , p i,i 1 = n (1 p ) p, for 0, 0 , for i . Since this is a birth-death process, we can use the local balance equations. We have π 0 p 01 = π 1 p 10 , implying that π 1 = π 0 1 p , and similarly, π n = ··· = π 2 = π 1 = π 0 1 p . Therefore, 1= n X i =0 π i = π 0 1+ n 1 p ² , which yields π 0 = 1 p n +1 p i = 1 n p , for all 0 . (b) Assume that Alvin is o± the island. Let A denote the event that the weather is nice but Alvin has no ²shing rods with him. Then, P ( A )= π 0 p = p p 2 n p .
Background image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 01/11/2011 for the course MATH 170 taught by Professor Staff during the Spring '08 term at UCLA.

Page1 / 10

9 - Substituting into the normalization equation 1 + (1...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online