# 9 - Substituting into the normalization equation 1 + (1...

This preview shows pages 1–3. Sign up to view the full content.

Substituting into the normalization equation π 1 + ` i =1 π (2 ,i ) = 1, we obtain 1= 1+ b ` X i =1 (1 r ) i 1 ! π 1 = b ( 1 (1 r ) ` ) r ! π 1 , or π 1 = r r + b ( 1 (1 r ) ` ) . Using the equation π (2 ,i ) =(1 r ) i 1 1 , we can also obtain explicit formulas for the π (2 ,i ) . Solution to Problem 7.11. We use a Markov chain model with 3 states, H , M , and E , where the state reﬂects the diﬃculty of the most recent exam. We are given the transition probabilities # r HH r HM r HE r MH r MM r ME r EH r EM r EE \$ = # 0 . 5 . 5 . 25 . 5 . 25 . 25 . 25 . 5 \$ . It is easy to see that our Markov chain has a single recurrent class, which is aperiodic. The balance equations take the form π 1 = 1 4 ( π 2 + π 3 ) , π 2 = 1 2 ( π 1 + π 2 )+ 1 4 π 3 , π 3 = 1 2 ( π 1 + π 3 1 4 π 2 , and solving these with the constraint i π i =1g ives π 1 = 1 5 2 = π 3 = 2 5 . Solution to Problem 7.12. (a) This is a generalization of Example 7.6. We may proceed as in that example and introduce a Markov chain with states 0 , 1 ,...,n , where state i indicates that there i available rods at Alvin’s present location. However, that Markov chain has a somewhat complex structure, and for this reason, we will proceed diFerently. We consider a Markov chain with states 0 , 1 , where state i indicates that Alvin is oF the island and has i rods available. Thus, a transition in this Markov chain reﬂects two trips (going to the island and returning). It is seen that this is a birth-death process. This is because if there are i rods oF the island, then at the end of the round trip, the number of rods can only be i 1, i or i +1. We now determine the transition probabilities. When i> 0, the transition prob- ability p i,i 1 is the probability that the weather is good on the way to the island, but is bad on the way back, so that p i,i 1 = p (1 p ). When 0 <i<n , the transition prob- ability p i,i +1 is the probability that the weather is bad on the way to the island, but is 81

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
good on the way back, so that p i,i +1 = p (1 p ). For i = 0, the transition probability p i,i +1 = p 0 , 1 is just the probability that the weather is good on the way back, so that p 0 , 1 = p . The transition probabilities p ii are then easily determined because the sum of the transition probabilities out of state i must be equal to 1. To summarize, we have p ii = ± (1 p ) 2 + p 2 , for i> 0, 1 p, for i =0 , 1 p + p 2 , for i = n , p i,i +1 , = n (1 p ) p, for 0 <i<n , p, for i , p i,i 1 = n (1 p ) p, for 0, 0 , for i . Since this is a birth-death process, we can use the local balance equations. We have π 0 p 01 = π 1 p 10 , implying that π 1 = π 0 1 p , and similarly, π n = ··· = π 2 = π 1 = π 0 1 p . Therefore, 1= n X i =0 π i = π 0 1+ n 1 p ² , which yields π 0 = 1 p n +1 p i = 1 n p , for all 0 . (b) Assume that Alvin is o± the island. Let A denote the event that the weather is nice but Alvin has no ²shing rods with him. Then, P ( A )= π 0 p = p p 2 n p .
This is the end of the preview. Sign up to access the rest of the document.

## This note was uploaded on 01/11/2011 for the course MATH 170 taught by Professor Staff during the Spring '08 term at UCLA.

### Page1 / 10

9 - Substituting into the normalization equation 1 + (1...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online