{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

Section 4 Solutions

# Section 4 Solutions - ORIE3510 Introduction to Engineering...

This preview shows pages 1–3. Sign up to view the full content.

ORIE3510 Introduction to Engineering Stochastic Processes Spring 2010 Section 4 Review Stationary distribution interpretations Computation of π Steady-state costs/rewards Transient state analysis (expected number of visits to transient states & absorption proba- bilities) Problem 4.67 Note that if X n = i , then we can only move to i - 1 or i + 1, or stay in i . Hence P i,i +1 = P ( X n +1 = i + 1 | X n = i, X n - 1 , . . . , X 0 ) = p N - i N P i,i = P ( X n +1 = i | X n = i, X n - 1 , . . . , X 0 ) = p i N + (1 - p ) N - i N P i,i - 1 = P ( X n +1 = i - 1 | X n = i, X n - 1 , . . . , X 0 ) = (1 - p ) i N . Since the transition probabilities depend only on the fact that X n = i , we see that this is a Markov chain. Clearly, all classes communicate, so the chain is irreducible and hence recurrent. Further, we see that the period of state 0 is 1 so the chain is aperiodic. Suppose N = 2. Then we have P = 1 - p p 0 (1 - p ) 1 2 1 2 p 1 2 0 1 - p p and π 0 = (1 - p ) π 0 + 1 2 (1 - p ) π 1 π 1 = 2 p 1 - p π 0 = 2 1 p 1 (1 - p ) - 1 π 0 and π 2 = 1 2 1 + 2 π 2 = p 1 - p 2 π 0 = 2 2 p 2 (1 - p ) - 2 π 0 . Since i π i = 1 we have π 0 = 1 + 2 X i =1 2 i p i (1 - p ) - i ! - 1 = 1 + (1 - p ) - 2 2 X i =1 2 i p i (1 - p ) 2 - i ! - 1 = ( 1 + (1 - p ) - 2 ( 1 - (1 - p ) 2 )) - 1 = (1 - p ) 2 so π 0 = (1 - p ) 2 . It then follows that π i = ( 2 i ) p i (1 - p ) 2 - i for i = 1 , 2.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
For general N
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 4

Section 4 Solutions - ORIE3510 Introduction to Engineering...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online