{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

Assign4

Assign4 - Introduction to Stochastic Processes Spring 2011...

This preview shows pages 1–2. Sign up to view the full content.

Introduction to Stochastic Processes, Spring 2011 Homework #4 Due: Thursday, February 10th, 2011 in class Stationary Distributions for Markov Chains 1. Consider a two-state Markov Chain with the states labelled 0 and 1. Let the transition matrix be 1 - α α β 1 - β where 0 < α < 1 and 0 < β < 1. The first row is for state 0, the second is for state 1. (i) Compute the stationary distribution π = [ π 0 π 1 ]. (ii) For n 1, let f ( n ) = P (the first return of the Markov chain to state 0 is at time n | X 0 = 0) . Compute f ( n ) for n 1. (iii) Let m 0 be the mean number of time steps it takes for the Markov chain to return to zero, given that the chain starts at position zero. Use the formula for f ( n ) to compute m 0 . Check that π 0 = 1 /m 0 . 2. Recall the umbrella problem of Assignment 3: A man has a total of N umbrellas that he splits between home and work. Before travelling from one place to the other, he looks out the window and if it’s raining he takes an umbrella with him, otherwise he leaves without one. At any given time there is a 20% chance that it is raining. Suppose that at time zero he is at home, at time one he is at work, at time two he’s back home and so on, and let X j , j 0 be the number of umbrellas at his location at time j .

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

Page1 / 3

Assign4 - Introduction to Stochastic Processes Spring 2011...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online