This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: MS&E 221 Final Examination  SOLUTIONS Ramesh Johari March 20, 2008 Part A (4 questions; 5 points each; 20 points total) Answer each of the following short answer questions. Explain your answers! (?? points per question) 1. True or false : Every Markov chain has at least one closed class. (Explain if true, and provide a counterexample if false.) Answer: FALSE. See Figure 1 for a counterexample. Note that it does not suffice to give an example of an infinite chain that is transient – remember that for infinite communicating classes, a class can be transient and still be closed! 2 points for correct answer; 3 points for correct justification. 2. You receive two types of email: spam, and mail from friends. Spam arrives according to a Poisson process of rate α S , and mail from friends as a Poisson process of rate α F . There is a probability p that a piece of spam is caught by your spam filter, and never enters your inbox. Find the probability that at least 3 spam messages enter your inbox before the first message from a friend. Answer: Spam enters the inbox as a Poisson process of rate (1 p ) α S . So the answer is: parenleftbigg (1 p ) α S (1 p ) α S + α F parenrightbigg 3 1 point for model; 1 point for minor math mistake. 3. True or false : Every Markov chain has at least one transient state. (Explain if true, and provide a counterexample if false.) Answer: FALSE. Any irreducible, recurrent chain is a counterexample. 2 points for correct answer; 3 points for correct justification. 4. Consider an irreducible discrete time Markov chain on the state space { 1 , 2 ,...,N } , with transition matrix P . Suppose that all column sums are equal to one: i.e., for all j , ∑ i P ij = 1 . Find the unique invariant distribution of the resulting Markov chain. Is any such Markov chain reversible? Answer: The unique invariant distribution is the uniform distribution, π ( i ) = 1 /N for all i . For such a chain to be reversible (or more precisely, for there to exist a twosided stationary version that is reversible), the transition matrix must be symmetric, i.e., P ij = P ji for all i,j . 2 points for writing basic invariant dist. equations; 1 point for solving; and 2 points for reversibility explanation. 1 Part B (3 questions; 80 points total) Problem 1. Consider a discrete time Markov chain on states 1 , 2 , 3 , 4 , 5 , 6 with the following transition matrix: P = 1 / 2 1 / 2 1 / 2 1 / 2 1 / 2 1 / 2 1 / 6 1 / 6 1 / 6 1 / 6 1 / 6 1 / 6 1 / 2 1 / 2 1 / 2 1 / 2 (a) (5 points) Find the communicating classes. Which are transient? recurrent? closed?...
View
Full
Document
This note was uploaded on 02/03/2011 for the course MS&E 221 taught by Professor Ramesh during the Winter '11 term at Stanford.
 Winter '11
 Ramesh

Click to edit the document details