final_sol_2008 - MS&E 221 Ramesh Johari Final Examination...

Info icon This preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
MS&E 221 Final Examination - SOLUTIONS Ramesh Johari March 20, 2008 Part A (4 questions; 5 points each; 20 points total) Answer each of the following short answer questions. Explain your answers! (?? points per question) 1. True or false : Every Markov chain has at least one closed class. (Explain if true, and provide a counterexample if false.) Answer: FALSE. See Figure 1 for a counterexample. Note that it does not suffice to give an example of an infinite chain that is transient – remember that for infinite communicating classes, a class can be transient and still be closed! 2 points for correct answer; 3 points for correct justification. 2. You receive two types of e-mail: spam, and mail from friends. Spam arrives according to a Poisson process of rate α S , and mail from friends as a Poisson process of rate α F . There is a probability p that a piece of spam is caught by your spam filter, and never enters your inbox. Find the probability that at least 3 spam messages enter your inbox before the first message from a friend. Answer: Spam enters the inbox as a Poisson process of rate (1 - p ) α S . So the answer is: parenleftbigg (1 - p ) α S (1 - p ) α S + α F parenrightbigg 3 1 point for model; -1 point for minor math mistake. 3. True or false : Every Markov chain has at least one transient state. (Explain if true, and provide a counterexample if false.) Answer: FALSE. Any irreducible, recurrent chain is a counterexample. 2 points for correct answer; 3 points for correct justification. 4. Consider an irreducible discrete time Markov chain on the state space { 1 , 2 , . . . , N } , with transition matrix P . Suppose that all column sums are equal to one: i.e., for all j , i P ij = 1 . Find the unique invariant distribution of the resulting Markov chain. Is any such Markov chain reversible? Answer: The unique invariant distribution is the uniform distribution, π ( i ) = 1 /N for all i . For such a chain to be reversible (or more precisely, for there to exist a two-sided stationary version that is reversible), the transition matrix must be symmetric, i.e., P ij = P ji for all i, j . 2 points for writing basic invariant dist. equations; 1 point for solving; and 2 points for reversibility explanation. 1
Image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Part B (3 questions; 80 points total) Problem 1. Consider a discrete time Markov chain on states 1 , 2 , 3 , 4 , 5 , 6 with the following transition matrix: P = 1 / 2 1 / 2 0 0 0 0 1 / 2 0 1 / 2 0 0 0 1 / 2 0 1 / 2 0 0 0 1 / 6 1 / 6 1 / 6 1 / 6 1 / 6 1 / 6 1 / 2 0 0 0 0 1 / 2 0 0 0 1 / 2 1 / 2 0 (a) (5 points) Find the communicating classes. Which are transient? recurrent? closed?
Image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern