# hw7.pdf - UC Berkeley Department of Electrical Engineering...

• Homework Help
• 4

This preview shows pages 1–3. Sign up to view the full content.

UC Berkeley Department of Electrical Engineering and Computer Sciences EECS 126: Probability and Random Processes Problem Set 7 Spring 2018 Issued: March 2, 2018 Due: Wednesday, March 7, 2018 1 . Two-State Chain with Linear Algebra Consider the Markov chain ( X n , n N ), shown in Figure 1 , where α, β (0 , 1). Figure 1: Markov chain for Problem 1 . (a) Find the probability transition matrix P . (b) Find two real numbers λ 1 and λ 2 such that there exists two non-zero vectors u 1 and u 2 such that Pu i = λ i u i for i = 1 , 2. Further, show that P can be written as P = U Λ U - 1 , where U and Λ are 2 × 2 matrices and Λ is a diagonal matrix. Hint: This is called the eigendecomposition of a matrix. (c) Find P n in terms of U and Λ for each n N . (d) Assume that X 0 = 0. Use the result in part (c) to compute the PMF of X n for all n N . (e) What does the fraction of time spent in state 0, n - 1 n i =1 { X i = 0 } , converge to (almost surely) as n → ∞ ? 2 . Reducible Markov Chain Consider the following Markov chain, for α, β, p, q (0 , 1). 0 1 2 3 4 5 1 - α α β 1 - β 1 / 2 1 / 2 1 / 2 1 / 2 p 1 - p q 1 - q 1

This preview has intentionally blurred sections. Sign up to view the full version.

(a) What are all of the communicating classes? (Two nodes x and y are said to belong to the same communicating class if x can reach y and y can reach x through paths of positive probability.) For each communicating class, classify it as recurrent or transient. (b) Given that we start in state 2, what is the probability that we will reach state 0 before state 5? (c) What are all of the possible stationary distributions of this chain? (Note that there is more than one.) (d) Suppose we start in the initial distribution π 0 := 0 0 γ 1 - γ 0 0 for some γ [0 , 1]. Does the distribution of the chain converge, and if so,
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern