markov_chain_practice_problems

# markov_chain_practice_problems - 9 Consider a Markov chain...

This preview shows page 1. Sign up to view the full content.

Stat 333 Summer 2007 Markov Chain: Practice Problems The following problems are from Chapter 4 of Ross (plus a few extra thrown in). The problem numbers are the same in both the 8th and 9th editions. Solutions will be provided for most of these problems. 1. #2, #3 2. #8 3. #13 4. #14 Find the classes for each chain and identify which classes are open, which classes are closed. Identify which states are transient, which states are recurrent. 5. #15, #20, #22 6. #25 7. #33, #57 8. Consider a modi±ed symmetric random walk on { 0 , 1 , . . ., N } with re²ecting barriers at both 0 (re²ecting to 1 once reaching 0) and N (re²ecting to N - 1 once reaching N ). For intermediate states 1 j N - 1, we have P j,j +1 = P j,j - 1 = 1 / 2. Find the unique equilibrium distribution of this chain. (Hint: guess at the solution and verify it rather than trying to derive it through a system of equations.) Find the expected number of steps between returns to 0, and the expected number of steps between returns to 1.
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 9. Consider a Markov chain with state space S = { , 1 , . . ., 7 } and transition matrix 1 / 2 1 / 4 1 / 4 1 / 4 3 / 4 1 / 3 2 / 3 1 / 5 1 / 5 1 / 5 1 / 5 1 / 5 1 / 6 1 / 3 1 / 6 1 / 6 1 / 6 1 1 / 4 3 / 4 2 / 3 1 / 3 (a) Determine the classes of this chain, and organize the matrix into simple form. Which states are transient? Determine the period of each class. (b) Find the equilibrium distribution corresponding to each closed class, and write down the general form of all equilibrium distributions for this chain. (c) Find the absorption probability from each transient state into each closed class. (d) If X = 0, describe the long-run behavior of this chain. Do the same for X = 3. 1...
View Full Document

{[ snackBarMessage ]}

Ask a homework question - tutors are online