hmwk7_11 - 2. A six-sided die is rolled repeatedly. After...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
Version: October 12, 2011 ISyE 3232 H. Ayhan Due: October 19th Homework 7 1. Let X 0 ,X 1 ,... be a Markov chain with state space { 0 , 1 , 2 } , initial distri- bution α = (1 / 5 , ? , 2 / 5), and transition matrix P = 1 / 5 4 / 5 ? 2 / 5 1 / 2 ? 0 1 / 10 ? Fill in the entries for P and α , and compute the following: (a) Compute Pr { X 1 = 0 | X 0 = 1 } . (b) The row vector α describes the distribution of X 0 . What is the row vector describing the distribution of X 1 ? (c) What is the row vector describing the distribution of X 15 ? (d) Is the Markov chain irreducible? Explain. (e) Is the Markov chain periodic or aperiodic? Explain and if it is peri- odic, also give the period. (f) Is the Markov chain positive recurrent? If so, why? If not, why not?
Background image of page 1
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 2. A six-sided die is rolled repeatedly. After each roll n = 1 , 2 ,... , let X n be the largest number rolled in the rst n rolls. Is { X n ,n 1 } a discrete-time Markov chain? If its not, show that it is not. If it is, answer the following questions: (a) What is the state space and the transition probabilities of the Markov chain? (b) What is the distribution of X 1 ? 3. Redo the previous problem except replace X n with Y n where Y n is the number of sixes among the rst n rolls. (So the rst question will be, is { Y n ,n 1 } a discrete-time Markov chain?) 1...
View Full Document

Ask a homework question - tutors are online