hmwk7_11

# hmwk7_11 - 2 A six-sided die is rolled repeatedly After...

This preview shows page 1. Sign up to view the full content.

Version: October 12, 2011 ISyE 3232 H. Ayhan Due: October 19th Homework 7 1. Let X 0 ,X 1 ,... be a Markov chain with state space { 0 , 1 , 2 } , initial distri- bution α = (1 / 5 , ? , 2 / 5), and transition matrix P = 1 / 5 4 / 5 ? 2 / 5 1 / 2 ? 0 1 / 10 ? Fill in the entries for P and α , and compute the following: (a) Compute Pr { X 1 = 0 | X 0 = 1 } . (b) The row vector α describes the distribution of X 0 . What is the row vector describing the distribution of X 1 ? (c) What is the row vector describing the distribution of X 15 ? (d) Is the Markov chain irreducible? Explain. (e) Is the Markov chain periodic or aperiodic? Explain and if it is peri- odic, also give the period. (f) Is the Markov chain positive recurrent? If so, why? If not, why not?
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 2. A six-sided die is rolled repeatedly. After each roll n = 1 , 2 ,... , let X n be the largest number rolled in the ﬁrst n rolls. Is { X n ,n ≥ 1 } a discrete-time Markov chain? If it’s not, show that it is not. If it is, answer the following questions: (a) What is the state space and the transition probabilities of the Markov chain? (b) What is the distribution of X 1 ? 3. Redo the previous problem except replace X n with Y n where Y n is the number of sixes among the ﬁrst n rolls. (So the ﬁrst question will be, is { Y n ,n ≥ 1 } a discrete-time Markov chain?) 1...
View Full Document

{[ snackBarMessage ]}

Ask a homework question - tutors are online