ass2sol - STAT455/855 Fall 2009 Applied Stochastic...

This preview shows pages 1–3. Sign up to view the full content.

STAT455/855 Fall 2009 Applied Stochastic Processes Assignment #2, Solutions Total Marks: 55 for 455 and 65 for 855. 1. From Sheet. (10 marks) (a) (4 marks) (i) (4 marks) Let D k and N k denote, respectively, the period and the number of equivalence classes of the Markov chain whose transition matrix is given by P k . By considering various example values of d and k one can guess that the correct values of D k and N k should be D k = d gcd( d,k ) and N k = gcd( d,k ) . (ii) (8 bonus marks) For k = 1, the values D 1 = d and N 1 = 1 are clearly correct. So let us assume k ∈ { 2 ,...,d } . We use the following notation. Let X denote the original Markov chain and let X ( k ) denote the Markov chain whose transition matrix is P k . We use the term “1-step” to mean a single step in the original Markov chain X and the term “ k -step” to mean a single step in the Markov chain X ( k ) (so a single k -step consists of k 1-steps of X ). Let D k and N k be as given in part(i) and let C 0 ,...,C d - 1 denote the cyclic classes of X . Let p ij ( n ) denote the n -step transition probability from i to j in the original chain and let p ( k ) ij ( n ) denote the “ n -step” transition probability from i to j in the chain with transition matrix P k , so each step here is a k -step. We divide the proof into 4 steps: Step 1. Proof that p ij ( nd ) > 0 for all n big enough for i and j in the same cyclic class: From class we have that if n 1 ,n 2 ,... is an inﬁnite sequence of positive integers with gcd 1 then there is a ﬁnite subset of this sequence, say b 1 ,...,b r , that also has gcd 1 and a ﬁnite integer M such that for every n > M , there exist nonnegative integers d 1 ,...,d r (which depend on n ) such that n = d 1 b 1 + ... + d r b r . Now consider an arbitrary state j . Since the period is d each possible return time to j is a multiple of d . Let m 1 d,m 2 d,. .. be the times at which a return to state j has positive probability; i.e., p jj ( m s d ) > 0

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
STAT 455/855 -- Assignment 2 Solutions p. 2 for s = 1 , 2 ,... . Then the sequence m 1 ,m 2 ,... has gcd 1 (otherwise the gcd of the sequence m 1 d,m 2 d,. .. would be greater than d ) and so there exists a ﬁnite subset { b 1 ,...,b r } of { m 1 ,m 2 ,... } and a ﬁnite integer M j such that for every n > M j we can write n = d 1 b 1 + ... + d r b r for some nonnegative integer coeﬃcients d 1 ,...,d r . Then we have that, for n > M j , p jj ( nd ) = p jj (( d 1 b 1 + ... + d r b r ) d ) p jj ( d 1 b 1 d ) ...p jj ( d r b r d ) p jj ( b 1 d ) d 1 ...p jj ( b r d ) d r > 0 , (1) since p jj ( b s d ) > 0 for s = 1 ,...,r by assumption. More generally, if i and j are in the same cyclic class then by irreducibility there is some s i such that p ij ( s i d ) > 0. Then for n > M j we have p ij (( s i + n ) d ) p ij ( s i d ) p jj ( nd ) > 0. That is,
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 09/15/2010 for the course STAT 455/855 taught by Professor Glentakahara during the Fall '09 term at Queens University.

Page1 / 13

ass2sol - STAT455/855 Fall 2009 Applied Stochastic...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online