MAST30001 Tute2 Sol 2022.pdf - MAST30001 Stochastic...

MAST30001 Stochastic Modelling
Tutorial Sheet 2
1. Show that the Markov property does not in general imply that for any event B , and
states i, j ,
P ( X
n +1
= j | X
n
∈ B, X
n - 1
= i ) = P ( X
n +1
= j | X
n
∈ B ) .
(Define a Markov chain, event B , and states i, j , where the equality doesn’t hold.)
2. Let ( Y
n
)
n ≥ 1
be i.i.d. random variables with P ( Y
i
= 1) = P ( Y
i
= - 1) = 1 / 2 and let
X
n
= ( Y
n +1
+ Y
n
) / 2.
(a) Find the “transition probabilities” P ( X
n + m
= k | X
n
= j ) for m = 1 , 2 , . . . and
j, k = 0 , ± 1. Hint: Separate the cases m = 1 and m 6 = 1 .
(b) Show that ( X
n
)
n ≥ 1
is not a Markov chain.
Ans.

3. Let ( X
n
) be a Markov chain with state space { 1 , 2 , 3 } and transition matrix


0 1 / 3 2 / 3
1 / 4 3 / 4 0
2 / 5 0 3 / 5


(a) Compute P ( X
3
= 1 , X
2
= 2 , X
1
= 2 | X
0
= 1).
(b) If X
0
is uniformly distributed on { 1 , 2 , 3 } , compute P ( X
3
= 1 , X
2
= 2 , X
1
= 2).
(c) Now assuming P ( X
0
= 1) = P ( X
0
= 3) = 2 / 5, compute P ( X
1
= 2 , X
4
=
2 , X
6
= 2).
(d) Assuming P ( X
0
= 1) = 1, use simulation to estimate the proportion of time
the chain spends in each state. Compare these proportions to the probability
that that the chain is in each state at time 100.
Ans.