STAT 333 Assignment 3 SOLUTIONS
1.
Consider a discrete
time Markov Chain with state space S = {0, …, 7} and transition matrix
P
=
1/2
1/4
0
0
0
0
1/4
0
1/4
0
0
0
0
0
3/4
0
0
0
1/3
0
0
0
0
2/3
0
1/5
1/5
1/5
1/5
1/5
0
0
1/6
0
0
0
1/3
1/6
1/6
1/6
0
0
0
0
0
1
0
0
1/4
0
0
0
0
0
3/4
0
0
0
2/3
0
0
0
0
1/3
a.
Determine the classes of this chain, which are open or closed, write
P
in simplified form,
and find the period of each closed class.
C
1
= {0, 1, 6}
–
closed so positive recurrent
–
aperiodic
C
2
= {2, 7}
–
closed so positive recurrent
–
aperiodic
C
3
= {3}
–
open so transient
C
4
= {4}
–
open so transient
C
5
= {5}
–
closed so positive recurrent
–
aperiodic
P
sim
=
1/2
1/4
1/4
0
0
0
0
0
1/4
0
3/4
0
0
0
0
0
1/4
0
3/4
0
0
0
0
0
0
0
0
1/3
2/3
0
0
0
0
0
0
2/3
1/3
0
0
0
0
0
0
0
0
1
0
0
0
1/5
0
1/5
0
1/5
1/5
1/5
1/6
0
1/6
0
1/6
1/6
0
1/3
with new order of states {0, 1, 6, 2, 7, 5, 3, 4}
b.
Find the equilibrium distribution corresponding to each closed class. Write down the
general form of all stationary distributions for this chain.
Within C
2
, we can see the submatrix is doubly stochastic so equilibrium is (½ , ½)
Within C
5
, there is only one state, so equilibrium is (1)
Within C
1
, we get
½ π
0
+ ¼ π
1
+ ¼ π
6
= π
0
¼ π
0
= π
1
π
0
+ π
1
+ π
6
= 1
solving gives (1/3, 1/12, 7/12)
The general form is:
α (1/3, 1/1
2
, 0, 0, 0, 0, 7/12, 0) + β (0, 0, ½, 0, 0, 0, 0, ½) + (1–
α –
β)(0, 0, 0, 0, 0, 1, 0, 0)
where α, β, and (1 –
α –
β) are between 0 and 1.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
c.
Find the absorption probabilities from each transient state into each closed class.
A
4, C1
= P
4, C1
+ P
4, 4
*A
4, C1
= (1/6 + 1/6) + 1/3 *A
4, C1
So A
4, C1
= ½
A
4, C2
= P
4, C2
+ P
4, 4
*A
4, C2
= 1/6 + 1/3 *A
4, C2
So A
4, C1
= ¼
So A
4, C5
= 1
–
A
4, C1
–
A
4, C2
= ¼
A
3, C1
= P
3, C1
+ P
3, 3
*A
3, C1
+ P
3, 4
*A
4, C1
= 1/5 + 1/5 *A
3, C1
+ 1/5*1/2
So A
3, C1
= 3/8
A
3, C2
= P
3, C2
+ P
3, 3
*A
3, C2
+ P
3, 4
*A
4, C2
= 1/5 + 1/5 *A
3, C1
+ 1/5*1/4
So A
3, C1
= 5/16
So A
3, C5
= 1
–
A
3, C1
–
A
3, C2
= 5/16
d.
Describe the longrun behaviour of the chain if X
0
= 0. Do the same if X
0
= 3.
If X0 = 0, we are starting in closed class C1, so the longrun behaviour is
(1/2, 1/12, 0, 0, 0, 0, 7/12, 0)
If X0 = 3, the longrun behavior is
(3/8)(1/3, 1/12, 0, 0, 0, 0, 7/12, 0)+(5/16)(0, 0, ½, 0, 0, 0, 0, ½)+(5/16)(0, 0, 0, 0, 0, 1,0,0)
2.
Consider a discretetime Markov Chain with five transient states (1, 2, 3, 4, and 5) and two
recurrent states (6 and 7). The chain is equally likely to start in any of the transient
states.
The transition matrix is
P
=
1/2
0
1/4
0
0
0
1/4
0
1/3
0
1/3
0
1/3
0
1/5
1/5
1/5
1/5
1/5
0
0
0
0
0
1/4
1/4
1/4
1/4
2/3
0
1/3
0
0
0
0
0
0
0
0
0
1/2
1/2
0
0
0
0
0
1/3
2/3
Note: The R code used is available in a separate document.
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '08
 Chisholm
 Probability theory, Exponential distribution, Poisson process, random variable equals

Click to edit the document details