Solutions to Homework 7
October 24, 2011
1. According to the fact that the row summation of transition matrix P
equals one, we can ﬁll the missing values in P:
1
5
4
5
0
2
5
1
2
1
10
0
1
10
9
10
.
And
α
is the initial distribution, so the summation of its entries equals one
too. So we have
α
= (
1
5
,
2
5
,
2
5
)
.
a) Pr(
x
1
= 0

x
0
= 1
) =
P
1
,
0
=
2
5
.
b) We let
α
i
denote the distribution of
X
i
(so
α
0
=
α
). Then we have
α
i
=
α
0
P
i
. So
X
1
’s distribution
α
1
=
αP
= (
1
5
,
2
5
,
2
5
)
1
5
4
5
0
2
5
1
2
1
10
0
1
10
9
10
= (
1
5
,
2
5
,
2
5
)
.
c)
α
15
=
αP
15
. We ﬁnd from b)
α
1
=
αP
= (
1
5
,
2
5
,
2
5
) =
α
. So
αP
i
+1
=
αP
i
=
...
=
α
. Let
i
= 14
; we have
α
15
=
αP
15
=
α
.
d) It’s irreducible. Every state can communicate with each other.
e) State
0
can reach itself in one step. So the Markov Chain is aperiodic.
f) Yes, it’s postively recurrent. Because it’s a irreducible and ﬁnite Markov
Chain.
1
ISyE 3232
H. Ayhan
Stochastic Manufacturing and Service Systems
Fall 2011
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document2.
(a) In order to show that
{
X
n
,n
This is the end of the preview.
Sign up
to
access the rest of the document.
 Fall '07
 Billings
 Markov chain, Stochastic Manufacturing, H. Ayhan, step transition probabilities

Click to edit the document details