ISyE 3232
Stochastic Manufacturing and Service Systems
Spring 2011
Section A
J. G. Dai
Test 2 (April 7, 2011)
This is a closed book test. No calculator is allowed.
1. (30 points) Let
X
=
{
X
n
:
n
= 0
,
1
,
2
,...
}
be a discrete time Markov chain on state
space
S
=
{
1
,
2
,
3
,
4
}
with transition matrix
P
=
0
1
2
0
1
2
1
4
0
3
4
0
0
1
2
0
1
2
1
4
0
3
4
0
.
(a) Draw a transition diagram
(b) Find
P
{
X
2
= 2

X
0
= 2
}
.
(c) Find
P
{
X
2
= 4
,X
4
= 2
5
= 3

X
0
= 2
}
.
(d) What is the period of each state?
(e) Let
π
= (
1
8
,
1
4
,
3
8
,
1
4
). Is
π
the unique stationary distribution of
X
? Explain your
answer.
(f) Let
P
n
be the
n
th power of
P
. Does lim
n
→∞
P
n
2
,
3
=
3
8
hold? Explain your
answer.
(g) Let
τ
2
be the ﬁrst
n
≥
1 such that
X
n
= 2. Compute
E
(
τ
2

X
0
= 2). (If it takes
you a long time to compute it, you are likely on a wrong track.)
Solution.
(a)
1
2
3
4
1/2
3/4
1/2
1/4
1/2
3/4
1/2
1/4
1
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document(b)
P
{
X
2
= 2

X
0
= 2
}
=
P
21
P
12
+
P
23
P
32
=
1
4
×
1
2
+
3
4
×
1
2
=
1
2
.
(c)
P
{
X
2
= 4
,X
4
= 2
5
= 3

X
0
= 2
}
=
P
{
X
5
= 3

X
4
= 2
}
P
{
X
4
= 2

X
2
= 4
}
P
{
X
2
= 4

X
0
= 2
}
=
3
4
×
1
2
×
1
2
=
3
16
.
(d) Each state has period 2.
(e) YES. Since
π
= (
1
8
,
1
4
,
3
8
,
1
4
) satisfy
π
=
πP
and sum of the elements are 1, then
π
must be the stationary distribution. It is unique since this DTMC is irreducible.
(f) No. The limit does not exist since the DTMC has period 2.
(g)
E
(
τ
2

X
0
= 2) =
1
π
2
= 4.
2. (20 points) Let
P
=
.
2
.
8
0
0
0
.
5
.
5
0
0
0
0
.
5
0
.
5
0
0
0
.
25
0
.
75
0
0
0
0
1
Find approximately
P
100
.
Solution.
Solving
(
π
1
,π
2
)
±
.
2
.
8
.
5
.
5
²
= (
π
1
2
)
we have (
π
1
2
) = (
5
13
,
8
13
). Let
X
be the DTMC with transition matrix
P
on state
space
S
=
{
1
,
2
,
3
,
4
,
5
}
. Starting
X
0
= 3, let
f
3
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '07
 Billings
 Exponential distribution, Poisson process, DTMC

Click to edit the document details