ISyE 3232
Stochastic Manufacturing and Service Systems
Spring 2011
H.Ayhan and J. Dai
Solutions to Homework 8
1.
(a) The state space is
{
$10
,
$20
}
. The transition matrix is
0
.
8
0
.
2
0
.
1
0
.
9
. It is irreducible.
(b) The state space is
{
$10
,
$25
}
. The transition matrix is
0
.
9
0
.
1
0
.
15
0
.
85
. It is irreducible.
(c) The stationary distribution is (
π
X
$10
, π
X
$20
) = (1
/
3
,
2
/
3).
(d) The stationary distribution is (
π
Y
$10
, π
Y
$25
) = (3
/
5
,
2
/
5).
(e) What you need look at is
E
(
∑
300
i
=1
X
i
) and
E
(
∑
300
i
=1
Y
i
). And choose the one with larger
expectation. By the stationary distribution obtained in (b) and (c), we have
E
(
300
X
i
=1
X
i
) = 300(10
×
1
3
+ 20
×
2
3
) = 5000
,
E
(
300
X
i
=1
Y
i
) = 300(10
×
3
5
+ 25
×
2
5
) = 4800
,
so you should pick stock 1.
2.
(a) Yes, it is a Markov Chain. The state space is
{
0
,
1
,
2
,
3
,
· · · }
.
a
= (1
,
0
,
0
,
· · ·
)
,
P
i,j
=
98
/
100
j
=
i
+ 1;
2
/
100
j
= 0
.
(b) Yes, it is irreducible.
Starting from any state
i
, the probability of visiting state 0 in
one step is 2
/
100. And starting from state 0, the probability of visiting state
i
in
i
step
is (98
/
100)
i
.
Therefore, state 0 communicates with any other state.
Thus any state
commutes with each other.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '07
 Billings
 Markov chain, stationary distribution

Click to edit the document details