6.262 Discrete Stochastic Processes, Spring 2010
Problem Set 9 — Solutions
due: Friday, April 23, 2010
Problem 1
(Exercise 5.10)
a)
M/M/
1
:
From (5.40), we have
π
i
=
ρ
i
(1

ρ
) for
i
≥
0 where
ρ
=
λ/μ
and
ρ <
1 (positive
recurrent).
M/M/m
:
First note that in the Markov chain, with
k
customers in service,
P
{
departure in (
t,t
+
δ
]
}
=
kμδ
+
o
(
δ
)
.
So, while the forward jump probability is still
λδ
, the backward jump probability is (min
{
k,m
}
μδ
).
Recalling that a recurrent birthdeath chain is positive recurrent if and only if it satisﬁes
Equation (5.38), deﬁning
ρ
=
λ/
(
μm
) we have
π
i
π
i

1
=
λ
iμ
for
i < m
and
π
i
π
i

1
=
ρ
for
i
≥
m,
which simpliﬁes to
π
i
=
( ±
λ
μ
²
i
π
0
i
!
, i < m
ρ
i π
0
m
m
m
!
,
i
≥
m
.
Since
∑
i
π
i
= 1, we have
π
0
=
1 +
m

1
X
i
=1
(
λ/μ
)
i
i
!
+
∞
X
i
=
m
ρ
i
m
m
m
!
!

1
=
1 +
m

1
X
i
=1
(
λ/μ
)
i
i
!
+
(
mρ
)
m
m
!(1

ρ
)
!

1
Finally, the other
π
i
can be obtained from the above relationships.
M/M/
∞
:
Following an analogous reasoning to the
M/M/m
case without a cutoﬀ at
m
, we
obtain
π
i
=
³
λ
μ
´
i
π
0
i
!
for all
i
≥
0
.
Using the Taylor series expansion of
e
λ/μ
=
∑
∞
i
=0
(
λ/μ
)
i
i
!
, we see that
π
0
=
e

λ/μ
. Thus,
π
i
=
³
λ
μ
´
i
e

λ/μ
i
!
,
≥
0
.
1
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Documentb)
M/M/
1
:
Recalling our results for birthdeath chains (Section 5.2), for the chain to be tran
sient we need
λ/μ >
1, for null recurrent
λ/μ
= 1, and for positive recurrent
λ/μ <
1.
M/M/m
:
Finitely many states do not aﬀect the transient/recurrent nature of an irreducible
chain (why?). It suﬃces therefore to look at the chain corresponding to states
m,m
+ 1
,...
.
For the chain to be transient, we need
λ/mμ >
1, for null recurrent,
λ/mμ
= 1, for positive
recurrent
λ/mμ <
1.
M/M/
∞
:
For the chain to be transient, we need
λ >
0 and
μ
= 0 (i.e. customers arrive
but they do not depart). We cannot have null recurrence for
μ >
0, as for any value of
λ
letting
n
=
b
λ/μ
c
+ 1 yields that
mμ > λ
for all
m
≥
n
. The chain corresponding to states
n,n
+ 1
,...
is therefore positive recurrent, and, taking into account the ﬁnitestate recurrent
chain corresponding to states 1
,
2
,...,n
, the result extends to the rest of the chain. Thus, for
This is the end of the preview. Sign up
to
access the rest of the document.
 Spring '08
 Moon,J
 Probability theory, Markov chain, Andrey Markov, Random walk, recurrent birthdeath chain

Click to edit the document details