=
n
j

1
with (
D
j
n
)
i
=
n
j
otherwise
(
T
jk
n
)
j
=
n
j

1
,
(
T
jk
n
)
k
=
n
k
=
n
k
+ 1
,
and (
T
j,k
n
)
i
=
n
j
otherwise
4.5.
QUEUEING NETWORKS
207
Note that if
n
j
= 0 then
D
j
n
and
T
jk
n
have

1 in the
j
th coordinate,
so in this case
q
(
n, D
j
n
) and
q
(
n, T
jk
n
) = 0.
In equilibrium the rate at which probability mass leaves
n
is the
same as the rate at which it enters
n
.
Taking into account the
various ways the chain can leave or enter state
n
it follows that
the condition for a stationary distribution
πQ
= 0 is equivalent to
π
i
λ
i
=
∑
j
=
i
π
j
q
(
j, i
) which in this case is
π
(
n
)
K
k
=1
q
(
n, A
k
n
) +
K
j
=1
q
(
n, D
j
n
) +
K
j
=1
K
k
=1
q
(
n, T
jk
n
)
=
K
k
=1
π
(
A
k
n
)
q
(
A
k
n, n
)
+
K
j
=1
π
(
D
j
n
)
q
(
D
j
n, n
) +
K
j
=1
K
k
=1
π
(
T
jk
n
)
q
(
T
jk
n, n
)
This will obviously be satisfied if we have
π
(
n
)
K
k
=1
q
(
n, A
k
n
) =
K
k
=1
π
(
A
k
n
)
q
(
A
k
n, n
)
(4.32)
and for each
j
we have
π
(
n
)
q
(
n, D
j
n
) +
K
k
=1
q
(
n, T
jk
n
)
=
π
(
D
j
n
)
q
(
D
j
n, n
) +
K
k
=1
π
(
T
jk
n
)
q
(
T
jk
n, n
)
(4.33)
Taking the second equation first, if
n
j
= 0, then both sides are
0, since
D
j
n
and
T
jk
n
are not in the state space. Supposing that
n
j
>
0 and filling in the values of our rates (4.33) becomes
π
(
n
)
φ
j
(
n
j
)
q
(
j
) +
K
k
=1
p
(
j, k
)
=
π
(
D
j
n
)
λ
j
+
K
k
=1
π
(
T
jk
n
)
φ
k
(
n
k
+ 1)
p
(
k, j
)
(4.34)
208
CHAPTER 4.
MARKOV CHAINS IN
CONTINUOUS TIME
The definition of
q
implies
q
(
j
) +
∑
K
k
=1
p
(
j, k
) = 1, so filling in the
proposed formula for
π
(
n
) the lefthand side of (4.33) is
π
(
n
)
φ
j
(
n
j
) =
K
i
=1
c
i
r
n
i
i
ψ
i
(
n
i
)
·
φ
j
(
n
j
) =
K
i
=1
c
i
r
ˆ
n
i
i
ψ
j
(ˆ
n
i
)
·
r
j
=
π
(ˆ
n
)
·
r
j
where ˆ
n
=
D
j
n
has ˆ
n
j
=
n
j

1 and ˆ
n
i
=
n
i
for
i
=
j
. To compute
the righthand side of (4.33) we note that (
T
jk
n
)
i
= ˆ
n
i
for
i
=
k
and
(
T
jk
n
)
k
= ˆ
n
k
+ 1 =
n
k
+ 1 so
π
(
T
jk
n
) =
π
(ˆ
n
)
·
r
k
φ
k
(
n
k
+ 1)
Since
D
j
n
= ˆ
n
, we can rewrite the righthand side of (4.33) as
=
π
(ˆ
n
)
λ
j
+
π
(ˆ
n
)
K
k
=1
r
k
p
(
k, j
) =
π
(ˆ
n
)
·
r
j
where the last equality follows from (4.27):
λ
j
+
∑
k
r
k
p
(
k, j
) =
r
j
.
At this point we have verified (4.33). Filling our rates into (4.32)
and noting that
π
(
A
k
n
) =
π
(
n
)
r
k
/φ
k
(
n
k
+ 1) we want to show
π
(
n
)
K
k
=1
λ
k
=
π
(
n
)
K
k
=1
r
k
φ
k
(
n
k
+ 1)
·
φ
k
(
n
k
+ 1)
q
(
k
)
(4.35)
To derive this, we note that summing (4.27) from
j
= 1 to
K
and
interchanging the order of summation in the double sum on the right
gives
K
j
=1
r
j
=
K
j
=1
λ
j
+
K
k
=1
r
k
K
j
=1
p
(
k, j
)
=
K
j
=1
λ
j
+
K
k
=1
r
k

K
k
=1
r
k
q
(
i
)
since
∑
K
j
=1
p
(
i, j
) = 1

q
(
i
). Rearranging now gives
K
k
=1
r
k
q
(
k
) =
K
j
=1
λ
j
This establishes (4.35), which implies (4.32), and completes the
proof.
4.6.
CLOSED QUEUEING NETWORKS
209
4.6
Closed Queueing Networks
At first, the notion of
N
customers destined to move forever between
K
servers may sound like a queueing hell that might be the subject
of a “Far Side” cartoon. However, as the next two examples show,
this concept is useful for applications.
Example 4.26. Manufacturing system.
The production of a
part at a factory requires two operations.
The first operation is
always done at machine 1. The second is done at machine 2 or ma
chine 3 with probabilities
p
and 1

p
after which the part leaves
the system. Suppose that the factory has only a limited number of
palettes, each of which holds one part. When a part leaves the sys
You've reached the end of your free preview.
Want to read all 280 pages?
 Spring '08
 Chisholm
 Markov Chains, Probability, Markov chain, transition probability