N j 1 with d j n i n j otherwise t jk n j n j 1 t jk

= n
j
- 1 with ( D
j
n )
i
= n
j
otherwise
( T
jk
n )
j
= n
j
- 1 , ( T
jk
n )
k
= n
k
= n
k
+ 1 , and ( T
j,k
n )
i
= n
j
otherwise

4.5. QUEUEING NETWORKS 207
Note that if n
j
= 0 then D
j
n and T
jk
n have - 1 in the j th coordinate,
so in this case q ( n, D
j
n ) and q ( n, T
jk
n ) = 0.
In equilibrium the rate at which probability mass leaves n is the
same as the rate at which it enters n . Taking into account the
various ways the chain can leave or enter state n it follows that
the condition for a stationary distribution πQ = 0 is equivalent to
π
i
λ
i
=
∑
j = i
π
j
q ( j, i ) which in this case is
π ( n )
K
k =1
q ( n, A
k
n ) +
K
j =1
q ( n, D
j
n ) +
K
j =1
K
k =1
q ( n, T
jk
n )
=
K
k =1
π ( A
k
n ) q ( A
k
n, n )
+
K
j =1
π ( D
j
n ) q ( D
j
n, n ) +
K
j =1
K
k =1
π ( T
jk
n ) q ( T
jk
n, n )
This will obviously be satisfied if we have
π ( n )
K
k =1
q ( n, A
k
n ) =
K
k =1
π ( A
k
n ) q ( A
k
n, n ) (4.32)
and for each j we have
π ( n ) q ( n, D
j
n ) +
K
k =1
q ( n, T
jk
n )
= π ( D
j
n ) q ( D
j
n, n ) +
K
k =1
π ( T
jk
n ) q ( T
jk
n, n ) (4.33)
Taking the second equation first, if n
j
= 0, then both sides are
0, since D
j
n and T
jk
n are not in the state space. Supposing that
n
j
> 0 and filling in the values of our rates (4.33) becomes
π ( n ) φ
j
( n
j
) q ( j ) +
K
k =1
p ( j, k )
= π ( D
j
n ) λ
j
+
K
k =1
π ( T
jk
n ) φ
k
( n
k
+ 1) p ( k, j ) (4.34)

208 CHAPTER 4. MARKOV CHAINS IN CONTINUOUS TIME
The definition of q implies q ( j ) +
∑
K
k =1
p ( j, k ) = 1, so filling in the
proposed formula for π ( n ) the left-hand side of (4.33) is
π ( n ) φ
j
( n
j
) =
K
i =1
c
i
r
n
i
i
ψ
i
( n
i
)
· φ
j
( n
j
) =
K
i =1
c
i
r
ˆ n
i
i
ψ
j
(ˆ n
i
)
· r
j
= π (ˆ n ) · r
j
where ˆ n = D
j
n has ˆ n
j
= n
j
- 1 and ˆ n
i
= n
i
for i = j . To compute
the right-hand side of (4.33) we note that ( T
jk
n )
i
= ˆ n
i
for i = k and
( T
jk
n )
k
= ˆ n
k
+ 1 = n
k
+ 1 so
π ( T
jk
n ) = π (ˆ n ) ·
r
k
φ
k
( n
k
+ 1)
Since D
j
n = ˆ n , we can rewrite the right-hand side of (4.33) as
= π (ˆ n ) λ
j
+ π (ˆ n )
K
k =1
r
k
p ( k, j ) = π (ˆ n ) · r
j
where the last equality follows from (4.27): λ
j
+
∑
k
r
k
p ( k, j ) = r
j
.
At this point we have verified (4.33). Filling our rates into (4.32)
and noting that π ( A
k
n ) = π ( n ) r
k
/φ
k
( n
k
+ 1) we want to show
π ( n )
K
k =1
λ
k
= π ( n )
K
k =1
r
k
φ
k
( n
k
+ 1)
· φ
k
( n
k
+ 1) q ( k ) (4.35)
To derive this, we note that summing (4.27) from j = 1 to K and
interchanging the order of summation in the double sum on the right
gives
K
j =1
r
j
=
K
j =1
λ
j
+
K
k =1
r
k
K
j =1
p ( k, j )
=
K
j =1
λ
j
+
K
k =1
r
k
-
K
k =1
r
k
q ( i )
since
∑
K
j =1
p ( i, j ) = 1 - q ( i ). Rearranging now gives
K
k =1
r
k
q ( k ) =
K
j =1
λ
j
This establishes (4.35), which implies (4.32), and completes the
proof.

4.6. CLOSED QUEUEING NETWORKS 209
4.6 Closed Queueing Networks
At first, the notion of N customers destined to move forever between
K servers may sound like a queueing hell that might be the subject
of a “Far Side” cartoon. However, as the next two examples show,
this concept is useful for applications.
Example 4.26. Manufacturing system. The production of a
part at a factory requires two operations. The first operation is
always done at machine 1. The second is done at machine 2 or ma-
chine 3 with probabilities p and 1 - p after which the part leaves
the system. Suppose that the factory has only a limited number of
palettes, each of which holds one part. When a part leaves the sys-