This preview shows page 1. Sign up to view the full content.
Unformatted text preview: Probability and Statistics with Reliability,
Queuing and Computer Science Applications:
second edition, by K.S. Trivedi
PublisherJohn Wiley & Sons
Chapter 9: Networks of Queues
Dept. of Electrical & Computer
engineering
Duke University
Email: kst@ee.duke.edu
URL: www.ee.duke.edu/~kst
Copyright © 2003 by K.S. Trivedi 1 Classes of Queuing Networks
Tandem Queues
Open queuing networks (Jackson’s network)
Closed queuing networks (GordonNewell network)
Multi class queuing networks (BCMP queues)
Nonproductform queuing networks.
In the last segment we will discuss the computation of
response time distribution (or percentiles) in
networks
Copyright © 2003 by K.S. Trivedi
y Networks of Queues
Two types of networks : Open and Closed
An open queuing network is characterized by one or more
sources of job arrivals and correspondingly one or more sinks
that absorb jobs departing from the network.
In a closed queuing network, jobs neither enter nor depart from
the network.
The behavior of jobs within the network is characterized by
the distribution of job service times at each center
the probabilities of transitions between service centers
For each center the number of servers, the scheduling discipline,
and the size of the queue must be specified.
For an open network, a characterization of jobarrival
processes is needed.
For a closed network, the number of jobs in the network must
be specified.
Copyright © 2003 by K.S. Trivedi
y Open Queuing Networks
M/M/1, etc. : single node queuing network
Two M/M/1 queues in tandem
Poisson stream Exponentially dist. service times at s0 and s1.
Underlying stochastic process is an HCTMC
State: (k0,k1),
ki: number of jobs at node i, i=0,1
The changes of state occur upon a completion of service at one of the
two servers or upon an external arrival.
Since all interevent times are exponentially distributed (by our
assumptions), the underlying stochastic process is a homogeneous
CTMC with the state diagram shown on the next page. Copyright © 2003 by K.S. Trivedi
y Tandem queuestate diagram
CTMC state diagram: Copyright © 2003 by K.S. Trivedi
y Tandem queue CTMC solution
Following steady state solution can be shown to
satisfy the CTMC balance equations:
Solution has a product form, i.e., product of the
solution of two independent M/M/1 queues.
For an M/M/1 queue: Burke showed that the
output process is also Poisson with rate λ
Therefore, for the two queue tandem network, the
second queue is also an independent M/M/1 queue.
Hence the product form of the result holds. Copyright © 2003 by K.S. Trivedi
y Tandem queue product form
Generalization to an nnode tandem network The solution can be shown to satisfy the balance
equations of the underlying CTMC
The solution can also be derived by repeatedly
invoking Burke’s result.
Copyright © 2003 by K.S. Trivedi
y Tandem Queuesexample
Repair facility: three sequential repair stations cumulative failures per hr Little’s formula gives the mean (repair+waiting) time at each station, Copyright © 2003 by K.S. Trivedi
y General Feed Forward Networks
Burke’s result together with the following
two properties of Poisson process can be
utilized to derive product form solution for
any feed forward queuing network:
Probabilistically splitting a Poisson stream
gives rise to two or more Poisson streams (see
Figure 6.13 on p. 309)
Joining two or more Poisson streams produces
a single Poisson stream (see Figure 6.12 on p.
308) Copyright © 2003 by K.S. Trivedi
y Open Queuing Networks with Feedback
Open queuing network is one in which jobs may arrive
from the outside world and on completion, jobs may leave
the network.
Jackson’s result: Product form solution is applicable to
open queuing networks with any arbitrary feedforward/feedback connections. (assuming each node is an
M/M/1 or M/M/m queue). Such networks are called open
PFQNs.
Assumptions:
Poisson arrival process(es)
Exponentially distributed service times
Each node follows FCFS queuing discipline
Infinite storage space at each node. Copyright © 2003 by K.S. Trivedi
y M/M/1 queue with Bernoulli feedback Burke’s second result: the queue above does not
have Poisson input process (I) (see Prob. 9.23),
even though processes at points A and D are both
Poisson.
Hence, the queues within a network with feedback
will not be M/M/m queues in general (as their
arrival process may not be Poisson), but they
behave as if they are independent M/M/m queues.
This is the beauty of Jackson’s remarkable result
of product form of networks with feedback.
Copyright © 2003 by K.S. Trivedi
y Open PFQN with Feedback
Open queuing network with 2 nodes
CTMC state diagram with state = (k0,k1) Copyright © 2003 by K.S. Trivedi
y Example (contd.)
The following product form solution can be shown
to satisfy the balance equations of the underlying
CTMC
Where λi is the average arrival rate at the ith node
In the steady state, the departure rate from the ith
node is also λi.
Equations relating λi’s are called Traffic equations
which are dealt with next. Copyright © 2003 by K.S. Trivedi
y Traffic Equations for the Example
In this example with two nodes, and λ0 and λ1 are the
arrival rates at the CPU and I/O node, respectively.
Arrivals to CPU are either from outside at a rate λ or from
the I/O node at rate λ1, therefore,
λ0 = λ + λ1
Also a job after completion of the CPU burst would go to
the I/O node with probability p1, therefore,
λ1= λ0p1 = λ0(1p0).
Solving these two traffic equations, we get, Copyright © 2003 by K.S. Trivedi
y Unfolding the Open PFQN with Feedback
Hence the product form solution for the 2node network with feedback:
Above solution suggests an equivalence
with the following network without
feedback: Copyright © 2003 by K.S. Trivedi
y Meaning of Equivalence
The previous equivalence established between an
open network with feedback and an open network
without feedback is restricted to only:
Steady state behavior and
Mean response times, queue length distribution This equivalence does not apply to other analysis,
e.g.,
Response time distributions
Transient behavior, etc. Copyright © 2003 by K.S. Trivedi
y Open PFQN General CSM
Consider the central server model example This is open queuing network of m+1 nodes
Single CPU node and m I/O nodes Copyright © 2003 by K.S. Trivedi
y General Open PFQN Solution
Generating and solving the underlying CTMC for
such a queuing network is neither feasible nor
necessary due to Jackson’s result
Consider a single tagged program executing on the
system (without any queuing or interference from
other programs), moving from one node to another
Observe the system only at times when a job
completes service at a node.
The underlying stochastic process forms a DTMC
which is characterized by its transition probability
matrix.
Copyright © 2003 by K.S. Trivedi
y DTMC model for the Open PFQN
The DTMC transition probability matrix: Routing matrix X Copyright © 2003 by K.S. Trivedi
y DTMC Solution
Using the technique from example 7.20:
Vj : Av. no. of visits made by a job to node j
before leaving the system λ jobs/unit time enter the system (from outside),
arrival rate λj of jobs at node j is, Copyright © 2003 by K.S. Trivedi
y DTMC Solution(contd.)
Jackson has shown that the steady state joint
probability is given by, Where the probability of finding kj jobs at node j is
given by the M/M/1 formula:
So the solution for the this open queuing network
has product form
Copyright © 2003 by K.S. Trivedi
y General Open PFQN Measures
Note: Despite the product form solution, the
arrival process at a node may not be Poisson
and still the Jackson’s result holds!
Average Queue length at node j:
Average response time at node j: Copyright © 2003 by K.S. Trivedi
y Open PFQN Measures (contd.)
Av. no. of jobs in the system
Av. system response time (Little’s formula) An equivalent unfolded tandem network can thus
be derived
Copyright © 2003 by K.S. Trivedi
y Open PFQN Equivalence
The total service requirement at each node j is given by :
E[ B j ] = 1
,
µ 0 p0
pj µ j p0 , j =0
j = 1,2,..., m The service rate at node j in the equivalent unfolded
network is given by 1 / E[ B j ] .
Hence the unfolded open network is as below:
µ 0 p0 µ1 p0
p1 µ m p0
pm Note that only the total service requirement at each node is
needed to find the rates in the equivalent unfolded network
Copyright © 2003 by K.S. Trivedi
y Closed PFQNs
Closed queuing network is one in which no job enters
into and leaves from the queuing network from/to the
external environment.
If a job actually departs the system being modeled,
conceptually it is immediately replaced by a new job
(from the outside), so that total number of jobs in the
QN model is always fixed.
The service times are assumed to be exponentially
distributed.
Gordon & Newell studied such networks and showed
that a product form solution holds. Copyright © 2003 by K.S. Trivedi
y A NonPFQN Example
In real practice when a job needs service, it has to be
admitted in the active set and allocated passive resources
such as main memory, process table entry etc. Two approximations of the above nonPFQN
Light load (low arrival rate), by removing the job scheduler queue,
we get open PFQN as a light load approximation.
Heavy load (high arrival rate)
There are always jobs (in the
scheduler queue) waiting to be scheduled when an existing job
leaves the QN closed PFQN as a heavy load approximation.
Copyright © 2003 by K.S. Trivedi
y Closed PFQN: Central Server Model
Closed PFQN with fixed no. of jobs. As an example consider a CSM with a single I/O node
PFQN state ={k0, k1} and k0+ k1=constant, n
µ0
CPU p1 µ1
IO p0
New program path Copyright © 2003 by K.S. Trivedi
y Closed PFQN Balance Equations
The steady state balance equations are: CPU utilization U0 : Copyright © 2003 by K.S. Trivedi
y Closed PFQN – CTMC based solution
For the general case of m I/O nodes in the
previous example, state = (k0, k1, …,km)
CTMC has far too many states, even for small
values n and m.
First consider the case of a single job (no queuing)
Resulting behavior was modeled as a DTMC in
Chapter 7. Copyright © 2003 by K.S. Trivedi
y Closed PFQN – First Step
Solution proposed by GordonNewell:
Consider a tagged job visiting various nodes
Ignore all queues, i.e., consider only nodes
IO1
1
p0
CPU p1 IO2 1 pm
1 IOm DTMC X: DTMC transition probability matrix (Routing matrix) Copyright © 2003 by K.S. Trivedi
y Closed PFQN: DTMC Solution
Steady state DTMC solution is given by solving
the eqs.:
Solve,
The routing matrix X has rank=m,
In the absence of the total probability constraint,
only m of the m+1 equations are independent
choose one vi arbitrarily, say, v0 = 1/p0, µ0 or 1 Copyright © 2003 by K.S. Trivedi
y Closed PFQN: DTMC Solution (contd.) Choice#1:
Define relative utilization of node i as,
Define, Bi = total service required at node i.
Therefore,
Other choices for v0 , viz. v0=µ0 or 1 are possible
As shown in open networks the total service
requirements at each node is all that matters. Copyright © 2003 by K.S. Trivedi
y Closed PFQN: GordonNewell result
GordonNewell result: steady state joint pmf is given by, Using the normalization condition: Copyright © 2003 by K.S. Trivedi
y Problem of large number of summands
Direct use of eq. (1) to compute C(n) is a problem: the
number of summands in (1) may be very large:
Number of summands grows exponentially with n and m
Better algorithm needed for computing C(n).
Computing C(n) using the GF approach.
Define, The coefficient of zn in G(z) equals C(n). Therefore, Copyright © 2003 by K.S. Trivedi
y Recursive solution for C(n)
Define, Copyright © 2003 by K.S. Trivedi
y Closed PFQN: Performance Measures
Some of the useful performance measures are:
Real utilization Ui of the ith I/O device (vs. relative utilization)
Average job population at node i.
Average throughput
Average response time Copyright © 2003 by K.S. Trivedi
y Closed PFQN: Real Utilization
Real utilization Ui
Define, Prob ( > 0 jobs at node i) zn coeff. of H(z): sum of the terms of the form,
Since, Copyright © 2003 by K.S. Trivedi
y Close PFQN: Real Utilization (contd.)
Real utilization Copyright © 2003 by K.S. Trivedi
y Closed PFQN: Average Queue Length
Following the previous approach,
Average queue length at node i then is, Copyright © 2003 by K.S. Trivedi
y Closed PFQN: Average Throughput
CPU Relative average throughput is v0 = µ0
I/O relative average throughput is vi = µ0 pi
The average overall system throughput is, Example: model parameters as in Table below: Copyright © 2003 by K.S. Trivedi
y Closed PFQN: SHARPE Code
* station types
cpu fcfs mu0
io1 fcfs mu1
io2 fcfs mu2
end
* closed queueing network has a total of ‘jobs’ customers.
cust jobs
end
loop i, 1,20,1
expr(i)
expr qlength(csm, cpu; i)
expr util(csm, cpu; i) *mu0*p0
expr qlength(csm, io1; i)
expr qlength(csm, io2; i)
end
end
end Copyright © 2003 by K.S. Trivedi
y bind
p0 0.1
p1 0.667
p2 0.233
p10 1
p20 1
mu0 1000/20
mu1 1000/30
mu2 1000/42.918
end
pfqn csm(jobs)
* routing probabilities
cpu io1 p1
cpu io2 p2
io1 cpu p10
io2 cpu p20
end Showing Thrashing
We take the same queuing network in the
previous example but assume a paging
system
Mean total CPU requirement per job is
fixed and so are the average disk service
times per request
The average number of requests to paging
disk is a function of the degree of
multiprogramming n
Copyright © 2003 by K.S. Trivedi
y Paging Example System throughput
drops sharply, as n
(multiprogramming)
increases thrashing Copyright © 2003 by K.S. Trivedi
y Terminal Oriented Distributed System
DBMS proc.
Front end proc. Comm. proc.
User terminals Main proc Copyright © 2003 by K.S. Trivedi
y Terminal Oriented Distributed System (contd.)
Solving, v = vX gives, Choosing vT =1 gives,
Since µT = λ, relative utilizations are given by, Copyright © 2003 by K.S. Trivedi
y Performance measures
State probability is given by:
F node has only one processor. Its utilization is:
Avg. throughput is:
Avg. response time is:
The system can also be solved by SHARPE
Copyright © 2003 by K.S. Trivedi
y SHARPE CODE Example 9.10 Copyright © 2003 by K.S. Trivedi
y Closed PFQN: web browsing model M Clients generate HTTP requests
Requests are routed to the web server
Responses are routed back to the clients Copyright © 2003 by K.S. Trivedi
y Web browsing closed PFQN solution
Assume the following service requirements estimated
from prior Internet measurements:
Let vc = Avg. number of visits VC to node C
Therefore, relative utilization
Joint (state) probability is: Copyright © 2003 by K.S. Trivedi
y Web browsing closed PFQN Measures
LAN utilization:
Avg. throughput
Avg. Response time
Avg. response time vs. # of clients Copyright © 2003 by K.S. Trivedi
y Mean Value Analysis (MVA)
Motivation
C(M) calculation can have numerical problems MVA is an efficient method for calculating
performance measures for closed PFQNs
MVA uses two simple known laws:
Little’s formula applied to the overall system
Theorem of the distribution at the arrival time
pmf of no. of jobs seen at the time of arrival at node i when there
are already n jobs in the network = steady state pmf of the no. of
jobs at this node with one less job in the network. Copyright © 2003 by K.S. Trivedi
y MVA for mean response time
Ni(j) jobs (out of total j) at node i. Assuming FCFS
scheduling,
Since, Using Little’s formula, mean time between arrivals is, Copyright © 2003 by K.S. Trivedi
y MVA for multiserver nodes
πi(k1j1) = P(Ni = k1j1 jobs in the QN) Then, Using induction, the mean response time is, Copyright © 2003 by K.S. Trivedi
y Other measures using MVA
Throughput of each node:
Other measures, e.g., utilization, mean
waiting time, mean queue length etc. can
also be computed in a similar manner. Copyright © 2003 by K.S. Trivedi
y MVA: Numerical Example
A closed queuing network with N=2 jobs
2 CPU queues
(1 /µ2 = 0.6 sec and 1 /µ3=0.8 sec.) Front end processor
(service rate µ1 = 2 tasks/sec) Copyright © 2003 by K.S. Trivedi
y User terminals
(avg. think time=1/µ0) MVA: Numerical Example Solution
Solving the system of equations for v,
v = vX
Performance measures using MVA:
Step 1: Initialize – for i=0,1,2,3,
Step 2: Iterate over the no. of jobs, n=1 and 2. For n=1, Copyright © 2003 by K.S. Trivedi
y MVA: Solution (contd.)
Substituting for E[T(1)] and E[Ri(1)] gives E[Ni(1)] : Similarly, compute E[T(2)], E[Ri(2)] and E[Ni(2)]: Copyright © 2003 by K.S. Trivedi
y SHARPE CODE Example 9.12
* station types
T is mu0
F ms c1, mu1
C fcfs mu2
D fcfs mu3
end
* closed queueing network has a total of ‘jobs’ customers.
cust jobs
end
loop i, 1,10,1
expr(i)
expr tput(cqn, F; i)
expr qlength(cqn, F; i)
expr rtime(cqn, F; i)
expr qlength(cqn, C; i)
expr rtime(cqn, C; i)
expr qlength(cqn, D; i)
expr rtime(cqn, D; i)
end
end
end
Copyright © 2003 by K.S. Trivedi
y bind
pt 0 1
p0 0.1
p1 0.4
p2 0.5
p10 1
p20 1
c1 2
mu0 1
mu1 2
mu2 1/0.6
mu3 1/0.8
end
pfqn cqn(jobs)
* routing probabilities
T F pt0
F T p0
F C p1
F D p2
C F p10
D F p20
end General Service Time Distribution & MultiClass
Jobs
QNs constrained to have a number of restriction so
as to get PF solution.
Many QNs do away with some of these restrictions
and yet yield a PF solution.
Akyilidiz (’87), Bolch et. al (’98), Chandy (’77) etc. Following QNs (open/closed, single/multi class
jobs, general service times) have PF solution. Copyright © 2003 by K.S. Trivedi
y General Service Time Distribution
Example: General service time – 2phase hyperEXP CPU p1 µ2
IO p0 System state k : # of jobs with the CPU node
State space: {0,1, . . , n}
Future state depends on (a) current state (b) τ , time spent by the CPU
on the current job
nonMarkov behavior.
State: (k, τ) state space becomes jointly discrete and continuous
Simplifications: (a) n=2 (b) Instead of τ, consider CPU phases (0,1)
CPU scheduling is processor sharing (PS) between 2 jobs with CPU
State space: {i,j}: ijobs in phase1, jjobs in phase2
{(0,0), (1,0), (0,1), (2,0), (0,2), (1,1)}
Copyright © 2003 by K.S. Trivedi
y General Service Time Distribution
State diagram: (contd.) balance equations: Solution: Copyright © 2003 by K.S. Trivedi
y NPFQNs– FES Method (Hierarchical)
In the example with job scheduler queue, product form solution does
not exist.
However, resulting QN may be represented by a CTMC.
CTMC Steady state solution is generally very difficult.
Either directly or with the help of its SPN.
In the example, merge the flowequivalent server to represent the
subnetwork
CPU p1 µ2
IO p0 Av. throughput depends of the multiprogramming degree
Hence, equivalent server has loaddependent service rate γi ,
Solve a CTMC given respective arrival & service rates of λ & γi
Copyright © 2003 by K.S. Trivedi
y NPFQN FES solution
Steady state solution of outer CTMC is given by, Copyright © 2003 by K.S. Trivedi
y NPFQN solution using SRN
Example: Terminal oriented system QN
Max. # jobs in CPU+IO <= n For the SRN model, define the
reward rate as,
then,
Parameters of the above figure SRN and CTMC characteristics:
1526
8800 2436
14160 3346
19520 4256
24880 5166
30240 Copyright © 2003 by K.S. Trivedi
y 615
4886 NPFQN : Approximate Solution
We could use an approximate method also :
Replace the terminal subsystem with a short circuit
Find the avg. throughput as a function of number of jobs in the network,
Throughput vector : ( E[T(1)], E[T(2)],…, E[T(n)])
Replace the central server subnetwork by an equivalent server. This could be represented by a birthdeath process
Birth rates :
Death rates : = mean think time of the users Copyright © 2003 by K.S. Trivedi
y NPFQN : Approximate Solution
Prob. that the eq. server is idle : Also, The expected throughput, E[T], of the equivalent server : Average response time : Copyright © 2003 by K.S. Trivedi
y NPFQN : Approximate Solution
For = 15s, n = 3 and m = 4 for different values of M : 1.023 1.22 1.54 2.23 3.84 6.78 For varying n we can
get a 3D plot for average
response time as a function
of M & n as : Copyright © 2003 by K.S. Trivedi
y Computing Response Time Distributions
Closed form solutions have been derived for LST of response
time distributions through a particular path in open PFQNs.
Numerical inversion of the LST is however difficult.
Results exist for specific topologies like tandem, central
server, etc.
Usually difficult to derive for nonrestrictive topologies and
service and arrival characteristics.
Methods for approximating the distribution can give good
results.
One method is to derive a HCTMC for the original network
(using the response time distribution of its components) and
find the distribution for the time to reach the absorbing state.
Copyright © 2003 by K.S. Trivedi
y Response Time Distributions in Open Networks
Recall that response time for an M/M/1 FCFS queue is exp
distributed with parameter (µλ) [see Chapter 8].
For the open network with feedback (Fig. 9.4(a)) we
construct the CTMC as below. State 0 is the starting state, and state C represents the job
leaving the network.
Time to reach state C is the response time (approximately)
By solving the time to absorption distribution in this
CTMC we get the response time distribution for the
network
Copyright © 2003 by K.S. Trivedi
y Response Time Distributions in Open
Networks (contd.)
For the tandem network equivalent (Fig. 9.4(b)), we have
the following CTMC. The response time distributions F(t) for the two models
are different, though differing only slightly, and with same
mean response time. Copyright © 2003 by K.S. Trivedi
y Response Time Distributions in Open
Networks : Response Time Blocks
Response Time Blocks : Could be used to approximate the
response time distributions F(t) of queueing model nodes.
Assume that nodes are FCFS with arrival rate λ and service rate of
each server of the node as µ.
If c servers are present in a node then assume λ < cµ. M/M/1 : M/M/∞ : Copyright © 2003 by K.S. Trivedi
y Response Time Distributions in Open
Networks : Response Time Blocks (contd.)
M/M/c : Where and Copyright © 2003 by K.S. Trivedi
y Response Time Distributions in Closed Networks
Usually very difficult to obtain for networks with general
structure.
Tagged Customer Approach:
Pick an arbitrary customer and call it the tagged customer.
Determine all states in which the tagged customer may find the
QN upon arrival.
Trace its passage through the network.
Construct a finite state CTMC for the same.
Find the time to absorption distribution of the finite state CTMC. We need to :
Compute the steady state probability vector of the CTMC for the
QN with one less customer, π(n1)
Use this vector to compute the unconditional response time
distribution, P[R<=t].
Copyright © 2003 by K.S. Trivedi
y Response Time Distributions in Closed
Networks : The Central Server Model Example
Consider a central server model (CSM) of a computing
system as below: Let the service rates of the CPU, Disk1 and Disk2 be µC ,
µD1, and µD2 , respectively.
We define the response time as the amount of time elapsed
from when the customer enters the CPU for its first service
until the instant it emerges on the new program path.
We can formulate the response time distribution in terms
of the absorption time distribution of a CTMC.
Copyright © 2003 by K.S. Trivedi
y Response Time Distributions in Closed Networks
The Central Server Model Example (contd.)
For simplicity assume that we have only 2 customers.
The CTMC is as : The initial states are (20012), (11011), (10111).
The absorbing states are (10000), (01000) and (00100).
Copyright © 2003 by K.S. Trivedi
y Response Time Distributions in Closed Networks
The Central Server Model Example (contd.)
We can get the conditional response time distribution from
the CTMC on the previous slide.
The CTMC below could be solved for computing the
steady state probabilities of the nontagged customer to be
used to get the unconditional response time distribution . Copyright © 2003 by K.S. Trivedi
y Response Time Distributions in Closed Networks
The Central Server Model Example (contd.)
In general if
I : the set of all states in the CTMC whose solution yields the
response time distribution.
A : set of all absorbing states in the CTMC.
Ri : a random variable representing the response time of an
arbitrary customer arriving when the QN is in state i . We have the conditional response time distribution as: Here pij(t) is the transient probability of state j at time t
given that i is the initial state of the CTMC. Copyright © 2003 by K.S. Trivedi
y Response Time Distributions in Closed Networks
The Central Server Model Example (contd.)
Let
πi(n1) : the probability that the tagged customer will
see the network in state i when it arrives.
S: set of all possible states of the network
with one less customer.
R: Random variable representing
unconditional response time distribution.
πj(t) represents the unconditional transient
probability of state j. We have : Copyright © 2003 by K.S. Trivedi
y ...
View
Full
Document
This note was uploaded on 04/08/2010 for the course COMPUTER E 409232 taught by Professor Mohammadabdolahiazgomiph.d during the Spring '10 term at Islamic University.
 Spring '10
 MohammadAbdolahiAzgomiPh.D

Click to edit the document details