chap9 - Probability and Statistics with Reliability,...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Probability and Statistics with Reliability, Queuing and Computer Science Applications: second edition, by K.S. Trivedi Publisher-John Wiley & Sons Chapter 9: Networks of Queues Dept. of Electrical & Computer engineering Duke University Email: kst@ee.duke.edu URL: www.ee.duke.edu/~kst Copyright © 2003 by K.S. Trivedi 1 Classes of Queuing Networks Tandem Queues Open queuing networks (Jackson’s network) Closed queuing networks (Gordon-Newell network) Multi class queuing networks (BCMP queues) Non-product-form queuing networks. In the last segment we will discuss the computation of response time distribution (or percentiles) in networks Copyright © 2003 by K.S. Trivedi y Networks of Queues Two types of networks : Open and Closed An open queuing network is characterized by one or more sources of job arrivals and correspondingly one or more sinks that absorb jobs departing from the network. In a closed queuing network, jobs neither enter nor depart from the network. The behavior of jobs within the network is characterized by the distribution of job service times at each center the probabilities of transitions between service centers For each center the number of servers, the scheduling discipline, and the size of the queue must be specified. For an open network, a characterization of job-arrival processes is needed. For a closed network, the number of jobs in the network must be specified. Copyright © 2003 by K.S. Trivedi y Open Queuing Networks M/M/1, etc. : single node queuing network Two M/M/1 queues in tandem Poisson stream Exponentially dist. service times at s0 and s1. Underlying stochastic process is an HCTMC State: (k0,k1), ki: number of jobs at node i, i=0,1 The changes of state occur upon a completion of service at one of the two servers or upon an external arrival. Since all inter-event times are exponentially distributed (by our assumptions), the underlying stochastic process is a homogeneous CTMC with the state diagram shown on the next page. Copyright © 2003 by K.S. Trivedi y Tandem queue-state diagram CTMC state diagram: Copyright © 2003 by K.S. Trivedi y Tandem queue CTMC solution Following steady state solution can be shown to satisfy the CTMC balance equations: Solution has a product form, i.e., product of the solution of two independent M/M/1 queues. For an M/M/1 queue: Burke showed that the output process is also Poisson with rate λ Therefore, for the two queue tandem network, the second queue is also an independent M/M/1 queue. Hence the product form of the result holds. Copyright © 2003 by K.S. Trivedi y Tandem queue product form Generalization to an n-node tandem network The solution can be shown to satisfy the balance equations of the underlying CTMC The solution can also be derived by repeatedly invoking Burke’s result. Copyright © 2003 by K.S. Trivedi y Tandem Queues-example Repair facility: three sequential repair stations cumulative failures per hr Little’s formula gives the mean (repair+waiting) time at each station, Copyright © 2003 by K.S. Trivedi y General Feed Forward Networks Burke’s result together with the following two properties of Poisson process can be utilized to derive product form solution for any feed forward queuing network: Probabilistically splitting a Poisson stream gives rise to two or more Poisson streams (see Figure 6.13 on p. 309) Joining two or more Poisson streams produces a single Poisson stream (see Figure 6.12 on p. 308) Copyright © 2003 by K.S. Trivedi y Open Queuing Networks with Feedback Open queuing network is one in which jobs may arrive from the outside world and on completion, jobs may leave the network. Jackson’s result: Product form solution is applicable to open queuing networks with any arbitrary feedforward/feedback connections. (assuming each node is an M/M/1 or M/M/m queue). Such networks are called open PFQNs. Assumptions: Poisson arrival process(es) Exponentially distributed service times Each node follows FCFS queuing discipline Infinite storage space at each node. Copyright © 2003 by K.S. Trivedi y M/M/1 queue with Bernoulli feedback Burke’s second result: the queue above does not have Poisson input process (I) (see Prob. 9.2-3), even though processes at points A and D are both Poisson. Hence, the queues within a network with feedback will not be M/M/m queues in general (as their arrival process may not be Poisson), but they behave as if they are independent M/M/m queues. This is the beauty of Jackson’s remarkable result of product form of networks with feedback. Copyright © 2003 by K.S. Trivedi y Open PFQN with Feedback Open queuing network with 2 nodes CTMC state diagram with state = (k0,k1) Copyright © 2003 by K.S. Trivedi y Example (contd.) The following product form solution can be shown to satisfy the balance equations of the underlying CTMC Where λi is the average arrival rate at the ith node In the steady state, the departure rate from the ith node is also λi. Equations relating λi’s are called Traffic equations which are dealt with next. Copyright © 2003 by K.S. Trivedi y Traffic Equations for the Example In this example with two nodes, and λ0 and λ1 are the arrival rates at the CPU and I/O node, respectively. Arrivals to CPU are either from outside at a rate λ or from the I/O node at rate λ1, therefore, λ0 = λ + λ1 Also a job after completion of the CPU burst would go to the I/O node with probability p1, therefore, λ1= λ0p1 = λ0(1-p0). Solving these two traffic equations, we get, Copyright © 2003 by K.S. Trivedi y Unfolding the Open PFQN with Feedback Hence the product form solution for the 2node network with feedback: Above solution suggests an equivalence with the following network without feedback: Copyright © 2003 by K.S. Trivedi y Meaning of Equivalence The previous equivalence established between an open network with feedback and an open network without feedback is restricted to only: Steady state behavior and Mean response times, queue length distribution This equivalence does not apply to other analysis, e.g., Response time distributions Transient behavior, etc. Copyright © 2003 by K.S. Trivedi y Open PFQN General CSM Consider the central server model example This is open queuing network of m+1 nodes Single CPU node and m I/O nodes Copyright © 2003 by K.S. Trivedi y General Open PFQN Solution Generating and solving the underlying CTMC for such a queuing network is neither feasible nor necessary due to Jackson’s result Consider a single tagged program executing on the system (without any queuing or interference from other programs), moving from one node to another Observe the system only at times when a job completes service at a node. The underlying stochastic process forms a DTMC which is characterized by its transition probability matrix. Copyright © 2003 by K.S. Trivedi y DTMC model for the Open PFQN The DTMC transition probability matrix: Routing matrix X Copyright © 2003 by K.S. Trivedi y DTMC Solution Using the technique from example 7.20: Vj : Av. no. of visits made by a job to node j before leaving the system λ jobs/unit time enter the system (from outside), arrival rate λj of jobs at node j is, Copyright © 2003 by K.S. Trivedi y DTMC Solution(contd.) Jackson has shown that the steady state joint probability is given by, Where the probability of finding kj jobs at node j is given by the M/M/1 formula: So the solution for the this open queuing network has product form Copyright © 2003 by K.S. Trivedi y General Open PFQN Measures Note: Despite the product form solution, the arrival process at a node may not be Poisson and still the Jackson’s result holds! Average Queue length at node j: Average response time at node j: Copyright © 2003 by K.S. Trivedi y Open PFQN Measures (contd.) Av. no. of jobs in the system Av. system response time (Little’s formula) An equivalent unfolded tandem network can thus be derived Copyright © 2003 by K.S. Trivedi y Open PFQN Equivalence The total service requirement at each node j is given by : E[ B j ] = 1 , µ 0 p0 pj µ j p0 , j =0 j = 1,2,..., m The service rate at node j in the equivalent unfolded network is given by 1 / E[ B j ] . Hence the unfolded open network is as below: µ 0 p0 µ1 p0 p1 µ m p0 pm Note that only the total service requirement at each node is needed to find the rates in the equivalent unfolded network Copyright © 2003 by K.S. Trivedi y Closed PFQNs Closed queuing network is one in which no job enters into and leaves from the queuing network from/to the external environment. If a job actually departs the system being modeled, conceptually it is immediately replaced by a new job (from the outside), so that total number of jobs in the QN model is always fixed. The service times are assumed to be exponentially distributed. Gordon & Newell studied such networks and showed that a product form solution holds. Copyright © 2003 by K.S. Trivedi y A Non-PFQN Example In real practice when a job needs service, it has to be admitted in the active set and allocated passive resources such as main memory, process table entry etc. Two approximations of the above non-PFQN Light load (low arrival rate), by removing the job scheduler queue, we get open PFQN as a light load approximation. Heavy load (high arrival rate) There are always jobs (in the scheduler queue) waiting to be scheduled when an existing job leaves the QN closed PFQN as a heavy load approximation. Copyright © 2003 by K.S. Trivedi y Closed PFQN: Central Server Model Closed PFQN with fixed no. of jobs. As an example consider a CSM with a single I/O node PFQN state ={k0, k1} and k0+ k1=constant, n µ0 CPU p1 µ1 IO p0 New program path Copyright © 2003 by K.S. Trivedi y Closed PFQN- Balance Equations The steady state balance equations are: CPU utilization U0 : Copyright © 2003 by K.S. Trivedi y Closed PFQN – CTMC based solution For the general case of m I/O nodes in the previous example, state = (k0, k1, …,km) CTMC has far too many states, even for small values n and m. First consider the case of a single job (no queuing) Resulting behavior was modeled as a DTMC in Chapter 7. Copyright © 2003 by K.S. Trivedi y Closed PFQN – First Step Solution proposed by Gordon-Newell: Consider a tagged job visiting various nodes Ignore all queues, i.e., consider only nodes IO1 1 p0 CPU p1 IO2 1 pm 1 IOm DTMC X: DTMC transition probability matrix (Routing matrix) Copyright © 2003 by K.S. Trivedi y Closed PFQN: DTMC Solution Steady state DTMC solution is given by solving the eqs.: Solve, The routing matrix X has rank=m, In the absence of the total probability constraint, only m of the m+1 equations are independent choose one vi arbitrarily, say, v0 = 1/p0, µ0 or 1 Copyright © 2003 by K.S. Trivedi y Closed PFQN: DTMC Solution (contd.) Choice#1: Define relative utilization of node i as, Define, Bi = total service required at node i. Therefore, Other choices for v0 , viz. v0=µ0 or 1 are possible As shown in open networks the total service requirements at each node is all that matters. Copyright © 2003 by K.S. Trivedi y Closed PFQN: Gordon-Newell result Gordon-Newell result: steady state joint pmf is given by, Using the normalization condition: Copyright © 2003 by K.S. Trivedi y Problem of large number of summands Direct use of eq. (1) to compute C(n) is a problem: the number of summands in (1) may be very large: Number of summands grows exponentially with n and m Better algorithm needed for computing C(n). Computing C(n) using the GF approach. Define, The coefficient of zn in G(z) equals C(n). Therefore, Copyright © 2003 by K.S. Trivedi y Recursive solution for C(n) Define, Copyright © 2003 by K.S. Trivedi y Closed PFQN: Performance Measures Some of the useful performance measures are: Real utilization Ui of the ith I/O device (vs. relative utilization) Average job population at node i. Average throughput Average response time Copyright © 2003 by K.S. Trivedi y Closed PFQN: Real Utilization Real utilization Ui Define, Prob ( > 0 jobs at node i) zn co-eff. of H(z): sum of the terms of the form, Since, Copyright © 2003 by K.S. Trivedi y Close PFQN: Real Utilization (contd.) Real utilization Copyright © 2003 by K.S. Trivedi y Closed PFQN: Average Queue Length Following the previous approach, Average queue length at node i then is, Copyright © 2003 by K.S. Trivedi y Closed PFQN: Average Throughput CPU Relative average throughput is v0 = µ0 I/O relative average throughput is vi = µ0 pi The average overall system throughput is, Example: model parameters as in Table below: Copyright © 2003 by K.S. Trivedi y Closed PFQN: SHARPE Code * station types cpu fcfs mu0 io1 fcfs mu1 io2 fcfs mu2 end * closed queueing network has a total of ‘jobs’ customers. cust jobs end loop i, 1,20,1 expr(i) expr qlength(csm, cpu; i) expr util(csm, cpu; i) *mu0*p0 expr qlength(csm, io1; i) expr qlength(csm, io2; i) end end end Copyright © 2003 by K.S. Trivedi y bind p0 0.1 p1 0.667 p2 0.233 p10 1 p20 1 mu0 1000/20 mu1 1000/30 mu2 1000/42.918 end pfqn csm(jobs) * routing probabilities cpu io1 p1 cpu io2 p2 io1 cpu p10 io2 cpu p20 end Showing Thrashing We take the same queuing network in the previous example but assume a paging system Mean total CPU requirement per job is fixed and so are the average disk service times per request The average number of requests to paging disk is a function of the degree of multiprogramming n Copyright © 2003 by K.S. Trivedi y Paging Example System throughput drops sharply, as n (multiprogramming) increases thrashing Copyright © 2003 by K.S. Trivedi y Terminal Oriented Distributed System DBMS proc. Front end proc. Comm. proc. User terminals Main proc Copyright © 2003 by K.S. Trivedi y Terminal Oriented Distributed System (contd.) Solving, v = vX gives, Choosing vT =1 gives, Since µT = λ, relative utilizations are given by, Copyright © 2003 by K.S. Trivedi y Performance measures State probability is given by: F node has only one processor. Its utilization is: Avg. throughput is: Avg. response time is: The system can also be solved by SHARPE Copyright © 2003 by K.S. Trivedi y SHARPE CODE- Example 9.10 Copyright © 2003 by K.S. Trivedi y Closed PFQN: web browsing model M Clients generate HTTP requests Requests are routed to the web server Responses are routed back to the clients Copyright © 2003 by K.S. Trivedi y Web browsing closed PFQN solution Assume the following service requirements estimated from prior Internet measurements: Let vc = Avg. number of visits VC to node C Therefore, relative utilization Joint (state) probability is: Copyright © 2003 by K.S. Trivedi y Web browsing closed PFQN Measures LAN utilization: Avg. throughput Avg. Response time Avg. response time vs. # of clients Copyright © 2003 by K.S. Trivedi y Mean Value Analysis (MVA) Motivation C(M) calculation can have numerical problems MVA is an efficient method for calculating performance measures for closed PFQNs MVA uses two simple known laws: Little’s formula applied to the overall system Theorem of the distribution at the arrival time pmf of no. of jobs seen at the time of arrival at node i when there are already n jobs in the network = steady state pmf of the no. of jobs at this node with one less job in the network. Copyright © 2003 by K.S. Trivedi y MVA for mean response time Ni(j) jobs (out of total j) at node i. Assuming FCFS scheduling, Since, Using Little’s formula, mean time between arrivals is, Copyright © 2003 by K.S. Trivedi y MVA for multiserver nodes πi(k-1|j-1) = P(Ni = k-1|j-1 jobs in the QN) Then, Using induction, the mean response time is, Copyright © 2003 by K.S. Trivedi y Other measures using MVA Throughput of each node: Other measures, e.g., utilization, mean waiting time, mean queue length etc. can also be computed in a similar manner. Copyright © 2003 by K.S. Trivedi y MVA: Numerical Example A closed queuing network with N=2 jobs 2 CPU queues (1 /µ2 = 0.6 sec and 1 /µ3=0.8 sec.) Front end processor (service rate µ1 = 2 tasks/sec) Copyright © 2003 by K.S. Trivedi y User terminals (avg. think time=1/µ0) MVA: Numerical Example Solution Solving the system of equations for v, v = vX Performance measures using MVA: Step 1: Initialize – for i=0,1,2,3, Step 2: Iterate over the no. of jobs, n=1 and 2. For n=1, Copyright © 2003 by K.S. Trivedi y MVA: Solution (contd.) Substituting for E[T(1)] and E[Ri(1)] gives E[Ni(1)] : Similarly, compute E[T(2)], E[Ri(2)] and E[Ni(2)]: Copyright © 2003 by K.S. Trivedi y SHARPE CODE Example 9.12 * station types T is mu0 F ms c1, mu1 C fcfs mu2 D fcfs mu3 end * closed queueing network has a total of ‘jobs’ customers. cust jobs end loop i, 1,10,1 expr(i) expr tput(cqn, F; i) expr qlength(cqn, F; i) expr rtime(cqn, F; i) expr qlength(cqn, C; i) expr rtime(cqn, C; i) expr qlength(cqn, D; i) expr rtime(cqn, D; i) end end end Copyright © 2003 by K.S. Trivedi y bind pt 0 1 p0 0.1 p1 0.4 p2 0.5 p10 1 p20 1 c1 2 mu0 1 mu1 2 mu2 1/0.6 mu3 1/0.8 end pfqn cqn(jobs) * routing probabilities T F pt0 F T p0 F C p1 F D p2 C F p10 D F p20 end General Service Time Distribution & Multi-Class Jobs QNs constrained to have a number of restriction so as to get PF solution. Many QNs do away with some of these restrictions and yet yield a PF solution. Akyilidiz (’87), Bolch et. al (’98), Chandy (’77) etc. Following QNs (open/closed, single/multi class jobs, general service times) have PF solution. Copyright © 2003 by K.S. Trivedi y General Service Time Distribution Example: General service time – 2-phase hyperEXP CPU p1 µ2 IO p0 System state k : # of jobs with the CPU node State space: {0,1, . . , n} Future state depends on (a) current state (b) τ , time spent by the CPU on the current job non-Markov behavior. State: (k, τ) state space becomes jointly discrete and continuous Simplifications: (a) n=2 (b) Instead of τ, consider CPU phases (0,1) CPU scheduling is processor sharing (PS) between 2 jobs with CPU State space: {i,j}: i-jobs in phase-1, j-jobs in phase-2 {(0,0), (1,0), (0,1), (2,0), (0,2), (1,1)} Copyright © 2003 by K.S. Trivedi y General Service Time Distribution State diagram: (contd.) balance equations: Solution: Copyright © 2003 by K.S. Trivedi y NPFQNs– FES Method (Hierarchical) In the example with job scheduler queue, product form solution does not exist. However, resulting QN may be represented by a CTMC. CTMC Steady state solution is generally very difficult. Either directly or with the help of its SPN. In the example, merge the flow-equivalent server to represent the subnetwork CPU p1 µ2 IO p0 Av. throughput depends of the multiprogramming degree Hence, equivalent server has load-dependent service rate γi , Solve a CTMC given respective arrival & service rates of λ & γi Copyright © 2003 by K.S. Trivedi y NPFQN FES solution Steady state solution of outer CTMC is given by, Copyright © 2003 by K.S. Trivedi y NPFQN solution using SRN Example: Terminal oriented system QN Max. # jobs in CPU+IO <= n For the SRN model, define the reward rate as, then, Parameters of the above figure SRN and CTMC characteristics: 1526 8800 2436 14160 3346 19520 4256 24880 5166 30240 Copyright © 2003 by K.S. Trivedi y 615 4886 NPFQN : Approximate Solution We could use an approximate method also : Replace the terminal subsystem with a short circuit Find the avg. throughput as a function of number of jobs in the network, Throughput vector : ( E[T(1)], E[T(2)],…, E[T(n)]) Replace the central server subnetwork by an equivalent server. This could be represented by a birth-death process Birth rates : Death rates : = mean think time of the users Copyright © 2003 by K.S. Trivedi y NPFQN : Approximate Solution Prob. that the eq. server is idle : Also, The expected throughput, E[T], of the equivalent server : Average response time : Copyright © 2003 by K.S. Trivedi y NPFQN : Approximate Solution For = 15s, n = 3 and m = 4 for different values of M : 1.023 1.22 1.54 2.23 3.84 6.78 For varying n we can get a 3-D plot for average response time as a function of M & n as : Copyright © 2003 by K.S. Trivedi y Computing Response Time Distributions Closed form solutions have been derived for LST of response time distributions through a particular path in open PFQNs. Numerical inversion of the LST is however difficult. Results exist for specific topologies like tandem, central server, etc. Usually difficult to derive for nonrestrictive topologies and service and arrival characteristics. Methods for approximating the distribution can give good results. One method is to derive a HCTMC for the original network (using the response time distribution of its components) and find the distribution for the time to reach the absorbing state. Copyright © 2003 by K.S. Trivedi y Response Time Distributions in Open Networks Recall that response time for an M/M/1 FCFS queue is exp distributed with parameter (µ-λ) [see Chapter 8]. For the open network with feedback (Fig. 9.4(a)) we construct the CTMC as below. State 0 is the starting state, and state C represents the job leaving the network. Time to reach state C is the response time (approximately) By solving the time to absorption distribution in this CTMC we get the response time distribution for the network Copyright © 2003 by K.S. Trivedi y Response Time Distributions in Open Networks (contd.) For the tandem network equivalent (Fig. 9.4(b)), we have the following CTMC. The response time distributions F(t) for the two models are different, though differing only slightly, and with same mean response time. Copyright © 2003 by K.S. Trivedi y Response Time Distributions in Open Networks : Response Time Blocks Response Time Blocks : Could be used to approximate the response time distributions F(t) of queueing model nodes. Assume that nodes are FCFS with arrival rate λ and service rate of each server of the node as µ. If c servers are present in a node then assume λ < cµ. M/M/1 : M/M/∞ : Copyright © 2003 by K.S. Trivedi y Response Time Distributions in Open Networks : Response Time Blocks (contd.) M/M/c : Where and Copyright © 2003 by K.S. Trivedi y Response Time Distributions in Closed Networks Usually very difficult to obtain for networks with general structure. Tagged Customer Approach: Pick an arbitrary customer and call it the tagged customer. Determine all states in which the tagged customer may find the QN upon arrival. Trace its passage through the network. Construct a finite state CTMC for the same. Find the time to absorption distribution of the finite state CTMC. We need to : Compute the steady state probability vector of the CTMC for the QN with one less customer, π(n-1) Use this vector to compute the unconditional response time distribution, P[R<=t]. Copyright © 2003 by K.S. Trivedi y Response Time Distributions in Closed Networks : The Central Server Model Example Consider a central server model (CSM) of a computing system as below: Let the service rates of the CPU, Disk1 and Disk2 be µC , µD1, and µD2 , respectively. We define the response time as the amount of time elapsed from when the customer enters the CPU for its first service until the instant it emerges on the new program path. We can formulate the response time distribution in terms of the absorption time distribution of a CTMC. Copyright © 2003 by K.S. Trivedi y Response Time Distributions in Closed Networks The Central Server Model Example (contd.) For simplicity assume that we have only 2 customers. The CTMC is as : The initial states are (20012), (11011), (10111). The absorbing states are (10000), (01000) and (00100). Copyright © 2003 by K.S. Trivedi y Response Time Distributions in Closed Networks The Central Server Model Example (contd.) We can get the conditional response time distribution from the CTMC on the previous slide. The CTMC below could be solved for computing the steady state probabilities of the non-tagged customer to be used to get the unconditional response time distribution . Copyright © 2003 by K.S. Trivedi y Response Time Distributions in Closed Networks The Central Server Model Example (contd.) In general if I : the set of all states in the CTMC whose solution yields the response time distribution. A : set of all absorbing states in the CTMC. Ri : a random variable representing the response time of an arbitrary customer arriving when the QN is in state i . We have the conditional response time distribution as: Here pij(t) is the transient probability of state j at time t given that i is the initial state of the CTMC. Copyright © 2003 by K.S. Trivedi y Response Time Distributions in Closed Networks The Central Server Model Example (contd.) Let πi(n-1) : the probability that the tagged customer will see the network in state i when it arrives. S: set of all possible states of the network with one less customer. R: Random variable representing unconditional response time distribution. πj(t) represents the unconditional transient probability of state j. We have : Copyright © 2003 by K.S. Trivedi y ...
View Full Document

This note was uploaded on 04/08/2010 for the course COMPUTER E 409232 taught by Professor Mohammadabdolahiazgomiph.d during the Spring '10 term at Islamic University.

Ask a homework question - tutors are online