Discrete-time stochastic processes

# Since this is true for every nite x 0 and x1 is nite

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: y says that if one can be found, the other can also be found. A(τ ) ✛ ✛ W1 0 ✛ ✲ W2 W3 ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣♣♣♣♣♣♣ ✲ ✲ D(τ ) S1 t S2 Figure 3.13: Arrival process, departure process, and waiting times for a queue. Renewals occur at S1 and S2 , i.e., when an arrival sees an empty system. The area Rt between A(τ ) and D(τ ) up to time t is 0 L(τ ) dτ where L(τ ) = A(τ ) − D(τ ). The sum W1 + · · · + WA(t) also includes the shaded area to the right of t. Figure 3.13 illustrates the setting for Little’s theorem. It is assumed that an arrival occurs at time 0, and that the subsequent interarrival intervals are IID. A(t) is the number of arrivals from time 0 to t, including the arrival at 0, so {A(t) − 1; t ≥ 0} is a renewal counting process. The departure process {D(t); t ≥ 0} is the number of departures from 0 to t, and thus increases by one each time a customer leaves the system. The diﬀerence, L(t) = A(t) − D(t), is the number in the system at time t. To be speciﬁc about the waiting time of each customer, we assume that customers enter service in the order of their arrival to the system. This service rule is called First-Come-FirstServe (FCFS).9 Assuming FCFS, the system time of customer n, i.e., the time customer n spends in the system, is the interval from the nth arrival to the nth departure. Finally, 9 For single server queues, this is also frequently referred to as First In First Out (FIFO) service. 118 CHAPTER 3. RENEWAL PROCESSES the ﬁgure shows the renewal points S1 , S2 , . . . at which arriving customers ﬁnd an empty system. As observed in Example 3.1.1, the system probabilistically restarts at each of these renewal instants, and the behavior of the system in one inter-renewal interval is independent of that in each other inter-renewal interval. It is important here to distinguish between two diﬀerent renewal processes. The arrival process, or more precisely, {A(t) − 1; t ≥ 0} is one renewal counting process, and the renewal epochs S1 , S2 , . . . in the ﬁgure generate another renewal process. In what follows, {A(t); t ≥ 0} is referred to as the arrival process and {N (t); t ≥ 0}, with renewal epochs S1 , S2 , . . . is referred to as the renewal process. The entire system can be viewed as starting anew at each renewal epoch, but not at each arrival epoch. We now regard L(t), the number of customers in the system at time t, as a reward function over the renewal process. This is slightly more general than the reward functions of Sections 3.4 and 3.5, since L(t) depends on the arrivals and departures within a busy period (i.e., within an inter-renewal interval). Conditional on the age Z (t) and duration X (t) of the interrenewal interval at time t, one could, in principle, calculate the expected value R(Z (t), X (t)) over the parameters other than Z (t) and X (t). Fortunately, this is not necessary and we can use the sample functions of the combined arrival and departure processes directly, which specify L(t) as A(t) − D(t). Assuming that the expected inter-renewal interval is ﬁnite, Theorem 3.6 asserts that the time average number of customers in the system (with probability 1) is equal to E [Ln ] /E [X ]. E [Ln ] is the expected area between A(t) and D(t) (i.e., the expected integral of L(t)) over an inter-renewal interval. An inter-renewal interval is a busy period followed by an idle period, so E [Ln ] is also the expected area over a busy period. E [X ] is the mean inter-renewal interval. From Figure 3.13, we observe that W1 + W2 + W3 is the area of the region between A(t) and D(t) in the ﬁrst inter-renewal interval for the particular sample path in the ﬁgure. This is the aggregate reward over the ﬁrst inter-renewal interval for the reward function L(t). More generally, for any time t, W1 + W2 + · · · + WA(t) is the area between A(t) and D(t) Rt up to a height of A(t). It is equal to 0 L(τ )dτ plus the remaining waiting time of each of the customers in the system at time t (see Figure 3.13). Since this remaining waiting time is at most the area between A(t) and D(t) from t until the next time when the system is empty, we have N (t) X n=1 Ln ≤ Z A(t) t τ =0 L(τ ) dτ ≤ X i=1 N (t)+1 Wi ≤ X Ln . (3.56) n=1 Assuming that the expected inter-renewal interval, E [X ], is ﬁnite, we can divide both sides of (3.56) by t and go to the limit t → 1. From the same argument as in Theorem 3.6, we get lim t→1 PA(t) i=1 t Wi = lim t→1 Rt τ =0 L(τ ) dτ t = E [Ln ] E [X ] with probability 1. (3.57) Rt We denote limt→1 (1/t) 0 L(τ )dτ as L . The quantity on the left of (3.57) can now be broken up as waiting time per customer multiplied by number of customers per unit time, 3.6. APPLICATIONS OF RENEWAL-REWARD THEORY 119 i.e., lim t→1 PA(t) i=1 t Wi = lim t→1 PA(t) Wi A(t) lim . t→1 A(t) t i=1 (3.58) From (3.57), the limit on the left side of (3.58) exists (and equals L) with probability 1. The second limit on the right also exists with probabi...
View Full Document

## This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.

Ask a homework question - tutors are online