Discrete-time stochastic processes

# Also no stopping node can be an initial segment of

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ess by not using the sup notation to refer to threshold crossings. 7.2. THE WAITING TIME IN A G/G/1 QUEUE: 281 S at k, i.e., that the event 1 {Sn ≥ k} occurs, but the question is considerably harder n=1 than for simple random walks. Since this random walk takes on only integer values, it can be represented as a Markov chain with the set of integers forming the state space. In the Markov chain representation, threshold crossing problems are ﬁrst passage-time problems. These problems can be attacked by the Markov chain tools we already know, but the special structure of the random walk provides new approaches and simpliﬁcations that will be explained in Sections 7.4 and 7.5. 7.1.3 Renewal processes as special cases of random walks If X1 , X2 , . . . are IID positive random variables, then {Sn ; n ≥ 1} is both a special case of a random walk and also the sequence of arrival epochs of a renewal counting process, {N (t); t ≥ 0}. In this special case, the sequence {Sn ; n ≥ 1} must eventually cross a threshold at any given positive value α, and the question of whether the threshold is ever crossed becomes uninteresting. However, the trial on which a threshold is crossed and the overshoot when it is crossed are familiar questions from the study of renewal theory. For the renewal counting process, N (α) is the largest n for which Sn ≤ α and N (α) + 1 is the smallest n for which Sn > α, i.e., the smallest n for which the threshold at α is strictly exceeded. Thus the trial at which α is crossed is a central issue in renewal theory. Also the overshoot, which is SN (α)+1 − α is familiar as the residual life at α. Figure 7.1 illustrates the diﬀerence between general random walks and positive random walks, i.e., renewal processes. Note that the renewal process is illustrated with the axes reversed from usual representation. We usually view each renewal epoch as a time (epoch) and view N (α) as the number of trials up to time α, whereas with random walks, we usually view the number of trials as a discrete time variable and view the sum of rv’s as some kind of amplitude or cost. Mathematically this makes no diﬀerence and it is often valuable to move from one point of view to another. 7.2 The waiting time in a G/G/1 queue: This section and the next introduce two important problems that are best solved by viewing them as random walks. In this section we represent the waiting time in a G/G/1 queue as a threshold crossing problem in a random walk. In the next section, we represent the error probability in a standard type of detection problem as a random walk problem. This detection problem will later be generalized to a sequential detection problem based on threshold crossings in a random walk. Consider a G/G/1 queue with ﬁrst come ﬁrst serve (FCFS) service. We shall ﬁnd how to associate the probability that a customer must wait more than some given time α in the queue with the probability that a certain random walk crosses a threshold at α. Let X1 , X2 , . . . be the inter-arrival times of a G/G/1 queueing system; thus these variables are IID with a given distribution function FX (x) = Pr {Xi ≤ x}. Assume that arrival 0 enters an empty system at time 0, so that Sn = X1 + X2 + · · · + Xn is the epoch of the nth arrival after time 0. Let Y0 , Y1 , . . . , be the service times of the successive customers. These are IID 282 CHAPTER 7. α Epoch S1 q S2 q S3 q RANDOM WALKS, LARGE DEVIATIONS, AND MARTINGALES S4 q Trial (a) S5 q α Epoch S1 q S2 q S3 q S4 q S5 q Trial S1 q Trial S3 q S2 q Epoch (b) Sq5 q S4 α (c) Figure 7.1: The sample function in (a) above illustrates a random walk with arbitrary (positive and negative) step sizes {Xi ; i ≥ 1}. The sample function in (b) illustrates a random walk restricted to positive step sizes {Xi > 0; i ≥ 1}, i.e., a renewal process. Note that the axes are reversed from the usual depiction of a renewal process. The same sample function is shown in part (c) using the customary axes for a renewal process. For both the arbitrary random walk of part (a) and the random walk with positive step sizes of parts (b) and (c), a ‘threshold’ at α is crossed on trial 4 with an overshoot S4 − α. with some given distribution function FY (y ) and are independent of {Xi ; i ≥ 1}. Figure 7.2 shows a sample path of arrivals and departures and illustrates the waiting time in queue for each arrival. To analyze the waiting time, note that the system time, i.e., the time in queue plus the time in service, for any given customer n is Wn + Yn , where Wn is the queueing time and Yn is the service time. As illustrated in Figure 7.2, customer n + 1 arrives Xn+1 time units after the beginning of this interval, i.e., after the arrival of customer n. If Xn+1 < Wn + Yn , then customer n + 1 arrives while customer n is still in the system, and thus must wait in the queue until n ﬁnishes service (in the ﬁgure, for example, customer 2 arrives while customer 1 is still in the queue). Thus Wn+1 = Wn + Yn − Xn+1 if Xn+1 ≤ Wn + Yn . (7.3) On the other hand, if Xn+1 > Wn + Yn , then customer n (and all earlier customers) have departed when n + 1 arrives....
View Full Document

## This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.

Ask a homework question - tutors are online