Discrete-time stochastic processes

# 40 e x2 e x t x 342 these ensemble averages agree

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ). Example 3.4.2. (time-average Age) Let Z (t) be the age of a renewal process at time t where age is deﬁned as the interval from the most recent arrival before (or at) t until t, 108 CHAPTER 3. RENEWAL PROCESSES 1/ε ❅ ❅ ❅ ❅ ❅Y (t) ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅ ❅✲ 1/ε ❅❅❅❅❅❅❅❅❅✛ ❅❅❅❅❅❅ Figure 3.7: Average Residual life is dominated by large interarrival intervals. Note the large intervals of time (within the large triangles) over which Y (t) is large, and small aggregate intervals over which it is small. i.e., Z (t) = t − SN (t) (see Figure 3.8). We notice that the age process, for a given sample function of the renewal process, is almost the same as the residual life process—the isosceles right triangles are simply turned around. Thus the same analysis as before can be used to show that the time average of Z (t) is the same as the time-average of the residual life, £§ Rt Z (τ ) dτ E X2 lim τ =0 = W.P.1. (3.25) t→1 t 2E [X ] ° ° Z (t)° ° ° ° ° ° ° ° ° ° °° ° ° ° t °° °° ° S1 S2 S3 S4 ° ° ° S5 S6 Figure 3.8: Age at time t: For any given sample function of the renewal process, the sample function of age increases linearly with a slope of 1 from the beginning to the end of each inter-renewal interval. Example 3.4.3. (time-average Duration) Let X (t) be the duration of the inter-renewal interval containing time t, i.e., X (t) = XN (t)+1 = SN (t)+1 − SN (t) (see Figure 3.9). It is clear that X (t) = Z (t) + Y (t), and thus the time-average of the duration is given by £§ Rt E X2 τ =0 X (τ ) dτ lim = W.P.1. (3.26) t→1 t E [X ] Again, long intervals are heavily weighted in this average, so that the time-average duration is at least as large as the mean inter-renewal interval and often much larger. 3.4.1 General renewal-reward processes In each of these examples, and in many other situations, we have a random function of time (i.e., Y (t), Z (t), or X (t)) whose value at time t depends only on where t is in the 3.4. RENEWAL-REWARD PROCESSES; TIME-AVERAGES 109 X (t) ✛ X5 ✲ t S1 S2 S3 S4 S5 S6 Figure 3.9: Duration X (t) = XN (t) of the inter-renewal interval containing t. current inter-renewal interval (i.e., on the age Z (t) and on the duration X (t) of the current inter-renewal interval). We now investigate the general class of reward functions for which the reward function at time t depends only on the age and the duration at t, i.e., the reward R(t) at time t is given explicitly as a function R(Z (t), X (t)) of the age and duration at t. For the three examples above, the function R is trivial. That is, the residual life, Y (t), is given by X (t) − Z (t) and the age and duration are given directly. Rt We now ﬁnd the time-average value of R(t), namely, limt→1 1 0 R(τ ) dτ . As in examples t 3.4.1 to 3.4.3 above, we ﬁrst want to look at the accumulated reward over each inter-renewal period separately. Deﬁne Rn as the accumulated reward in the nth renewal interval, Z Sn Z Sn Rn = R(τ ) d(τ ) = R[Z (τ ), X (τ )] dτ . (3.27) Sn−1 Sn−1 For residual life (see Example 3.4.1), Rn is the area of the nth isosceles right triangle in Figure 3.6. In general, since Z (τ ) = τ − Sn−1 , Z Sn Z Xn Rn = R(τ −Sn−1 , Xn ) dτ = R(z , Xn ) dz . (3.28) Sn−1 z =0 Note that Rn depends only on the value of Xn and the form of the function R(Z, X ). From this, it is clear that {Rn ; n ≥ 1} is essentially7 a set of IID random variables. For residual 2 life, R(z , Xn ) = Xn − z , so the integral in (3.28) is Xn /2, as calculated by inspection before. In general, from (3.28), the expected value of Rn is given by Z1 Zx E [Rn ] = R(z , x) dz dFX (x). (3.29) x=0 z =0 Rt Breaking 0 R(τ ) dτ into the reward over the successive renewal periods, we get Zt Z S1 Z S2 Z SN (t) Zt R(τ ) dτ = R(τ ) dτ + R(τ ) dτ + · · · + R(τ ) dτ + R(τ )dτ 0 0 S1 N (t) = X n=1 7 Rn + Z SN (t)−1 SN (t) t R(τ ) dτ . (3.30) SN (t) One can certainly deﬁne functions R(Z, X ) for which the integral in (3.28) is inﬁnite or undeﬁned for some values of Xn , and thus Rn becomes a defective rv. It seems better to handle this type of situation when it arises rather than handling it in general. 110 CHAPTER 3. RENEWAL PROCESSES The following theorem now generalizes the results of Examples 3.4.1, 3.4.2, and 3.4.3 to general renewal-reward functions. Theorem 3.6. Let {R(t); t > 0} ≥ 0 be a non-negative renewal-reward function for a renewal process with expected inter-renewal time E [X ] = X . If X < 1 or E [Rn ] < 1, then with probability 1 Z 1t E [Rn ] lim R(τ ) dτ = . (3.31) t→1 t τ =0 X Proof: Using (3.30), the accumulated reward up to time t can be bounded between the accumulated reward up to the renewal before t and that to the next renewal after t, Rt PN (t) PN (t)+1 Rn n=1 Rn τ =0 R(τ ) dτ ≤ ≤ n=1 . (3.32) t t t The left hand side of (3.32) can now be broken into PN (t) n=1 Rn t = PN (t) Rn N (t) . N (t) t n=1 (3.3...
View Full Document

## This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.

Ask a homework question - tutors are online