Discrete-time stochastic processes

14 however it is clear that fcfs order is not

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ♣♣ ♣♣ ♣♣ ♣♣ ♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣ ♣♣ ♣♣ ♣♣ ♣♣ x>z ♣♣ ♣♣ ♣♣ ♣♣ ♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ (3.40) (3.41) ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ ♣♣ Figure 3.12: Joint density of age and duration. Similarly, the mean duration can be found from (3.40). £§ E X2 E [X (t)] = . X (3.42) These ensemble-averages agree with the time-averages found in (3.25) and (3.26). In calculating time-averages, the somewhat paradoxical result that the time-average duration is greater than E [X ] was explained by the large inter-renewal intervals being weighted more heavily in the time-average. Here the same effect occurs, but it can be given a different interpretation: the joint density, at z and x, for age and duration, is proportional to the 114 CHAPTER 3. RENEWAL PROCESSES inter-renewal density fX (x), but the marginal density for duration is weighted by x since the range for age is proportional to x. Using the joint probability density of Z (t) and X (t) to evaluate the expectation of an arbitrary reward function R(t) = R(Z (t), X (t)) in the limit t → 1, we get Z1Zx fX (x) dx E [Rn ] lim E [R(t)] = R(z , x) dz = . (3.43) t→1 X X x=0 z =0 where E [Rn ] is defined in (3.29). Thus the limiting ensemble-average is the same as the time-average. This result should not be surprising. Since we are dealing with non-arithmetic renewals, the probability of a renewal in a small interval becomes independent of where the interval is, so long as the interval is far enough removed from 0 for the process to be in “steady-state”. Since the reward function depends on the process only through the current renewal interval, the reward function must also become independent of time. In the intuitive derivation above, we assumed a density for the inter-renewal variables {Xn ; n ≥ 1}, and we ignored the mathematical issues of taking limits. Here we correct those defects, but the reader should bear in mind that the logical train of the argument is exactly the same as before. Assume throughout that Xn is non-arithmetic with distribution function FX (x). Since m(τ ) = E [N (τ )], the expected number of renewals in (t − z , t − z + δ ] is m(t − z + δ ) − m(t − z ). As shown in Exercise 3.15, the assumption that the interrenewal variables are strictly positive ensures that the probability of more than one renewal in (t − z , t − z + δ ] is insignificant, and more specifically, it also shows that for any t > z > 0, Pr {renewal ∈ (t − z , t − z + δ ]} = m(t − z + δ ) − m(t − z ) − o(δ ), (3.44) where limδ→0 o(δ )/δ = 0 and o(δ ) ≥ 0. Conditional on a renewal at a sample value t − z , the inter-renewal interval X starting at t − z has the distribution FX . Thus P {renewal ∈ (t−z , t−z +δ1 ], X ∈ (x, x+δ2 ]} = = [m(t−z +δ1 ) − m(t−z ) − o(δ1 )][FX (x+δ2 ) − FX (x)]. For x > z , the joint event {renewal ∈ (t − z , t − z + δ1 ], X ∈ (x, x + δ2 ]} is the same as the joint event {Z (t) ∈ [z − δ1 , z ), X (t) ∈ (x, x + δ2 ]}. Thus, Pr {Z (t)∈[z −δ1 , z ), X (t)∈(x, x+δ2 ]} = [m(t−z +δ1 ) − m(t−z ) − o(δ1 )][FX (x+δ2 ) − FX (x)]. (3.45) Now let R(z , x) ≥ 0 be a renewal-reward function (the non-negativity restriction will be relaxed later). We want to find E [R(t)] for this function. In principle, we can find this expectation by quantizing the region x ≥ z in Figure 3.12 into rectangles δ1 by δ2 , multiplying the probability of each rectangle by R(z , x) in that rectangle, summing, and shrinking the δ ’s to 0. That is, without worrying about the range of the sums, X E [R(t)] ≈ Pr {Z (t)∈[nδ1 −δ1 , nδ1 ), X (t)∈(kδ2 , kδ2 +δ2 ]} R(nδ1 , kδ2 ) n,k X = [m(t−nδ1 +δ1 ) − m(t−nδ1 ) − o(δ1 )][FX (kδ2 +δ2 ) − FX (kδ2 )]R(nδ1 , kδ2 ). n,k 3.5. RENEWAL-REWARD PROCESSES; ENSEMBLE-AVERAGES 115 It is simpler, in going to the limit as δ1 , δ2 → 0 to take δ2 → 0 first, since the integration over x does not involve m(t), which is the difficult quantity to work with. Thus define r(z ) by8 Z1 r(z ) = R(z , x)dFX (x). (3.46) x=z Going to the limit δ2 → 0 in (3.46) and simplifying with (3.46), we have X E [R(t)] ≈ [m(t−nδ1 +δ1 ) − m(t−nδ1 ) − o(δ1 )]r(nδ1 ). (3.47) n We can now make this precise by upper and lower bounding E [R(t)]. Using (3.44) to upper and lower bound the first term of (3.47) and upper bounding r(z ) over an interval (nδ1 − δ1 , nδ1 ] by r(n, δ1 ) = supnδ1 −δ1 ≤z≤nδ1 r(z ), E [R(t)] ≤ t/δ1 X [m(t − nδ1 + δ1 ) − m(t − nδ1 )]r(n, δ1 ). (3.48) n=1 We can similarly lower bound R(t) as E [R(t)] ≥ t/δ1 X n=1 [m(t − nδ1 + δ1 ) −...
View Full Document

Ask a homework question - tutors are online