This preview shows page 1. Sign up to view the full content.
Unformatted text preview: ♣♣
♣♣
♣♣
♣♣
♣
♣♣
♣♣
♣♣
♣♣ ♣♣
♣♣
♣♣
♣♣
♣
♣♣
♣♣
♣♣
♣♣ ♣♣
♣♣
♣♣
♣♣
♣
♣♣
♣♣
♣♣
♣♣ x>z ♣♣
♣♣
♣♣
♣♣
♣
♣♣
♣♣
♣♣
♣♣ ♣♣
♣♣
♣♣
♣♣
♣♣
♣♣
♣♣
♣♣
♣♣
♣♣ ♣♣
♣♣
♣♣
♣♣
♣♣
♣♣
♣♣
♣♣
♣♣
♣♣ ♣♣
♣♣
♣♣
♣♣
♣♣
♣♣
♣♣
♣♣
♣♣
♣♣ (3.40) (3.41) ♣♣
♣♣
♣♣
♣♣
♣♣
♣♣
♣♣
♣♣
♣♣
♣♣ Figure 3.12: Joint density of age and duration.
Similarly, the mean duration can be found from (3.40).
£§
E X2
E [X (t)] =
.
X (3.42) These ensembleaverages agree with the timeaverages found in (3.25) and (3.26). In calculating timeaverages, the somewhat paradoxical result that the timeaverage duration is
greater than E [X ] was explained by the large interrenewal intervals being weighted more
heavily in the timeaverage. Here the same eﬀect occurs, but it can be given a diﬀerent
interpretation: the joint density, at z and x, for age and duration, is proportional to the 114 CHAPTER 3. RENEWAL PROCESSES interrenewal density fX (x), but the marginal density for duration is weighted by x since
the range for age is proportional to x.
Using the joint probability density of Z (t) and X (t) to evaluate the expectation of an
arbitrary reward function R(t) = R(Z (t), X (t)) in the limit t → 1, we get
Z1Zx
fX (x) dx
E [Rn ]
lim E [R(t)] =
R(z , x) dz
=
.
(3.43)
t→1
X
X
x=0 z =0
where E [Rn ] is deﬁned in (3.29). Thus the limiting ensembleaverage is the same as the
timeaverage. This result should not be surprising. Since we are dealing with nonarithmetic
renewals, the probability of a renewal in a small interval becomes independent of where the
interval is, so long as the interval is far enough removed from 0 for the process to be in
“steadystate”. Since the reward function depends on the process only through the current
renewal interval, the reward function must also become independent of time.
In the intuitive derivation above, we assumed a density for the interrenewal variables
{Xn ; n ≥ 1}, and we ignored the mathematical issues of taking limits. Here we correct
those defects, but the reader should bear in mind that the logical train of the argument is
exactly the same as before. Assume throughout that Xn is nonarithmetic with distribution
function FX (x). Since m(τ ) = E [N (τ )], the expected number of renewals in (t − z , t − z + δ ]
is m(t − z + δ ) − m(t − z ). As shown in Exercise 3.15, the assumption that the interrenewal variables are strictly positive ensures that the probability of more than one renewal
in (t − z , t − z + δ ] is insigniﬁcant, and more speciﬁcally, it also shows that for any t > z > 0,
Pr {renewal ∈ (t − z , t − z + δ ]} = m(t − z + δ ) − m(t − z ) − o(δ ), (3.44) where limδ→0 o(δ )/δ = 0 and o(δ ) ≥ 0. Conditional on a renewal at a sample value t − z ,
the interrenewal interval X starting at t − z has the distribution FX . Thus
P {renewal ∈ (t−z , t−z +δ1 ], X ∈ (x, x+δ2 ]} = = [m(t−z +δ1 ) − m(t−z ) − o(δ1 )][FX (x+δ2 ) − FX (x)]. For x > z , the joint event {renewal ∈ (t − z , t − z + δ1 ], X ∈ (x, x + δ2 ]} is the same as the
joint event {Z (t) ∈ [z − δ1 , z ), X (t) ∈ (x, x + δ2 ]}. Thus,
Pr {Z (t)∈[z −δ1 , z ), X (t)∈(x, x+δ2 ]} = [m(t−z +δ1 ) − m(t−z ) − o(δ1 )][FX (x+δ2 ) − FX (x)].
(3.45)
Now let R(z , x) ≥ 0 be a renewalreward function (the nonnegativity restriction will be
relaxed later). We want to ﬁnd E [R(t)] for this function. In principle, we can ﬁnd this expectation by quantizing the region x ≥ z in Figure 3.12 into rectangles δ1 by δ2 , multiplying
the probability of each rectangle by R(z , x) in that rectangle, summing, and shrinking the
δ ’s to 0. That is, without worrying about the range of the sums,
X
E [R(t)] ≈
Pr {Z (t)∈[nδ1 −δ1 , nδ1 ), X (t)∈(kδ2 , kδ2 +δ2 ]} R(nδ1 , kδ2 )
n,k X
=
[m(t−nδ1 +δ1 ) − m(t−nδ1 ) − o(δ1 )][FX (kδ2 +δ2 ) − FX (kδ2 )]R(nδ1 , kδ2 ).
n,k 3.5. RENEWALREWARD PROCESSES; ENSEMBLEAVERAGES 115 It is simpler, in going to the limit as δ1 , δ2 → 0 to take δ2 → 0 ﬁrst, since the integration
over x does not involve m(t), which is the diﬃcult quantity to work with. Thus deﬁne r(z )
by8
Z1
r(z ) =
R(z , x)dFX (x).
(3.46)
x=z Going to the limit δ2 → 0 in (3.46) and simplifying with (3.46), we have
X
E [R(t)] ≈
[m(t−nδ1 +δ1 ) − m(t−nδ1 ) − o(δ1 )]r(nδ1 ). (3.47) n We can now make this precise by upper and lower bounding E [R(t)]. Using (3.44) to
upper and lower bound the ﬁrst term of (3.47) and upper bounding r(z ) over an interval
(nδ1 − δ1 , nδ1 ] by r(n, δ1 ) = supnδ1 −δ1 ≤z≤nδ1 r(z ),
E [R(t)] ≤ t/δ1
X [m(t − nδ1 + δ1 ) − m(t − nδ1 )]r(n, δ1 ). (3.48) n=1 We can similarly lower bound R(t) as
E [R(t)] ≥ t/δ1
X n=1 [m(t − nδ1 + δ1 ) −...
View Full
Document
 Spring '09
 R.Srikant

Click to edit the document details