This preview shows page 1. Sign up to view the full content.
Unformatted text preview: robability
of a renewal at time kd is given by
lim Pr {N (kd) − N (kd − d) = 1} = k→1 3.4 d
.
X (3.20) Renewalreward processes; timeaverages There are many situations in which, along with a renewal counting process {N (t); t ≥ 0},
there is another randomly varying function of time, called a reward function {R(t); t ≥ 0}. 106 CHAPTER 3. RENEWAL PROCESSES R(t) models a rate at which the process is accumulating a reward. We shall illustrate many
examples of such processes and see that a “reward” could also be a cost or any randomly
varying quantity of interest. The important restriction on these reward functions is that
R(t) at a given t depends only on the particular interrenewal interval containing t. We
start with several examples to illustrate the kinds of questions addressed by this type of
process.
Example 3.4.1. (timeaverage Residual Life) For a renewal counting process {N (t), t ≥
0}, let Y (t) be the residual life at time t. The residual life is deﬁned as the interval from t
until the next renewal epoch, i.e., as SN (t)+1 − t. For example, if we arrive at a bus stop at
time t and buses arrive according to a renewal process, Y (t) is the time we have to wait for
a bus to arrive (see Figure 3.6). We interpret {Y (t); t ≥ R } as a reward function. The time0
6 (1/t) t Y (τ )dτ . We are interested in the
average of Y (t), over the interval (0, t], is given by
0
limit of this average as t → 1 (assuming that it exists in some sense). Figure 3.6 illustrates
a sample function of a renewal counting process {N (t); t ≥ 0} and shows the residual life
Y (t) for R
that sample function. Note that, for a given sample function {Y (t) = y (t)}, the
t
integral 0 y (τ ) dτ is simply a sum of isosceles right triangles, with part of a ﬁnal triangle
at the end. Thus it can be expressed as
Z t 0 n(t) 1X 2
y (τ )dτ =
xi +
2
i=1 Z t y (τ )dτ τ =sn(t) where {xi ; 0 < i < 1} is the set of sample values for the interrenewal intervals.
Since this relationship holds for every sample point, we see that the random variable
Rt
0 Y (τ )dτ can b e expressed in terms of the interrenewal random variables Xn as
Z t Y (τ )dτ = τ =0 Zt
N (t)
1X 2
Xn +
Y (τ )dτ .
2 n=1
τ =SN (t) Although the ﬁnal term above can be easily evaluated for a given SN (t) (t), it is more
convenient to use the following bound:
Z
N (t)
N (t)+1
1X 2 1 t
1X
2
X≤
Y (τ )dτ ≤
Xn .
2t n=1 n
t τ =0
2t n=1 (3.21) The term on the left can now be evaluated in the limit t → 1 (for all sample functions
except a set of probability zero) as follows:
lim t→1
6 Rt PN (t)
n=1 2t 2
Xn = lim t→1 £§
2
E X2
Xn N (t)
=
.
N (t)
2t
2E [X ] PN (t)
n=1 (3.22) Y (τ )dτ is a random variable just like any other function of a set of variables. It has a sample
0
value for each sample function of {N (t); t ≥ 0}, and its distribution function could be calculated in a
straightforward but tedious way. For arbitrary stochastic processes, integration and diﬀerentiation can
require great mathematical sophistication, but none of those subtleties occur here. 3.4. RENEWALREWARD PROCESSES; TIMEAVERAGES 107 N (t) ✛ X2 ✲
X1
S1 S2 S3 S4 S5 S6 X5 ❅
❅
❅Y (t)
X2
X4
❅
❅
❅
X6
❅
❅
X1
❅
❅
❅
❅
❅
X3
❅
❅
❅❅
❅ ❅❅
t
❅
❅❅
❅
❅
S1 S2 S3 S4 S5 S6 Figure 3.6: Residual life at time t. For any given sample function of the renewal
process, the sample function of residual life decreases linearly with a slope of −1 from
the beginning to the end of each interrenewal interval. The second equality above follows by applying the strong law of large numbers (Theorem
P
2
1.5) to n≤N (t) Xn /N (t) as N (t) approaches inﬁnity, and by applying the strong law for
renewal processes (Theorem 3.1) to N (t)/t as t → 1. The right hand term of (3.21) is
handled almost the same way:
£§
PN (t)+1 2
PN (t)+1 2
E X2
Xn
Xn N (t) + 1 N (t)
n=1
n=1
lim
= lim
=
.
(3.23)
t→1
t→1
2t
N (t) + 1
N (t)
2t
2E [X ]
Combining these two results, we see that, with probability 1 (abbreviated as W.P.1), the
timeaverage residual life is given by
£§
Rt
E X2
τ =0 Y (τ ) dτ
lim
=
W.P.1.
(3.24)
t→1
t
2E [X ]
2 2 Note that this timeaverage depends on the second moment of X ; this is X + σ 2 ≥ X ,
so the timeaverage residual life is at least half the expected interrenewal interval (which
is not surprising). On the other hand, the second moment of X can be arbitrarily large
(even inﬁnite) for any given value of E [X ], so that the time average residual life can be
arbitrarily large relative to E [X ]. This can be explained intuitively by observing that large
interrenewal intervals are weighted more heavily in this timeaverage than small interrenewal intervals. As an example, consider an interrenewal random variable X that takes
on value ≤ with §
probability 1 − ≤ and value 1/≤ with probability ≤. Then, for small ≤,
£
E [X ] ∼ 1, E X 2 ∼ 1/≤, and the time average residual life is approximately 1/(2≤) (see
Figure 3.7...
View
Full
Document
This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.
 Spring '09
 R.Srikant

Click to edit the document details