Discrete-time stochastic processes

# Thus taking as the arrival rate lim rt 0 t1 r d lim

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: m(t − nδ1 ) − o(δ1 )] r(n, δ1 ). (3.49) where r(n, δ ) = inf nδ−δ≤z≤nδ r(z ). Aside from the term o(δ1 ) in (3.49), we see from the deﬁnition of a Stieltjes integral in (1.7) that if the Stieltjes integral exists, we can express E [R(t)] as E [R(t)] = Z t τ =0 r(t − τ ) dm(τ ). (3.50) This is valid for all t, but is not always easy to calculate because of the complex behavior of m(t), especially for non-airthmetic discrete distributions. In the limit of large t, we can use Blackwell’s theorem to evaluate the limit in t (if it exists) of the right hand side of (3.48) lim t→1 t/δ X n=1 [m(t − nδ + δ ) − m(t − nδ )] r(n, δ ) = 1 δX r(n, δ ). X n=1 (3.51) Similarly, the limit of the right hand side of (3.49), if it exists, is lim t→1 8 t/δ X n=1 [m(t − nδ + δ ) − m(t − nδ ) − o(δ )] r(n, δ ) = µ ∂X 1 δ − o(δ ) r(n, δ ). X n=1 (3.52) If one insists on interpreting r(z ), one can see that r(z )/(1 − FX (z )) is E [R(t) | Z (t) = z ]. It is probably better to simply view r(z ) as a step in a derivation. 116 CHAPTER 3. RENEWAL PROCESSES P P A function r(z ) is called directly Riemann integrable ifP n≥1 r(n, δ ) and n≥1 r(n, δ ) are P ﬁnite for all δ &gt; 0 and if limδ→0 n≥1 δ r(n, δ ) = limδ→0 n≥1 δ r(n, δ ). If this latter equality R holds, then each limit is equal to z≥0 r(z )dz . If r(z ) is directly Riemann integrable, then the right hand sides of (3.51) and (3.52) are equal in the limit δ → 0. Since one is an upper bound and the other a lower bound to limt→1 E [R(t)], we see that the limit exists and is R equal to [ z≥0 r(z )dz ]/X . This can be summarized in the following theorem, known as the Key renewal theorem: Theorem 3.7 (Key renewal theorem). Let r(z ) ≥ 0 be a directly Riemann integrable function, and let m(t) = E [N (t)] for a non-arithmetic renewal process. Then Zt Z1 1 lim r(t − τ )dm(τ ) = r(z )dz . (3.53) t→1 τ =0 X z=0 Since R(z , x) ≥ 0, (3.46) shows that r(z ) ≥ 0. Also, from (3.29), E [Rn ] = Thus, combining (3.50) with (3.53), we have the corollary R z ≥0 r (z )dz . Corollary 3.2. Let {N (t); t ≥ 0} be a non-arithmetic renewal process, let R(z , x) ≥ 0, and let r(z ) ≥ 0 in (3.46) be directly Riemann integrable. Then lim E [R(t)] = t→1 E [Rn ] . X (3.54) The ma jor restrictions imposed by r(z ) being directly Riemann integrable are, ﬁrst, that R E [Rn ] = z≥0 r(z ) dz is ﬁnite, second, that r(z ) contains no impulses, and third, that r(z ) is not too wildly varying (being continuous and bounded by a decreasing integrable function is enough). It is also not necessary to assume R(z , x) ≥ 0, since, if E [Rn ] exists, one can break a more general R into positive and negative parts. The above development assumed a non-arithmetic renewal process. For an arithmetic process, the situation is somewhat simpler mathematically, but in general E [R(t)] depends on the remainder when t is divided by the span d. Usually with such processes, one is interested only in reward functions that remain constant over intervals of length d, so we can consider E [R(t)] only for t equal to multiples of d, and thus we assume t = nd here. Thus the function R(z , x) is of interest only when z and x are multiples of d, and in particular, only for x = d, 2d, . . . and for z = d, 2d, . . . , x. We follow the convention that an inter-renewal interval is open on the left and closed on the right, thus including the renewal that ends the interval. E [R(nd)] = 1i XX i=1 j =1 R(j d, id) Pr {renewal at (n − j )d, next renewal at (n − j + i)d} . Let Pi be the probability that an inter-renewal interval has size id. Using (3.20) for the limiting probability of a renewal at (n − j )d, this becomes lim E [R(nd)] = n→1 1 i XX i=1 j =1 R(j d, id) d E [Rn ] Pi = , X X (3.55) 3.6. APPLICATIONS OF RENEWAL-REWARD THEORY 117 where E [Rn ] is the expected reward over a renewal period. In using this formula, remember that R(t) is piecewise constant, so that the aggregate reward over an interval of size d around nd is dR(nd). It has been important, and theoretically assuring, to be able to ﬁnd ensemble-averages for renewal-reward functions in the limit of large t and to show (not surprisingly) that they are the same as the time-average results. The ensemble-average results are quite tricky, though, and it is wise to check results achieved that way with the corresponding time-average results. 3.6 Applications of renewal-reward theory 3.6.1 Little’s theorem Little’s theorem is an important queueing result stating that the expected number of customers in a queueing system is equal to the expected time each customer waits in the system times the arrival rate. This result is true under very general conditions; we use the G/G/1 queue as a speciﬁc example, but the reason for the greater generality will be clear as we proceed. Note that the theorem does not tell us how to ﬁnd either the expected number or expected wait; it onl...
View Full Document

## This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.

Ask a homework question - tutors are online