Discrete-time stochastic processes

Hint sketch this expected reward as a function of t

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ) − N (t)] 1 = . δ X2 (3.74) If {Xi ; i ≥ 2} are arithmetic with span d and X1 is arithmetic with span md for some positive integer m, then lim Pr {renewal at t = id} = i→1 3.7.1 d . X2 (3.75) Delayed renewal-reward processes We have seen that the distribution of the first renewal epoch has no effect on the time or ensemble-average behavior of a renewal process (other than the ensemble dependence on time for an arithmetic process). This carries over to reward functions with almost no change. In particular, the extended version of Theorem 3.6 is as follows: 3.7. DELAYED RENEWAL PROCESSES 125 Theorem 3.12. Let {N (t); t≥0} be a delayed renewal counting process, let Z (t) = t−SN (t) , let X (t) = SN (t)+1 − SN (t) , and let R(t) = R(Z (t), X (t)) be a reward function. Assume that Z1 Zx E [Rn ] = R(z , x) dz dFXn (x) < 1 for al l n. x=0 z =0 Then, with probability one, 1 lim t→1 t Z t R(τ )dτ = τ =0 E [Rn ] for n ≥ 2. X2 (3.76) We omit the proof of this since it is a minor variation of that of theorem 3.6. Finally, since Blackwell’s theorem holds for delayed renewal processes, Eq. (3.54), giving the ensembleaverage reward for non-arithmetic processes, follows as before, yielding lim E [R(t)] = t→1 3.7.2 E [Rn ] . X2 (3.77) Transient behavior of delayed renewal processes Let m(t) = E [N (t)] for a delayed renewal process. As in (3.5), we have m(t) = 1 X n=1 Pr {N (t) ≥ n} = 1 X n=1 Pr {Sn ≤ t} . (3.78) For n ≥ 2, Sn = Sn−1 + Xn where Xn and Sn−1 are independent. From the convolution equation (1.12), Zt Pr {Sn ≤ t} = Pr {Sn−1 ≤ t − x} dF (x) for n ≥ 2. (3.79) x=0 For n = 1, Pr {Sn ≤ t} = G(t). Substituting this in (3.78) and interchanging the order of integration and summation, m(t) = G(t) + = G(t) + = G(t) + Z t 1 X x=0 n=2 Zt X 1 x=0 n=1 Zt x=0 Pr {Sn−1 ≤ t − x} dF (x) Pr {Sn ≤ t − x} dF (x) m(t − x)dF (x) ; t ≥ 0. (3.80) This is called the renewal equation and is a generalization of (3.6). [9], Section 11.1, Theorem 3.1, shows that it has a unique solution. There is another useful integral equation very similar to (3.80) that arises from breaking up b b Sn as the sum of X1 and Sn−1 where Sn−1 = X2 + · · · + Xn . Letting m(t) be the expected b number of renewals in time t for an ordinary renewal process with interarrival distribution F , 126 CHAPTER 3. RENEWAL PROCESSES a similar argument to that above, starting with Pr {Sn ≤ t} = yields m(t) = G(t) + Z n o b Pr Sn−1 ≤ t − x dG(x) 0 Rt t x=0 m(t − x)dG(x). b (3.81) This equation brings out the effect of the initial renewal interval clearly, and is useful in computation if one already knows m(t). b Frequently, the most convenient way of dealing with m(t) is through transforms. Following the same argument as that in (3.7), we get Lm (r) = (1/r)LG (r) + Lm (r)LF (r). Solving, we get Lm (r) = LG (r) . r[1 − LF (r)] (3.82) We can find m(t) from (3.82) by finding the inverse Laplace transform, using the same procedure as in Example 3.3.1. There is a second order pole at r = 0 again, and, evaluating the residue, it is 1/L0 (0) = 1/X 2 , which is not surprising in terms of Blackwell’s theorem. F We can also expand numerator and denominator of (3.82) in a power series, as in (3.8). The inverse transform, corresponding to (3.9), is £ 2§ E X2 t X1 m(t) = + − + ε(t) for t → 0. X2 2X 2 X2 (3.83) where limt→1 ε(t) = 0. 3.7.3 The equilibrium process Consider an ordinary non-arithmetic renewal process with an inter-renewal interval X of distribution F (x). We have seen that Rthe distribution of the interval from t to the next y renewal approaches FY (y ) = (1/E [X ]) 0 [1 − F (x)]dx as t → 1. This suggests that if we look at this renewal process starting at some very large t, we should see a delayed renewal process for which the distribution G(x) of the first renewal is equal to the residual life distribution FY (x) above and subsequent inter-renewal intervals should have the original distribution F (x) above. Thus it appears that such a delayed renewal process is the same as the original ordinary renewal process, except that it starts in “steady-state.” To verify this, we show that m(t) = t/X 2 is a solution to (3.80) if G(t) = FY (t). Substituting (t − x)/X 2 for m(t − x), the right hand side of (3.80) is Rt 0 [1 − F (x)]dx X2 + Rt 0 (t − x)dF (x) X2 = Rt 0 [1 − F (x)]dx X2 + Rt 0 F (x)dx X2 = t . X2 where we have used integration by parts for the first equality. This particular delayed renewal process is called the equilibrium process, since it starts off in steady state, and thus one need not worry about transients. 3.8. SUMMARY 3.8 127 Summary Sections 3.1 to 3.3 give the central results about renewal processes that form the basis for many of the subsequent chapters. The chapter starts with the strong law for renewal processes, showing that the time average rate of renewals, N (t)/t, approaches 1/X with probability 1 as t → 1. This, combined with the strong law of large numbers in Chapter 1, is the basis for most subs...
View Full Document

This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.

Ask a homework question - tutors are online