Discrete-time stochastic processes

# Discrete-time stochastic processes

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: equent results about time-averages. The next topic is the expected renewal rate, E [N (t)] /t. If the Laplace transform of the inter-renewal density is rational, E [N (t)] /t can be easily calculated. In general, the Wald equality shows that limt→1 E [N (t)] /t = 1/X . Finally, Blackwell’s theorem shows that the renewal epochs reach a steady-state as t → 1. The form of this steady-state depends on whether the interrenewal distribution is arithmetic (see (3.18)) or non-arithmetic (see (3.17) and (3.19)). Sections 3.4 and 3.5 add a reward function R(t) to the underlying renewal process; R(t) depends only on the inter-renewal interval containing t. The time-average value of reward exists with probability 1 and is equal to the expected reward over a renewal interval divided by the expected length of an inter-renewal interval. Under some minor restrictions imposed by the key renewal theorem, we also found that, for non-arithmetic inter-renewal distributions, limt→1 E [R(t)] is the same as the time-average value of reward. These general results were applied to residual life, age, and duration, and were also used to derive and understand Little’s theorem and the Pollaczek-Khinchin expression for the expected delay in an M/G/1 queue. Finally, all the results above were shown to apply to delayed renewal processes. For further reading on renewal processes, see Feller,[9], Ross, [16], or Wolﬀ, [22]. Feller still appears to be the best source for deep understanding of renewal processes, but Ross and Wolﬀ are somewhat more accessible. 3.9 Exercises Exercise 3.1. The purpose of this exercise is to show that for an arbitrary renewal process, the number of renewals in (0, t] is a random variable for each t > 0, i.e., to show that N (t), for each t > 0, is an actual rv rather than a defective rv. a) Let X1 , X2 , . . . , be a sequence of IID inter-renewal rv’s . Let Sn = X1 + · · · + Xn be the corresponding renewal epochs for each n ≥ 1. Assume that each Xi has a ﬁnite expectation X > 0 and use the weak law of large numbers to show that limn→1 Pr {Sn < t} = 0. b) Use part a) to show that limn→1 Pr {N ≥ n} = 0 and explain why this means that N (t) is not defective. c) Prove that N (t) is not defective without assuming that each Xi has a mean (but assuming that Pr {X = 0} 6= 1.) Hint: Lower bound each Xi by a binary rv Yi (i.e., Xi (ω ) ≥ Yi (ω ) for each sample point ω ) and show that this implies that FXi (xi ) ≤ FYi (yi ). Be sure you understand this strange reversal of inequality signs. 128 CHAPTER 3. RENEWAL PROCESSES Exercise 3.2. Let {Xi ; i ≥ 1} be the inter-renewal intervals of a renewal process generalized to allow for inter-renewal intervals of size 0 and let Pr {Xi = 0} =α, 0 < α < 1. Let {Yi ; i ≥ 1} be the sequence of non-zero interarrival intervals. For example, if X1 = x1 >0, X2 = 0, X3 = x3 >0, . . . , then Y1 =x1 , Y2 =x3 , . . . , . a) Find the distribution function of each Yi in terms of that of the Xi . b) Find the PMF of the number of arrivals of the generalized renewal process at each epoch at which arrivals occur. c) Explain how to view the generalized renewal process as an ordinary renewal process with inter-renewal intervals {Yi ; i ≥ 1} and bulk arrivals at each renewal epoch. Exercise 3.3. Let {Xi ; i≥1} be the inter-renewal intervals of a renewal process and assume e that E [Xi ] = 1. Let b > 0 be an arbitrary number and Xi be a truncated random variable e e deﬁned by Xi = Xi if Xi ≤ b and Xi = b otherwise. hi e a) Show that for any constant M > 0, there is a b suﬃciently large so that E Xi ≥ M . e e b) Let {N (t); t≥0} be the renewal counting process with inter-renewal intervals {Xi ; i ≥ 1} e and show that for all t > 0, N (t) ≥ N (t). c) Show that for all sample functions N (t, ω ), except a set of probability 0, N (t, ω )/t < 2/M for all suﬃciently large t. Note: Since M is arbitrary, this means that lim N (t)/t = 0 with probability 1. Exercise 3.4. a) Let J be a stopping rule and In be the indicator random variable of the P event {J ≥ n}. Show that J = n≥1 In . b) Show that I1 ≥ I2 ≥ I3 ≥ . . . , i.e., show that for each n > 1, In (ω ) ≥ In+1 (ω ) for each ω ∈ ≠ (except perhaps for a set of probability 0). Exercise 3.5. Is it true for a renewal process that: a) N (t) < n if and only if Sn > t? b) N (t) ≤ n if and only if Sn ≥ t? c) N (t) > n if and only if Sn < t? Exercise 3.6. Let {N (t); t ≥ 0} be a renewal counting process and let m(t) = E [N (t)] be the expected number of arrivals up to and including time t. Let {Xi ; i ≥ 1} be the inter-renewal times and assume that FX (0) = 0. a) For all x > 0 and t > x show that E [N (t)|X1 =x] = E [N (t − x)] + 1. Rt b) Use part (a) to show that m(t) = FX (t) + 0 m(t − x)dFX (x) for t > 0. This equation is the renewal equation derived diﬀerently in (3.6). c) Suppose that X is an exponential random variable of parameter ∏. Evaluate Lm (s) from (3.7); verify that the inverse La...
View Full Document

## This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.

Ask a homework question - tutors are online