Discrete-time stochastic processes

# Since n t 1 is a stopping time however walds equality

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: at Pr {S1 ≤ t} = FX (t), we can interchange the order of integration and summation to get Zt X 1 m(t) = FX (t) + Pr {Sn−1 ≤ t − x} dFX (x) x=0 n=2 = FX (t) + Z t 1 X x=0 n=1 = FX (t) + Z t x=0 Pr {Sn ≤ t − x} dFX (x) m(t − x)dFX (x) ; t ≥ 0. (3.6) An alternative derivation is given in Exercise 3.6. This integral equation is called the renewal equation. 3.3.1 Laplace transform approach If we assume that X has a density fX (x), and that this density has a Laplace transform R1 LX (s) = 0 fX (x)e−sx dx, we can take the Laplace transform of both sides of (3.6). Note 3.3. EXPECTED NUMBER OF RENEWALS 99 that the ﬁnal term in (3.6) is the convolution of m with fX , so that the Laplace transform of m(t) satisﬁes Lm (s) = LX (s)/s + Lm (s)LX (s). Solving for Lm (s), Lm (s) = LX (s) . s[1 − LX (s)] (3.7) Example 3.3.1. As a simple example of how this can be used to calculate m(t), suppose fX (x) = (1/2)e−x + e−2x for x ≥ 0. The Laplace transform is given by LX (s) = 1 1 (3/2)s + 2 + = . 2(s + 1) s + 2 (s + 1)(s + 2) Substituting this into (3.7) yields Lm (s) = (3/2)s + 2 4 1 1 = 2+ − . 2 (s + 3/2) s 3s 9s 9(s + 3/2) Taking the inverse Laplace transform, we then have m(t) = 4t 1 − exp[−(3/2)t] + . 3 9 The procedure in this example can be used for any inter-renewal density fX (x) for which the Laplace transform is a rational function, i.e., a ratio of polynomials. In such cases, Lm (s) will also be a rational function. The Heaviside inversion formula (i.e., factoring the denominator and expressing Lm (s) as a sum of individual poles as done above) can then be used to calculate m(t). In the example above, there was a second order pole at s = 0 leading to the linear term 4t/3 in m(t), there was a ﬁrst order pole at s = 0 leading to the constant 1/9, and there was a pole at s = −3/2 leading to the exponentially decaying term. We now show that a second order pole at s = 0 always occurs when LX (s) is a rational function. To see this, note that LX (0) is just the integral of fX (x), which is 1; thus 1 − LX (s) has a zero at s = 0 and Lm (s) has a second order pole at s = 0. To evaluate the residue for this second order pole, we recall that the ﬁrst and second derivatives of LX (s) at s = 0 £§ are −E [X ] and E X 2 respectively. £ § Expanding LX (s) in a power series around s = 0 then yields LX (s) = 1 − sE [X ] + (s2 /2)E X 2 plus terms of order s3 or higher. This gives us √£ § ! £§ 1 − sX + (s2 /2)E X 2 + · · · 1 1 E X2 §= Lm (s) = 2 £ + (3.8) 2 − 1 + ··· . s s X − (s/2)E [X 2 ] + · · · s2 X 2X The remaining terms are the other poles of Lm (s) with their residues. RFor values of s with R R <(s) ≥ 0, we have |LX (s)| = | fX (x)e−sx dx| ≤ fX (x)|e−sx |dx ≤ fX (x)dx = 1 with strict inequality except for s = 0. Thus LX (s) cannot have any poles on the imaginary axis or the right half plane, and 1 − LX (s) cannot have any zeros there other than the one at s = 0. It follows that all the remaining poles of Lm (s) are strictly in the left half plane. 100 CHAPTER 3. RENEWAL PROCESSES This means that the inverse transforms for all these remaining poles die out as t → 1. Thus the inverse Laplace transform of Lm (s) is £§ E X2 t m(t) = + (3.9) 2 − 1 + ε(t) for t ≥ 0. X 2X where limt→1 ε(t) = 0. We have derived (3.9) only for the special case in which fX (x) has a rational Laplace transform. For this case, (3.9) implies both the elementary renewal theorem (limt→1 m(t)/t = 1/X ) and also Blackwell’s theorem (limt→1 [m(t + δ ) − m(t)] = δ /X ). We will interpret the £§ 2 meaning of the constant term E X 2 /(2X ) − 1 in Section 3.4. In what follows, we derive the elementary renewal theorem for the general case. Some of the machinery used to prove the elementary renewal theorem is very useful in its own right and involves repeating some given experiment a random number of times. This involves some care about the rule for terminating the set of repeated experiments, and leads to a result called Wald’s equality. This will then be applied to renewal processes, and in particular to the expected epoch of the ﬁrst arrival after some given time t and the expected number of arrivals up to that epoch. We then use this result to demonstrate that m(t)/t → 1/X . 3.3.2 Random stopping times Visualize performing an experiment repeatedly, observing successive sample outputs of a given random variable (i.e., observing an outcome of X1 , X2 , . . . where the Xi are IID). The experiment is stopped when enough data has been accumulated for the purposes at hand. This type of situation occurs frequently in applications. For example, we might be required to make a choice from several hypotheses, and might repeat an experiment until the hypotheses are suﬃciently discriminated. If the number of trials is allowed to depend on the outcome, the mean number of trials required to achieve a given error probability is typically a small fraction of the...
View Full Document

## This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.

Ask a homework question - tutors are online