Discrete-time stochastic processes

# Unfortunately in general e zn is unequal to limn1 e zn

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: g to any stopping time such that E [N ] < 1. 7.5. THRESHOLDS, STOPPING RULES, AND WALD’S IDENTITY 295 The second derivative of (7.24) with respect to r is ££ §§ E (SN − N ∞ 0 (r))2 − N ∞ 00 (r) exp{rSN − N ∞ (r)} = 0. At r = 0, this is h i 2 2 2 E SN − 2N SN X + N 2 X = E [N ] σX . (7.26) This equation is often diﬃcult to use because of the cross term between SN and N , but its main application comes in the case where X = 0. In this case, Wald’s equality provides no information about E [N ], but (7.26) simpliﬁes to £ 2§ 2 E SN = E [N ] σX . (7.27) Example 7.5.1 (Simple random walks again). As an example, consider the simple random walk of Section 7.1.1 with Pr {X =1} = Pr {X = − 1} = 1/2, and assume that α > 0 and β < 0 are integers. Since Sn takes on only integer values and changes only by ±1, it takes on the value α or β before exceeding either of these values. Thus SN = α or SN = β . Let qα denote Pr {SN = α}. The expected value of SN is then αqα + β (1 − qα ). From Wald’s equality, E [SN ] = 0, so qα = −β ; α−β 1 − qα = α . α−β (7.28) From (7.27), £ 2§ 2 E [N ] σX = E SN = α2 qα + β 2 (1 − qα ). (7.29) 2 Using the value of qα from (7.28) and recognizing that σX = 1, 2 E [N ] = −β α/σX = −β α. (7.30) As a sanity check, note that if α and β are each multiplied by some large constant k, then 2 E [N ] increases by k2 . Since σSn = n, we would expect Sn to ﬂuctuate with increasing n √ with typical values growing as n, and thus it is reasonable for the time to reach a threshold to increase with the square of the distance to the threshold. We also notice that if β is decreased toward −1, while holding α constant, then qα → 1 and E [N ] → 1, which helps explain the possibility of winning one coin with probability 1 in a coin tossing game, assuming we have an inﬁnite capital to risk and an inﬁnite time to wait. For more general random walks with X = 0, there is usually an overshoot when the threshold is crossed. If the magnitudes of α and β are large relative to the range of X , however, it is 2 often reasonable to ignore the overshoots, and then −β α/σX becomes a good approximation to E [N ]. If one wants to include the overshoot, then the eﬀect of the overshoot must be taken into account both in (7.28) and (7.29). We next apply Wald’s identity to upper bound Pr {SN ≥ α} for the case where X < 0. 296 CHAPTER 7. RANDOM WALKS, LARGE DEVIATIONS, AND MARTINGALES Corollary 7.1. Under the conditions of 7.2, assume that ∞ (r) has a root at r∗ > 0. Then Pr {SN ≥ α} ≤ exp(−r∗ α). (7.31) Proof: Wald’s identity, with r = r∗ , reduces to E [exp(r∗ SN )] = 1. We can express this as Pr {SN ≥ α} E [exp(r∗ SN ) | SN ≥ α] + Pr {SN ≤ β } E [exp(r∗ SN ) | SN ≤ β ] = 1. (7.32) Since the second term on the left is non-negative, Pr {SN ≥ α} E [exp(r∗ SN ) | SN ≥ α] ≤ 1. (7.33) Given that SN ≥ α, we see that exp(r∗ SN ) ≥ exp(r∗ α). Thus Pr {SN ≥ α} exp(r∗ α) ≤ 1. (7.34) which is equivalent to (7.31). This bound is valid for all β < 0, and thus is also valid in the limit β → −1 (see Exercise 7.12 for a more careful demonstration that (7.31) is valid without a lower threshold). Equation (7.31) is also valid for the case of Figure 7.5, where ∞ (r) < 0 for all r ∈ (0, r+ ). The exponential bound in (7.22) shows that Pr {Sn ≥ α} ≤ exp(−r∗ α) for each n; (7.31) is S stronger than this. It shows that Pr { n {Sn ≥ α}} ≤ exp(−r∗ α). This also holds in the limit β → −1. When Corollary 7.1 is applied to the G/G/1 queue in Theorem 7.1, (7.31) is referred to as the Kingman Bound. Corollary 7.2 (Kingman Bound). Let {Xi ; i ≥ 1} and {Yi ; i ≥ 0} be the interarrival intervals and service times of a G/G/1 queue that is empty£ at time 0 when customer 0 § arrives. Let {Ui = Yi−1 − Xi ; i ≥ 1}, and let ∞ (r) = ln{E eU r } be the semi-invariant moment generating function of each Ui . Assume that ∞ (r) has a root at r∗ > 0. Then Wn , the waiting time of the nth arrival and W , the steady state waiting time, satisfy Pr {Wn ≥ α} ≤ Pr {W ≥ α} ≤ exp(−r∗ α) ; for al l α > 0. (7.35) In most applications, a positive threshold crossing for a random walk with a negative drift corresponds to some exceptional, and usually undesirable, circumstance (for example an error in the hypothesis testing problem or an overﬂow in the G/G/1 queue). Thus an upper bound such as (7.31) provides an assurance of a certain level of performance and is often more useful than either an approximation or an exact expression that is very diﬃcult to evaluate. For a random walk with X > 0, the exceptional circumstance is Pr {SN ≤ β }. This can be analyzed by changing the sign of X and β and using the results for a negative expected value. These exponential bounds do not work for X = 0, and we will not analyze that case here. Note that (7.31) is an upper bound because, ﬁrs...
View Full Document

## This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.

Ask a homework question - tutors are online