Discrete-time stochastic processes

The version of the slln in theorem 14 is not the

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: sted with each one T contained in the previous one, i.e., Bn ⊆ Bn−1 . It follows from this that Bn = n=1 Bk for k each n, and thus Pr {Bn } = Pr n\n k=1 o Bk . (1.64) Since {Bn ; n ≥ 1} is nested, Pr {Bn } is nonincreasing in n. Since Pr {Bn } is also nonnegative for each n, the limit exists and (1.61) says that limn→1 Pr {Bn } = 0. The righthand side of (1.64), as a function of n, is the same sequence of numbers, which must then also approach 0. This is (1.62), which is thus equivalent to (1.61). T TS The event n Bn = n m≥n Am occurs frequently in probability theory and is called the lim sup of the sequence {An ; n ≥ 1}. This event is the set of sample points which are in each T n , and which thus, for each n, occur in some Am with m ≥ n. This means that if B ω ∈ n Bn , it must occur in an inﬁnite collection of the Am . For Am = {| Sm − X | > ≤}, m these sample points correspond to sample averages more than ≤ from the mean for an inﬁnite number of m. The SLLN thus says that the collective probability of these sample points is 0. We now prove Theorem 1.4 for the special case where the moment generating function gX (r) exists around r = 0. The proof is based on the following lemmas: Ø ©Ø ™ Lemma 1.1. Assume that gX (r) exists around r = 0 and let Am = ØSm /m − X Ø > ≤ for P each m ≥ 1. Then 1=1 Pr {Am } < 1. m Proof: First we break Pr {Am } into two terms, Ø ΩØ æ Ω æ Ω æ Ø Sm Ø Sm Sm Ø Pr {Am } = Pr Ø Ø m − X Ø > ≤ = Pr m − X > ≤ + Pr m − X < −≤ . (1.65) m From (1.41), the moment generating function of Sm is = gX (r). We can now apply the 36 CHAPTER 1. INTRODUCTION AND REVIEW OF PROBABILITY exponential bound of (1.45) to the ﬁrst term above, Ω æ © ™ Sm Pr −X >≤ = Pr Sm > mX + m≤ m ≤ gSm (r)e−mrX −mr≤ h im = gX (r)e−rX −r≤ . As pointed out in (1.47), gX (r)e−rX −r≤ < 1 for small enough r > 0. gX (r)e−rX −r≤ for some such r, it follows that 0 < α < 1 and Ω æ Sm Pr − X > ≤ ≤ αm . m Letting α = (1.66) Using (1.48) in the same way on the second term in (1.65), there is some β , 0 < β < 1, such that Ω æ Sm Pr − X < −≤ ≤ β m . m Substituting these inequalities in (1.65), Pr {Am } ≤ αm + β m . Thus X1 m=1 Pr {Am } ≤ completing the proof. X1 m=1 (αm + β m ) = α β + < 1, 1−α 1−β Lemma 1.2. (Borel Cantelli) Let A1 , A2 , . . . , be a sequence of arbitrary events for which P1 m=1 Pr[Am ] < 1. Then h\1 [1 i Pr Am = 0. (1.67) n=1 m=n P P Proof: The hypothesis 1=1 Pr {Am } < 1 means that n =1 Pr {Am } converges to a m m ﬁnite limit as n → 1, and thus that X1 lim Pr {Am } = 0. (1.68) n→1 m=n Upper bounding the probability of an intersection of events by the probability of any term in that intersection, we get n\ [ o n[ o Pr Am ≤ Pr Am for each n ≥ 0 n m≥n ≤ 1 X m=n m≥n Pr {Am } , where we have used the union bound. Since this inequality holds for each n, it also holds in the limit n → 1, completing the proof. 1.4. THE LAWS OF LARGE NUMBERS 37 Ø ©Ø ™ P Proof of Theorem 1.4: Lemma 1.1 shows that m Pr {Am } < 1 for Am = Ø Sm − X Ø > ≤ m and Lemma 1.2 shows that this implies (1.68) and (1.67). Note that the assumption of Pmoment generating function was used only in Lemma 1.1, and a P was only used to show that m Pr {Am } < 1. Exercise 1.34 shows that m Pr {Am } < 1 also follows from the assumption that X has a ﬁnite fourth moment. Section 7.8.1 uses more powerful tools to show the suﬃciency of a second moment, and advanced texts on mathematical probability are demonstrate the suﬃciency of a ﬁnite mean. The version of the SLLN in Theorem 1.4 is not the version that is most useful. Before proceeding to that version, the following restatement of the previous version will be useful in proving and understanding the more useful version. Lemma 1.3. A sequence of rv’s {Sn /n; n ≥ 1} satisﬁes the condition Ø Ω\ ΩØ ææ [ Ø Sm Ø Ø Ø>≤ Pr − XØ =0 for every ε > 0 n≥1 m≥n Ø m if and only if it satisﬁes Ω[ Pr k≥1 \ n≥1 Ø ΩØ ææ Ø Sm Ø1 Ø Ø> − XØ = 0. m≥n Ø m k [ (1.69) (1.70) Proof: First assume that (1.69) is satisﬁed. Using the union bound on k in (1.70), Ø Ø æ ΩØ æ [ \ [ ΩØ S Øm Ø 1 X \ [ Ø Sm Ø 1 Ø Ø Ø Ø Pr Pr Ø m − XØ > k ≤ Ø m − XØ > k . k≥1 n≥1 m≥n k≥1 n≥1 m≥n Each term on the right side above is zero by choosing ε in (1.69) to be 1/k. The sum of these terms is then also 0, showing that (1.69) implies (1.70). Next assume that (1.70) is satisﬁed. Since the probability of a union can be no less than the probability of any term in the union, we see that for every integer k, Ø Ø æ æ [ \ [ ΩØ S \ [ ΩØ S Øm Ø 1 Øm Ø 1 Ø Ø Ø Ø 0 = Pr Ø m − X Ø > k ≥ Pr Ø m − XØ > k . k≥1 n≥1 m≥n n≥1 m≥n This implies (1.69) for every ε > 1/k and, since k is arbitrary, for every ε > 0. We are now...
View Full Document

This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.

Ask a homework question - tutors are online