This preview shows page 1. Sign up to view the full content.
Unformatted text preview: sted with each one
T
contained in the previous one, i.e., Bn ⊆ Bn−1 . It follows from this that Bn = n=1 Bk for
k
each n, and thus
Pr {Bn } = Pr n\n k=1 o
Bk . (1.64) Since {Bn ; n ≥ 1} is nested, Pr {Bn } is nonincreasing in n. Since Pr {Bn } is also nonnegative
for each n, the limit exists and (1.61) says that limn→1 Pr {Bn } = 0. The righthand side of
(1.64), as a function of n, is the same sequence of numbers, which must then also approach
0. This is (1.62), which is thus equivalent to (1.61).
T
TS
The event n Bn = n m≥n Am occurs frequently in probability theory and is called the
lim sup of the sequence {An ; n ≥ 1}. This event is the set of sample points which are in
each T n , and which thus, for each n, occur in some Am with m ≥ n. This means that if
B
ω ∈ n Bn , it must occur in an inﬁnite collection of the Am . For Am = { Sm − X  > ≤},
m
these sample points correspond to sample averages more than ≤ from the mean for an inﬁnite
number of m. The SLLN thus says that the collective probability of these sample points is
0.
We now prove Theorem 1.4 for the special case where the moment generating function gX (r)
exists around r = 0. The proof is based on the following lemmas:
Ø
©Ø
™
Lemma 1.1. Assume that gX (r) exists around r = 0 and let Am = ØSm /m − X Ø > ≤ for
P
each m ≥ 1. Then 1=1 Pr {Am } < 1.
m
Proof: First we break Pr {Am } into two terms,
Ø
ΩØ
æ
Ω
æ
Ω
æ
Ø Sm
Ø
Sm
Sm
Ø
Pr {Am } = Pr Ø
Ø m − X Ø > ≤ = Pr m − X > ≤ + Pr m − X < −≤ . (1.65) m
From (1.41), the moment generating function of Sm is = gX (r). We can now apply the 36 CHAPTER 1. INTRODUCTION AND REVIEW OF PROBABILITY exponential bound of (1.45) to the ﬁrst term above,
Ω
æ
©
™
Sm
Pr
−X >≤
= Pr Sm > mX + m≤
m
≤ gSm (r)e−mrX −mr≤
h
im
= gX (r)e−rX −r≤ . As pointed out in (1.47), gX (r)e−rX −r≤ < 1 for small enough r > 0.
gX (r)e−rX −r≤ for some such r, it follows that 0 < α < 1 and
Ω
æ
Sm
Pr
− X > ≤ ≤ αm .
m Letting α = (1.66) Using (1.48) in the same way on the second term in (1.65), there is some β , 0 < β < 1, such
that
Ω
æ
Sm
Pr
− X < −≤ ≤ β m .
m
Substituting these inequalities in (1.65), Pr {Am } ≤ αm + β m . Thus
X1 m=1 Pr {Am } ≤ completing the proof. X1 m=1 (αm + β m ) = α
β
+
< 1,
1−α 1−β Lemma 1.2. (Borel Cantelli) Let A1 , A2 , . . . , be a sequence of arbitrary events for which
P1
m=1 Pr[Am ] < 1. Then
h\1 [1
i
Pr
Am = 0.
(1.67)
n=1 m=n P
P
Proof: The hypothesis 1=1 Pr {Am } < 1 means that n =1 Pr {Am } converges to a
m
m
ﬁnite limit as n → 1, and thus that
X1
lim
Pr {Am } = 0.
(1.68)
n→1 m=n Upper bounding the probability of an intersection of events by the probability of any term
in that intersection, we get
n\ [
o
n[
o
Pr
Am
≤ Pr
Am
for each n ≥ 0
n m≥n ≤ 1
X m=n m≥n Pr {Am } , where we have used the union bound. Since this inequality holds for each n, it also holds
in the limit n → 1, completing the proof. 1.4. THE LAWS OF LARGE NUMBERS 37 Ø
©Ø
™
P
Proof of Theorem 1.4: Lemma 1.1 shows that m Pr {Am } < 1 for Am = Ø Sm − X Ø > ≤
m
and Lemma 1.2 shows that this implies (1.68) and (1.67).
Note that the assumption of Pmoment generating function was used only in Lemma 1.1, and
a
P
was only used to show that m Pr {Am } < 1. Exercise 1.34 shows that m Pr {Am } < 1
also follows from the assumption that X has a ﬁnite fourth moment. Section 7.8.1 uses
more powerful tools to show the suﬃciency of a second moment, and advanced texts on
mathematical probability are demonstrate the suﬃciency of a ﬁnite mean. The version of the SLLN in Theorem 1.4 is not the version that is most useful. Before
proceeding to that version, the following restatement of the previous version will be useful
in proving and understanding the more useful version.
Lemma 1.3. A sequence of rv’s {Sn /n; n ≥ 1} satisﬁes the condition
Ø
Ω\
ΩØ
ææ
[
Ø Sm
Ø
Ø
Ø>≤
Pr
− XØ
=0
for every ε > 0
n≥1
m≥n Ø m
if and only if it satisﬁes
Ω[
Pr k≥1 \ n≥1 Ø
ΩØ
ææ
Ø Sm
Ø1
Ø
Ø>
− XØ
= 0.
m≥n Ø m
k [ (1.69) (1.70) Proof: First assume that (1.69) is satisﬁed. Using the union bound on k in (1.70), Ø
Ø
æ
ΩØ
æ [ \ [ ΩØ S
Øm
Ø 1 X \ [ Ø Sm
Ø 1
Ø
Ø
Ø
Ø
Pr
Pr
Ø m − XØ > k ≤
Ø m − XØ > k . k≥1 n≥1 m≥n k≥1 n≥1 m≥n Each term on the right side above is zero by choosing ε in (1.69) to be 1/k. The sum of
these terms is then also 0, showing that (1.69) implies (1.70).
Next assume that (1.70) is satisﬁed. Since the probability of a union can be no less than
the probability of any term in the union, we see that for every integer k, Ø
Ø
æ
æ [ \ [ ΩØ S \ [ ΩØ S
Øm
Ø 1
Øm
Ø 1
Ø
Ø
Ø
Ø
0 = Pr
Ø m − X Ø > k ≥ Pr Ø m − XØ > k . k≥1 n≥1 m≥n n≥1 m≥n This implies (1.69) for every ε > 1/k and, since k is arbitrary, for every ε > 0. We are now...
View
Full
Document
This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.
 Spring '09
 R.Srikant

Click to edit the document details