This preview shows page 1. Sign up to view the full content.
Unformatted text preview: verage as being essentially equal
to the mean is highly appropriate.
At the next intuitive level down, the meaning of the word essential ly becomes important
and thus involves the details of the above laws. All of the results involve how the rv’s Sn /n
change with n and in particular become better and better approximated by X . When we
talk about a sequence of rv’s (namely a sequence of functions on the sample space) being
approximated by a numerical constant, we are talking about some kind of convergence, but
it clearly is not as simple as a sequence of real numbers (such as 1/n for example) converging
to some other number (0 for example).
The purpose of this section, is to give names and deﬁnitions to these various forms of
convergence. This will give us increased understanding of the laws of large numbers already
developed, but, more important, we can later apply these convergence results to sequences
of random variables other than sample averages of IID rv’s.
We discuss four types of convergence in what follows, convergence in distribution, in probability, in mean square, and with probability 1. For each, we ﬁrst recall the type of large
number result with that type of convergence and then give the general deﬁnition. The examples are all based on a sequence {Xn ; n ≥ 1} of rv’s with partial sums Sn = X1 + · · · + Xn
and the deﬁnitions are given in terms of an arbitrary sequence {Yn ; n ≥ 1} of rv’s .
We start with the central limit theorem, which, from (1.56) says
Ω
æ Zy
µ 2∂
Sn − nX
1
−x
√
√ exp
lim Pr
≤y =
dx
for every y ∈ R.
n→1
2
nσ
2π
−1
This is illustrated in Figure 1.10 and says that the sequence (in n) of distribution functions
n
o
√
Pr Sn −nX ≤ y converges at every y to the normal distribution function at y . This is an
nσ
example of convergence in distribution. 40 CHAPTER 1. INTRODUCTION AND REVIEW OF PROBABILITY Deﬁnition 1.7. A sequence of random variables, Y1 , Y2 , . . . , converges in distribution to a
random varible Z if limn→1 FYn (z ) = FZ (z ) at each z for which FZ (z ) is continuous.
Convergence in distribution does not say that the rv’s themselves are converging in any
sense (see Exercise 1.29), but only that their distribution functions are converging. Note
√
that for the CLT, it is the rv’s Sn −nX that are converging in distribution. Convergence in
nσ
distribution does not say that the PMF’s or densities of the rv’s are converging, and it is
easy to see that a sequence of PMF’s can never converge to a density. ★✥
★✥
MS ✧✦
✧✦
W.P.1 In probability Distribution Figure 1.12: Relationship between diﬀerent kinds of convergence. Convergence in distribution is the most general and is implied by all the others. Convergence in probability
is the next most general and is implied by convergence with probability 1 (W.P.1) and
by mean square (MS) convergence, neither of which imply the other. Next, the weak law of large numbers, in the form of (1.57), says that
ΩØ
æ
Ø
Ø Sn
Ø
lim Pr Ø
− XØ ≥ ≤ = 0
for every ≤ > 0.
n→1
n As illustrated in Figure 1.8, this means that Sn /n converges in distribution to a unit step
function at X . This is an example of convergence in probability,
Deﬁnition 1.8. A sequence of random variables Y1 , Y2 , . . . , converges in probability to a
real number z if limn→1 Pr {Yn − z  ≥ ε} = 0 for every ε > 0.
An equivalent statement, as illustrated in Figure 1.9, is that the sequence Y1 , Y2 , . . . , converges in probability to z if limn→1 FYn (y ) = 0 for all y < z and limn→1 FYn (y ) = 1 for all
y > z . This shows (as illustrated in Figure 1.12) that convergence in probability is a special
case of convergence in distribution, since with convergence in probability, the sequence FYn
of distribution functions converges to a unit step at z . Note that limn→1 FYn (z ) for the z
where the step occurs is not speciﬁed. However, the step function is not continuous at z ,
so the limit there need not be speciﬁied for convergence in distribution.
Convergence in probability says quite a bit more than convergence in distribution. As an
important example of this, consider the diﬀerence Yn − Ym for n and m both large. If
{Yn ; n ≥ 1} converges in probability to z , then Yn − z and Ym − z are close to 0 with high
probability for large n and m. Thus Yn − Ym is close to 0 with high probability. More
precisely, limm→1,n→1 Pr {Yn − Ym  > ε} = 0 for every ε > 0. If the sequence {Yn ; n ≥ 1} 1.4. THE LAWS OF LARGE NUMBERS 41 merely converges in distribution to some arbitrary distribution, then Yn − Ym can be large
with high probability, even when n and m are large. An example of this is given in Exercise
1.29. In other words, convergence in distribution need not mean that the random variables
converge in any sense — it is only their distribution functions that must converge.
There is something slightly paradoxical here, since the CLT seems to say a great deal more
about how Sn /n approaches X than the weak law, but it...
View
Full
Document
This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.
 Spring '09
 R.Srikant

Click to edit the document details