lectures3 - MATH 235A Probability Theory Lecture Notes Fall...

MATH 235A – Probability Theory
Lecture Notes, Fall 2009
Part III: The central limit theorem
Dan Romik
Department of Mathematics, UC Davis
Draft (December 4, 2009)
Lecture 10: Stirling’s formula and the de Moivre-Laplace
theorem
We start with a motivating example that was also historically the first instance in which
the phenomenon that came to be known as the Central Limit Theorem was observed. Let
X
1
, X
2
, . . . be an i.i.d. of Binom(1 , p ) random variables, and let S
n
=
∑
n
k =1
X
k
, a r.v. with
distribution Binom( n, p ).
Theorem 1 (The de Moivre-Laplace theorem) . For any t ∈ R ,
P
S
n
- np
p
np (1 - p )
≤ t
!
---→
n →∞
Φ( t ) =
1
√
2 π
Z
t
-∞
e
- x
2
/ 2
dx.
Since this is such a concrete example, the proof will simply require us to estimate a sum
of the form
∑
0 ≤ k ≤ t
(
n
k
)
p k (1 - p ) n - k . Knowing how to estimate such sums is a useful skill in
its own right. Since the binomial coefficients are involved, we also need some preparation
related to Stirling’s formula.
Lemma 2. The limit C = lim
n →∞
n !
√
n ( n/e ) n
exists.
1

Proof.
log n ! =
n X
k =1
log k =
n X
=1
Z
k
1
dx
x
=
Z
n
1
n - b x c
x
dx
=
Z
n
1
n + 1
2
+ ( { x } - 1
2
) - x
x
dx = ( n + 1 / 2) log n - n + 1 +
Z
n
1
{ x } - 1
2
x
dx
= ( n + 1 / 2) log n - n + 1 +
Z
∞
1
{ x } - 1
2
x
dx + o (1) ,
where the last integral converges because
R
t
1
( { x } - 1
2
) dx is bounded and 1 /x goes to 0 as
x → ∞ .
Note that an easy consequence of Lemma 2 is that
(
2 n
n
)
= (1 + o (1))2 2 n /C
p
n/ 2. We
shall now use this to find the value of C .
Lemma 3. Let f : R → R be an n + 1 times continuously-differentiable function. Then for
all x ∈ R , we have
f ( x ) = f (0) + f
0
(0) x +
f 00 (0)
2
x
2
+ . . . +
f ( n ) (0)
n !
x
n
+ R
n
( x ) ,
where
R
n
( x ) =
1
n !
Z
x
0
f
( n +1)
( t )( x - t )
n
dt.
Proof. This follows by induction on n , using integration by parts.
Lemma 4. C =
√
2 π .
Proof. Apply Lemma 3 with f ( x ) = (1 + x ) 2 n +1 to compute R
n
(1):
1
2 2 n +1
R
n
(1) =
1
2 2 n +1
·
1
n !
Z
1
0
(2 n + 1)(2 n ) · · · ( n + 1)(1 + t )
n
(1 - t )
n
dt
=
2
(
2 n
n
)
2 2 n +1
( n +
1
2
)
Z
1
0
(1 - t
2
)
n
dt =
(
2 n
n
)
√
n
2 2 n
(1 +
1
2 n
)
Z
√
n
0
1 -
u
2
n
n
du
---→
n →∞
√
2
C
Z
∞
0
e
- u
2
du =
√
2
C
·
√
π
2
.
2

The convergence of the integrals is justified by the fact that (1 - u 2 /n ) n ≤ e - u
2
for all
0 ≤ u ≤
√
n , and (1 - u 2 /n ) n → e - u
2
as n → ∞ , uniformly on compact intervals. To finish
the proof, note that
1
2
2 n +1
R
n
(1) =
X
n<k ≤ 2 n +1
(
2 n +1
k
)
2
2 n +1
=
1
2
(this is the probability that a Binom(2 n + 1 , 1 / 2) random variable takes a value > n ).
Therefore C =
√
2 π , as claimed.
Corollary 5 (Stirling’s formula) . lim
n →∞
n !
√
2 πn ( n/e ) n
= 1 .
Note that the proof is based on computing P ( S
2 n +1
> n ) in two different ways, when
S
2 n +1
∼ Binom(2 n + 1 , 1 / 2). This is just the special case p = 1 / 2 , t = 0 of Theorem 1!
In this very special case, by symmetry the probability is equal to 1 / 2; on the other hand,
Lemma 3 enables us to relate this to the asymptotic behavior of n ! and to (half of) the
gaussian integral
R
∞
-∞
e
- x
2
dx . The evaluation of the constant C in Stirling’s formula is the
part that is attributed to James Stirling. The form that appears in Lemma 2 is due to
Abraham de Moivre (1733).