Chapter 6
Limit Theorems
•
In this chapter, we consider various bounds on probabilities that don’t depend on speciﬁc
forms of CDFs/pdfs/pmfs
•
Relative frequencies and large numbers of trials
•
Laws of large numbers
•
Asymptotic distributions of random variables
•
P{
X
> u} = 1 – F
X
(u)
→
0 as u
→
∞
•
Given the pdf/pmf/CDF, we can ﬁnd P{
X
> 5} exactly
•
What can be said about P{
X
> 5} when we don’t know the probabilistic description?
•
How small is P{
X
> 5}?
•
Fundamental idea:
If g(x)
≤
h(x) for all x
∈
(a,b), then
∫
a
b
g(x)dx
≤
∫
a
b
h(x)dx
•
Markov’s Inequality
is an upper bound on P{
X
>
α
} for nonnegative random variables with
finite mean
μ
•
Markov’s Inequality:
If
X
is a nonnegative random variable with finite mean
μ
, then,
for any
α
> 0,
P{
X
>
α
}
≤
μ
α
•
The bound is uninteresting for
α
≤
μ
•
Bound
→
0 as
α
→
∞
•
Proof:
P{
X
>
α
} =
∫
a
∞
f(u)du =
∫
0
∞
g(u)f(u)du
where g(u) =
1,
u >
α
,
0,
elsewhere.
α
α
u
u
g(u)
1
f(u)
•
P{
X
>
α
} =
∫
0
∞
g(u)f(u)du
≤
∫
0
∞
h(u)f(u)du where h(u) = u/
α
α
u
1
α
1
h(u) = u/
α
u
•
P{
X
>
α
}
≤
∫
0
∞
(u/
α
)f(u)du =
μ
α
•
P{
X
>
α
}
≤
μ
/
α
© 1997 by Dilip V. Sarwate.
All rights reserved.
No part of this manuscript may be reproduced, stored in a retrieval
system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise
without the prior written permission of the author.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentChapter 6
Limit Theorems
159
•
Because the bound is so general, it can be applied when very little is known about the
distribution
•
Because the bound is so generally applicable, it is often quite loose
•
Example:
P{
X
> 10}
≤
10
–1
for
all
random variables with average value 1
•
If
X
is an exponential random variable with parameter 1, i.e.,
μ
= 1, then
P{
X
> 10} = exp(–10)
≈
4.54
×
10
–5
<< 10
–1
•
But, do we
know
the distribution of
X
?
•
For
λ
> 0, the
Chernoff bound
uses exp
λ
(u–
α
) as an upper bound on the step function g
•
P{
X
>
α
}
≤
∫
0
∞
exp
λ
(u–
α
)f(u)du = exp(–
λα
) E[exp(
λ
X
)]
•
For exponential RV, P{
X
>
α
}
≤
exp(–
λα
)/(1–
λ
)
•
Example:
A bit transmitted over a data link is received incorrectly with probability p.
To improve reliability, each data bit is sent n times.
This is the end of the preview.
Sign up
to
access the rest of the document.
 Fall '06
 SCHWAGER
 Central Limit Theorem, Probability theory, relative frequency, University of Illinois

Click to edit the document details