This preview shows page 1. Sign up to view the full content.
Unformatted text preview: ess in the realworld. It was shown, via the laws
of large numbers, that probabilities become essentially observable via relative frequencies
calculated over repeated experiments.
There are too many texts on elementary probability to mention here, and most of them
serve to give added understanding and background to the material in this chapter. We
recommend Bertsekas and Tsitsiklis [2] and also [15] and [23], as both sound and readable.
Kolmogorov [14] is readable for the mathematically mature and is also of historical interest
as the translation of the 1933 book that ﬁrst put probability on a ﬁrm mathematical basis.
Feller [8] is the classic extended and elegant treatment of elementary material from a mature
point of view. Rudin [17] is an excellent text on measure theory for those with advanced
mathematical preparation. 1.7. APPENDIX 1.7 49 Appendix 1.7.1 Table of standard random variables The following tables summarize the properties of some common random variables. If a
density or PMF is speciﬁed only in a given region, it is assumed to be zero elsewhere.
Name
Exponential
Erlang
Gaussian
Uniform Density fX (x) Mean Variance MGF ∏ exp(−∏x); x≥0 1
∏ 1
∏2 ∏
∏−r x≥0 n
∏ n
∏2 ¥ a σ2 exp(ra + r2 σ 2 /2) σ>0 a
2 a2
12 exp(ra)−1
ra a>0 ∏n xn−1 exp(−∏x)
;
(n−1)!
1
√
σ 2π exp
1
a; ≥ −(x−a)2
2σ 2 0≤x≤a ≥ ∏
∏−r ¥n ∏>0
∏ > 0, n ≥ 1 Name PMF pN (n) Mean Variance MGF Binary pN (0) = 1−q ; pN (1) = q q q (1−q ) 1 − q + q er 0≤q≤1 mq mq (1 − q ) [1 − q + q er ]m 0≤q≤1 1
q 1−q
q2 q er
1−(1−q )er 0<q≤1 ∏ ∏ exp[∏(er − 1)] ∏>0 Binomial
Geometric
Poisson °m¢ n
m−n ; 0≤n≤m
n q (1 − q )
q (1−q )n−1 ; n≥1
∏n exp(−∏)
;
n! n≥0 50 1.8 CHAPTER 1. INTRODUCTION AND REVIEW OF PROBABILITY Exercises Exercise 1.1. Consider a sequence A1 , A2 , . . . of events each of which have probability
zero.
P
P
a) Find Pr { m An } and ﬁnd limm→1 Pr { m An }. What you have done is to show
n=1
n=1
that the sum of a countably inﬁnite set of numbers each equal to 0 is perfectly well deﬁned
as 0.
b) For a sequence of possible phases, a1 , a2 , . . . between 0 and 2π , and a sequence of singleS
ton events, An = {an }, ﬁnd Pr { n An } assuming that the phase is uniformly distributed.
c) Now let An be the empty event φ for all n. Use (1.1) to show that Pr {φ} = 0. S
Exercise 1.2. Let A1 and A2 be arbitrary events and show that Pr {A1 A2 }+Pr {A1 A2 } =
Pr {A1 } + Pr {A2 }. Explain which parts of the sample space are being double counted on
both sides of this equation and which parts are being counted once.
Exercise 1.3. Let A1 , A2 , . . . , be a sequenceS disjoint events and assume that Pr {An } =
of
2−n−1 for each n ≥ 1. Assume also that ≠ = 1 An .
n=1
a) Show that these assumptions violate the axioms of probability. b) Show that if (1.3) is substituted for the third of those axioms, then the above assumptions
satisfy the axioms.
This shows that the countable additivity of Axiom 3 says something more than the ﬁnite
additivity of (1.3).
Exercise 1.4. This exercise derives the probability of an arbitrary (nondisjoint) union of
events and also derives the union bound.
a) For 2 arbitrary events A1 and A2 , show that
[
[
A1 A2 = A1 (A2 −A1 ), where A2 −A1 = A2 Ac and where A1 and A2 − A1 are disjoint Hint: This is what Venn
1
diagrams were invented for.
b Show that for any n ≥ 2 and any events A1 , . . . , An ,
≥[n−1 ¥ [
≥[n−1 ¥ [ ≥
[n−1 ¥
Ai
An =
Ai
An −
Ai ,
i=1 i=1 i=1 where the two parenthesized expressions on the right are disjoint.
c) Show that
Pr n[ n oX
n
[n−1 o
An =
Pr An −
Ai .
n i=1 1.8. EXERCISES 51 n
o
S
d) Show that for each n, Pr An − n−1 Ai ≤ Pr {An }. Use this to show that
i=1
n[
oX
Pr
An ≤
Pr {An } .
n n Exercise 1.5. Consider a sample space of 8 equiprobable sample points and let A1 , A2 , A3
be three events each of probability 1/2 such that Pr {A1 A2 A3 } = Pr {A1 } Pr {A2 } Pr {A3 }. a) Create an example where Pr {A1 A2 } = Pr {A1 A3 } = 1 but Pr {A2 A3 } = 1 . Hint: Make
4
8
a table with a row for each sample point and a column for each event and experiment with
choosing which sample points belong to each event (the answer is not unique).
b) Show that, for your example, A2 and A3 are not independent. Note that the deﬁnition
of statistical independence would be very strange if it allowed A1 , A2 , A3 to be independent
while A2 and A3 are dependent. This illustrates why the deﬁnition of independence requires
(1.11) rather than just (1.12).
Exercise 1.6. Suppose X and Y are discrete rv’s with the PMF pX Y (xi , yj ). Show (a
picture will help) that this is related to the joint distribution function by
pX Y (xi , yj ) = lim δ >0,δ →0 [F (xi , yj ) − F (xi − δ, yj ) − F (xi , yj − δ ) + F (xi − δ, yj − δ )] . Exercise 1.7. The text shows that, for a nonnegative rv X with distribution function
R1
FX (x), E [X ] = 0 [1 − FX (x)]dx.
a) Write this...
View
Full
Document
 Spring '09
 R.Srikant

Click to edit the document details