Discrete-time stochastic processes

# A write this integral as a sum where x is a non

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ess in the real-world. It was shown, via the laws of large numbers, that probabilities become essentially observable via relative frequencies calculated over repeated experiments. There are too many texts on elementary probability to mention here, and most of them serve to give added understanding and background to the material in this chapter. We recommend Bertsekas and Tsitsiklis [2] and also [15] and [23], as both sound and readable. Kolmogorov [14] is readable for the mathematically mature and is also of historical interest as the translation of the 1933 book that ﬁrst put probability on a ﬁrm mathematical basis. Feller [8] is the classic extended and elegant treatment of elementary material from a mature point of view. Rudin [17] is an excellent text on measure theory for those with advanced mathematical preparation. 1.7. APPENDIX 1.7 49 Appendix 1.7.1 Table of standard random variables The following tables summarize the properties of some common random variables. If a density or PMF is speciﬁed only in a given region, it is assumed to be zero elsewhere. Name Exponential Erlang Gaussian Uniform Density fX (x) Mean Variance MGF ∏ exp(−∏x); x≥0 1 ∏ 1 ∏2 ∏ ∏−r x≥0 n ∏ n ∏2 ¥ a σ2 exp(ra + r2 σ 2 /2) σ&gt;0 a 2 a2 12 exp(ra)−1 ra a&gt;0 ∏n xn−1 exp(−∏x) ; (n−1)! 1 √ σ 2π exp 1 a; ≥ −(x−a)2 2σ 2 0≤x≤a ≥ ∏ ∏−r ¥n ∏&gt;0 ∏ &gt; 0, n ≥ 1 Name PMF pN (n) Mean Variance MGF Binary pN (0) = 1−q ; pN (1) = q q q (1−q ) 1 − q + q er 0≤q≤1 mq mq (1 − q ) [1 − q + q er ]m 0≤q≤1 1 q 1−q q2 q er 1−(1−q )er 0&lt;q≤1 ∏ ∏ exp[∏(er − 1)] ∏&gt;0 Binomial Geometric Poisson °m¢ n m−n ; 0≤n≤m n q (1 − q ) q (1−q )n−1 ; n≥1 ∏n exp(−∏) ; n! n≥0 50 1.8 CHAPTER 1. INTRODUCTION AND REVIEW OF PROBABILITY Exercises Exercise 1.1. Consider a sequence A1 , A2 , . . . of events each of which have probability zero. P P a) Find Pr { m An } and ﬁnd limm→1 Pr { m An }. What you have done is to show n=1 n=1 that the sum of a countably inﬁnite set of numbers each equal to 0 is perfectly well deﬁned as 0. b) For a sequence of possible phases, a1 , a2 , . . . between 0 and 2π , and a sequence of singleS ton events, An = {an }, ﬁnd Pr { n An } assuming that the phase is uniformly distributed. c) Now let An be the empty event φ for all n. Use (1.1) to show that Pr {φ} = 0. S Exercise 1.2. Let A1 and A2 be arbitrary events and show that Pr {A1 A2 }+Pr {A1 A2 } = Pr {A1 } + Pr {A2 }. Explain which parts of the sample space are being double counted on both sides of this equation and which parts are being counted once. Exercise 1.3. Let A1 , A2 , . . . , be a sequenceS disjoint events and assume that Pr {An } = of 2−n−1 for each n ≥ 1. Assume also that ≠ = 1 An . n=1 a) Show that these assumptions violate the axioms of probability. b) Show that if (1.3) is substituted for the third of those axioms, then the above assumptions satisfy the axioms. This shows that the countable additivity of Axiom 3 says something more than the ﬁnite additivity of (1.3). Exercise 1.4. This exercise derives the probability of an arbitrary (non-disjoint) union of events and also derives the union bound. a) For 2 arbitrary events A1 and A2 , show that [ [ A1 A2 = A1 (A2 −A1 ), where A2 −A1 = A2 Ac and where A1 and A2 − A1 are disjoint Hint: This is what Venn 1 diagrams were invented for. b Show that for any n ≥ 2 and any events A1 , . . . , An , ≥[n−1 ¥ [ ≥[n−1 ¥ [ ≥ [n−1 ¥ Ai An = Ai An − Ai , i=1 i=1 i=1 where the two parenthesized expressions on the right are disjoint. c) Show that Pr n[ n oX n [n−1 o An = Pr An − Ai . n i=1 1.8. EXERCISES 51 n o S d) Show that for each n, Pr An − n−1 Ai ≤ Pr {An }. Use this to show that i=1 n[ oX Pr An ≤ Pr {An } . n n Exercise 1.5. Consider a sample space of 8 equiprobable sample points and let A1 , A2 , A3 be three events each of probability 1/2 such that Pr {A1 A2 A3 } = Pr {A1 } Pr {A2 } Pr {A3 }. a) Create an example where Pr {A1 A2 } = Pr {A1 A3 } = 1 but Pr {A2 A3 } = 1 . Hint: Make 4 8 a table with a row for each sample point and a column for each event and experiment with choosing which sample points belong to each event (the answer is not unique). b) Show that, for your example, A2 and A3 are not independent. Note that the deﬁnition of statistical independence would be very strange if it allowed A1 , A2 , A3 to be independent while A2 and A3 are dependent. This illustrates why the deﬁnition of independence requires (1.11) rather than just (1.12). Exercise 1.6. Suppose X and Y are discrete rv’s with the PMF pX Y (xi , yj ). Show (a picture will help) that this is related to the joint distribution function by pX Y (xi , yj ) = lim δ &gt;0,δ →0 [F (xi , yj ) − F (xi − δ, yj ) − F (xi , yj − δ ) + F (xi − δ, yj − δ )] . Exercise 1.7. The text shows that, for a non-negative rv X with distribution function R1 FX (x), E [X ] = 0 [1 − FX (x)]dx. a) Write this...
View Full Document

Ask a homework question - tutors are online