probability - Notes on Probability A.E Frazho 2 Contents 1...

Info icon This preview shows pages 1–6. Sign up to view the full content.

View Full Document Right Arrow Icon
Notes on Probability A.E. Frazho
Image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
2
Image of page 2
Contents 1 The Probability Measure 7 1.1 The axioms of probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.2 Conditional probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 1.3 The birthday problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.4 The gambler’s ruin problem . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.5 Bayes Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.5.1 Drug testing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 1.5.2 A classical example of Bayes rule . . . . . . . . . . . . . . . . . . . . . 17 1.5.3 The Monty Hall Problem. . . . . . . . . . . . . . . . . . . . . . . . . 18 1.6 Independent events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 1.7 PageRank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 1.7.1 Markov matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 2 Random Variables 35 2.1 The distribution functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 2.2 The density function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 2.3 The exponential density . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 2.4 The uniform distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 2.5 The Gaussian random variable . . . . . . . . . . . . . . . . . . . . . . . . . . 43 2.5.1 An affine function of a random variable . . . . . . . . . . . . . . . . . 46 2.6 The diffusion equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 2.7 The binomial random variable . . . . . . . . . . . . . . . . . . . . . . . . . . 49 2.8 The geometric random variable . . . . . . . . . . . . . . . . . . . . . . . . . 50 2.9 The Poisson random variable . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 2.10 Functions of a uniform random variable . . . . . . . . . . . . . . . . . . . . . 54 2.11 The joint distribution function . . . . . . . . . . . . . . . . . . . . . . . . . . 56 2.11.1 Independent random variables. . . . . . . . . . . . . . . . . . . . . . 58 2.12 The joint density function . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 2.12.1 Independent random variables. . . . . . . . . . . . . . . . . . . . . . 61 2.13 A uniformly distributed dart board . . . . . . . . . . . . . . . . . . . . . . . . 62 2.14 The Raleigh density. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 2.14.1 The Box-Muller transform . . . . . . . . . . . . . . . . . . . . . . . . 66 2.15 The maximum and minimum of random variables . . . . . . . . . . . . . . . 69 2.15.1 The minimum of random variables . . . . . . . . . . . . . . . . . . . 70 3
Image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
4 CONTENTS 2.16 The Buffon needle problem. . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 2.17 The meeting problem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 3 Expectation 77 3.1 The expectation of a random variable . . . . . . . . . . . . . . . . . . . . . . 77 3.1.1 The uniform distribution: mean and variance. . . . . . . . . . . . . . 79 3.1.2 The exponential distribution: mean and variance . . . . . . . . . . . 80 3.1.3 The Gaussian distribution: mean and variance . . . . . . . . . . . . . 80 3.2 Properties of the expectation . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 3.2.1 A kinetic energy example . . . . . . . . . . . . . . . . . . . . . . . . . 82 3.2.2 A sinusoid example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 3.2.3 Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 3.3 The median . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 3.4 The binomial distribution: mean and variance . . . . . . . . . . . . . . . . . 90 3.4.1 Chebyshev’s inequality . . . . . . . . . . . . . . . . . . . . . . . . . . 91 3.5 The geometric distribution: mean and variance . . . . . . . . . . . . . . . . 93 3.6 The Poisson distribution: mean and variance . . . . . . . . . . . . . . . . . . 94 3.7 The expectation of several random variables . . . . . . . . . . . . . . . . . . 95 3.8 A uniform dart board. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 3.9 Independent random variables and expectation . . . . . . . . . . . . . . . . . 99 3.10 The gambler’s ruin: mean and variance . . . . . . . . . . . . . . . . . . . . . 101 3.11 The expectation for n random variables . . . . . . . . . . . . . . . . . . . . . 102 3.11.1 The binomial density revisited . . . . . . . . . . . . . . . . . . . . . . 104 3.12 A lawn sprinkler problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 3.12.1 The density for Θ such that x = γ sin(2Θ) is uniform . . . . . . . . . 108 3.13 Random vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 4 The Projection Theorem 111 4.1 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 4.2 The projection theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 4.3 The Gram matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 4.3.1 An approximation of e t . . . . . . . . . . . . . . . . . . . . . . . . . . 120 4.4 The mean and variance revisited . . . . . . . . . . . . . . . . . . . . . . . . . 121 4.5 A least squares optimization problem . . . . . . . . . . . . . . . . . . . . . . 122 4.5.1 An application to curve fitting . . . . . . . . . . . . . . . . . . . . . . 123 5 Least Squares 127 5.1 Random vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 5.2 Least squares estimation of a random vector . . . . . . . . . . . . . . . . . . 128 5.2.1 An example with additive noise . . . . . . . . . . . . . . . . . . . . . 130 5.2.2 Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 5.3 Classical least squares estimation . . . . . . . . . . . . . . . . . . . . . . . . 133
Image of page 4
CONTENTS 5 6 Conditional Expectation 137 6.1 The condition expectation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 6.2 Applications of E x = EE ( x | y ) . . . . . . . . . . . . . . . . . . . . . . . . . 140 6.2.1 The geometric distribution revisited . . . . . . . . . . . . . . . . . . . 141 6.2.2 A classical maze example. . . . . . . . . . . . . . . . . . . . . . . . . 142 6.3 An example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 6.4 Another example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 6.5 Functions of two random variables . . . . . . . . . . . . . . . . . . . . . . . . 149 6.5.1 The Box-Muller transformation revisited . . . . . . . . . . . . . . . . 150 6.6 An area example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 6.7 An example: The sum independent random variables . . . . . . . . . . . . . . 156 6.8 The sum of two exponential random variables . . . . . . . . . . . . . . . . . . 158 6.8.1 The case when E v = 1 . . . . . . . . . . . . . . . . . . . . . . . . . . 159 6.8.2 The case when E v ̸ = 1 . . . . . . . . . . . . . . . . . . . . . . . . . . 160 6.8.3 The linear estimate . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 6.9 A uniform random variable example. . . . . . . . . . . . . . . . . . . . . . . 162 6.10 Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 7 Gaussian Random Vectors 171 7.1 Linear estimation revisited . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 7.2
Image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 6
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern