Isye 2027

# Isye 2027 - Probability with Engineering Applications ECE...

This preview shows pages 1–4. Sign up to view the full content.

Probability with Engineering Applications ECE 313 Course Notes Bruce Hajek Department of Electrical and Computer Engineering University of Illinois at Urbana-Champaign July 14, 2011 c 2011 by Bruce Hajek All rights reserved. Permission is hereby given to freely print and circulate copies of these notes so long as the notes are left intact and not reproduced for commercial purposes. Email to [email protected], pointing out errors or hard to understand passages or providing comments, is welcome.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Contents 1 Foundations 3 1.1 Embracing uncertainty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2 Axioms of probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.3 Calculating the size of various sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.4 Probability experiments with equally likely outcomes . . . . . . . . . . . . . . . . . . 16 1.5 Countably infinite sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2 Discrete-type random variables 21 2.1 Random variables and probability mass functions . . . . . . . . . . . . . . . . . . . . 21 2.2 The mean and variance of a random variable . . . . . . . . . . . . . . . . . . . . . . 23 2.3 Conditional probabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 2.4 Independence and the binomial distribution . . . . . . . . . . . . . . . . . . . . . . . 30 2.4.1 Mutually independent events . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 2.4.2 Independent random variables (of discrete-type) . . . . . . . . . . . . . . . . 33 2.4.3 Bernoulli distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 2.4.4 Binomial distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 2.5 Geometric distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 2.6 Bernoulli process and the negative binomial distribution . . . . . . . . . . . . . . . . 38 2.7 The Poisson distribution–a limit of Bernoulli distributions . . . . . . . . . . . . . . . 41 2.8 Maximum likelihood parameter estimation . . . . . . . . . . . . . . . . . . . . . . . . 43 2.9 Markov and Chebychev inequalities and confidence intervals . . . . . . . . . . . . . . 45 2.10 The law of total probability, and Bayes formula . . . . . . . . . . . . . . . . . . . . . 47 2.11 Binary hypothesis testing with discrete-type observations . . . . . . . . . . . . . . . 54 2.11.1 Maximum likelihood (ML) decision rule . . . . . . . . . . . . . . . . . . . . . 55 2.11.2 Maximum a posteriori probability (MAP) decision rule . . . . . . . . . . . . . 56 2.12 Reliability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 2.12.1 Union bound . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 2.12.2 Network outage probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 2.12.3 Distribution of the capacity of a flow network . . . . . . . . . . . . . . . . . . 64 2.12.4 Analysis of an array code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 2.12.5 Reliability of a single backup . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 iii
iv CONTENTS 3 Continuous-type random variables 69 3.1 Cumulative distribution functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 3.2 Continuous-type random variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 3.3 Uniform distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 3.4 Exponential distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 3.5 Poisson processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 3.5.1 Time-scaled Bernoulli processes . . . . . . . . . . . . . . . . . . . . . . . . . . 80 3.5.2 Definition and properties of Poisson processes . . . . . . . . . . . . . . . . . . 82 3.5.3 The gamma distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 3.6 Linear scaling of pdfs and the Gaussian distribution . . . . . . . . . . . . . . . . . . 87 3.6.1 Scaling rule for pdfs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 3.6.2 The Gaussian (normal) distribution . . . . . . . . . . . . . . . . . . . . . . . 89 3.6.3 The central limit theorem and the Gaussian approximation . . . . . . . . . . 93 3.7 ML parameter estimation for continuous-type variables . . . . . . . . . . . . . . . . . 97 3.8 Functions of a random variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 3.8.1 The distribution of a function of a random variable . . . . . . . . . . . . . . . 98 3.8.2 Generating a random variable with a specified distribution . . . . . . . . . . 109 3.8.3 The area rule for expectation based on the CDF . . . . . . . . . . . . . . . . 111 3.9 Failure rate functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 3.10 Binary hypothesis testing with continuous-type observations . . . . . . . . . . . . . . 113 4 Jointly Distributed Random Variables 119 4.1 Joint cumulative distribution functions . . . . . . . . . . . . . . . . . . . . . . . . . . 119 4.2 Joint probability mass functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 4.3 Joint probability density functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 4.4 Independence of random variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 4.4.1 Definition of independence for two random variables . . . . . . . . . . . . . . 132 4.4.2 Determining from a pdf whether independence holds . . . . . . . . . . . . . . 132 4.5 Distribution of sums of random variables . . . . . . . . . . . . . . . . . . . . . . . . . 135 4.5.1 Sums of integer-valued random variables . . . . . . . . . . . . . . . . . . . . . 135 4.5.2

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern