Isye 2027 - Probability with Engineering Applications ECE...

Info iconThis preview shows pages 1–4. Sign up to view the full content.

View Full Document Right Arrow Icon
Probability with Engineering Applications ECE 313 Course Notes Bruce Hajek Department of Electrical and Computer Engineering University of Illinois at Urbana-Champaign July 14, 2011 c ± 2011 by Bruce Hajek All rights reserved. Permission is hereby given to freely print and circulate copies of these notes so long as the notes are left intact and not reproduced for commercial purposes. Email to [email protected], pointing out errors or hard to understand passages or providing comments, is welcome.
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Contents 1 Foundations 3 1.1 Embracing uncertainty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2 Axioms of probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.3 Calculating the size of various sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.4 Probability experiments with equally likely outcomes . . . . . . . . . . . . . . . . . . 16 1.5 Countably infinite sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2 Discrete-type random variables 21 2.1 Random variables and probability mass functions . . . . . . . . . . . . . . . . . . . . 21 2.2 The mean and variance of a random variable . . . . . . . . . . . . . . . . . . . . . . 23 2.3 Conditional probabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 2.4 Independence and the binomial distribution . . . . . . . . . . . . . . . . . . . . . . . 30 2.4.1 Mutually independent events . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 2.4.2 Independent random variables (of discrete-type) . . . . . . . . . . . . . . . . 33 2.4.3 Bernoulli distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 2.4.4 Binomial distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 2.5 Geometric distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 2.6 Bernoulli process and the negative binomial distribution . . . . . . . . . . . . . . . . 38 2.7 The Poisson distribution–a limit of Bernoulli distributions . . . . . . . . . . . . . . . 41 2.8 Maximum likelihood parameter estimation . . . . . . . . . . . . . . . . . . . . . . . . 43 2.9 Markov and Chebychev inequalities and confidence intervals . . . . . . . . . . . . . . 45 2.10 The law of total probability, and Bayes formula . . . . . . . . . . . . . . . . . . . . . 47 2.11 Binary hypothesis testing with discrete-type observations . . . . . . . . . . . . . . . 54 2.11.1 Maximum likelihood (ML) decision rule . . . . . . . . . . . . . . . . . . . . . 55 2.11.2 Maximum a posteriori probability (MAP) decision rule . . . . . . . . . . . . . 56 2.12 Reliability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 2.12.1 Union bound . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 2.12.2 Network outage probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 2.12.3 Distribution of the capacity of a flow network . . . . . . . . . . . . . . . . . . 64 2.12.4 Analysis of an array code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 2.12.5 Reliability of a single backup . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 iii
Background image of page 2
iv CONTENTS 3 Continuous-type random variables 69 3.1 Cumulative distribution functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 3.2 Continuous-type random variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 3.3 Uniform distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 3.4 Exponential distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 3.5 Poisson processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 3.5.1 Time-scaled Bernoulli processes . . . . . . . . . . . . . . . . . . . . . . . . . . 80 3.5.2 Definition and properties of Poisson processes . . . . . . . . . . . . . . . . . . 82 3.5.3 The gamma distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 3.6 Linear scaling of pdfs and the Gaussian distribution . . . . . . . . . . . . . . . . . . 87 3.6.1 Scaling rule for pdfs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 3.6.2 The Gaussian (normal) distribution . . . . . . . . . . . . . . . . . . . . . . . 89 3.6.3 The central limit theorem and the Gaussian approximation . . . . . . . . . . 93 3.7 ML parameter estimation for continuous-type variables . . . . . . . . . . . . . . . . . 97 3.8 Functions of a random variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 3.8.1 The distribution of a function of a random variable . . . . . . . . . . . . . . . 98 3.8.2 Generating a random variable with a specified distribution . . . . . . . . . . 109 3.8.3 The area rule for expectation based on the CDF . . . . . . . . . . . . . . . . 111 3.9 Failure rate functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 3.10 Binary hypothesis testing with continuous-type observations . . . . . . . . . . . . . . 113 4 Jointly Distributed Random Variables 119 4.1 Joint cumulative distribution functions . . . . . . . . . . . . . . . . . . . . . . . . . . 119 4.2 Joint probability mass functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 4.3 Joint probability density functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 4.4 Independence of random variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 4.4.1 Definition of independence for two random variables . . . . . . . . . . . . . . 132 4.4.2 Determining from a pdf whether independence holds . . . . . . . . . . . . . . 132 4.5 Distribution of sums of random variables . . . . . . . . . . . . . . . . . . . . . . . . . 135 4.5.1 Sums of integer-valued random variables . . . . . . . . . . . . . . . . . . . . . 135 4.5.2 Sums of jointly continuous-type random variables .
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 4
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

Page1 / 188

Isye 2027 - Probability with Engineering Applications ECE...

This preview shows document pages 1 - 4. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online