David Morin - Probability for the Enthusiastic Beginner-CreateSpace (2016).pdf - PROBABILITY For the Enthusiastic Beginner David Morin Harvard

# David Morin - Probability for the Enthusiastic Beginner-CreateSpace (2016).pdf

This preview shows page 1 out of 370 pages.

#### You've reached the end of your free preview.

Want to read all 370 pages?

Unformatted text preview: PROBABILITY For the Enthusiastic Beginner David Morin Harvard University © David Morin 2016d ISBN-10: 1523318678 ISBN-13: 978-1523318674 Printed by CreateSpace Additional resources located at: ˜ djmorin/book.html Contents Preface 1 2 Combinatorics 1.1 Factorials . . . . . . . . . . . . . . . . 1.2 Permutations . . . . . . . . . . . . . . 1.3 Ordered sets, repetitions allowed . . . . 1.4 Ordered sets, repetitions not allowed . . 1.5 Unordered sets, repetitions not allowed . 1.6 What we know so far . . . . . . . . . . 1.7 Unordered sets, repetitions allowed . . . 1.8 Binomial coeﬃcients . . . . . . . . . . 1.8.1 Coins and Pascal’s triangle . . . 1.8.2 (a + b) n and Pascal’s triangle . 1.8.3 Properties of Pascal’s triangle . 1.9 Summary . . . . . . . . . . . . . . . . 1.10 Exercises . . . . . . . . . . . . . . . . 1.11 Problems . . . . . . . . . . . . . . . . 1.12 Solutions . . . . . . . . . . . . . . . . vii . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2 3 7 12 14 20 21 29 29 31 33 35 35 36 41 Probability 2.1 Definition of probability . . . . . . . . . . . . . . . . . . 2.2 The rules of probability . . . . . . . . . . . . . . . . . . . 2.2.1 AND: The “intersection” probability, P( A and B) 2.2.2 OR: The “union” probability, P( A or B) . . . . . 2.2.3 (In)dependence and (non)exclusiveness . . . . . . 2.2.4 Conditional probability . . . . . . . . . . . . . . . 2.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.1 The art of “not” . . . . . . . . . . . . . . . . . . . 2.3.2 Picking seats . . . . . . . . . . . . . . . . . . . . 2.3.3 Socks in a drawer . . . . . . . . . . . . . . . . . . 2.3.4 Coins and dice . . . . . . . . . . . . . . . . . . . 2.3.5 Cards . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Four classic problems . . . . . . . . . . . . . . . . . . . . 2.4.1 The Birthday Problem . . . . . . . . . . . . . . . 2.4.2 The Game-Show Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 57 59 60 68 71 73 75 75 76 79 81 83 85 85 87 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.3 The Prosecutor’s Fallacy 2.4.4 The Boy/Girl Problem . 2.5 Bayes’ theorem . . . . . . . . . 2.6 Stirling’s formula . . . . . . . . 2.7 Summary . . . . . . . . . . . . 2.8 Exercises . . . . . . . . . . . . 2.9 Problems . . . . . . . . . . . . 2.10 Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 93 97 106 108 109 109 114 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 133 141 146 150 155 163 164 165 168 4 Distributions 4.1 Discrete distributions . . . . . . . . . . . . . . . . 4.2 Continuous distributions . . . . . . . . . . . . . . 4.2.1 Motivation . . . . . . . . . . . . . . . . . 4.2.2 Probability density . . . . . . . . . . . . . 4.2.3 Probability equals area . . . . . . . . . . . 4.3 Uniform distribution . . . . . . . . . . . . . . . . 4.4 Bernoulli distribution . . . . . . . . . . . . . . . . 4.5 Binomial distribution . . . . . . . . . . . . . . . . 4.6 Exponential distribution . . . . . . . . . . . . . . . 4.6.1 Discrete case . . . . . . . . . . . . . . . . 4.6.2 Rates, expectation values, and probabilities 4.6.3 Continuous case . . . . . . . . . . . . . . 4.7 Poisson distribution . . . . . . . . . . . . . . . . . 4.7.1 Discrete case . . . . . . . . . . . . . . . . 4.7.2 Continuous case . . . . . . . . . . . . . . 4.8 Gaussian distribution . . . . . . . . . . . . . . . . 4.9 Summary . . . . . . . . . . . . . . . . . . . . . . 4.10 Exercises . . . . . . . . . . . . . . . . . . . . . . 4.11 Problems . . . . . . . . . . . . . . . . . . . . . . 4.12 Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182 182 184 184 186 189 191 192 193 196 196 199 202 207 207 209 215 221 222 222 227 5 Gaussian approximations 5.1 Binomial and Gaussian . . . . . 5.2 The law of large numbers . . . . 5.3 Poisson and Gaussian . . . . . . 5.4 Binomial, Poisson, and Gaussian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250 250 256 260 263 3 Expectation values 3.1 Expectation value . . . . . . . 3.2 Variance . . . . . . . . . . . . 3.3 Standard deviation . . . . . . 3.4 Standard deviation of the mean 3.5 Sample variance . . . . . . . . 3.6 Summary . . . . . . . . . . . 3.7 Exercises . . . . . . . . . . . 3.8 Problems . . . . . . . . . . . 3.9 Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5 5.6 5.7 5.8 5.9 6 7 The central limit theorem Summary . . . . . . . . Exercises . . . . . . . . Problems . . . . . . . . Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264 269 270 270 272 Correlation and regression 6.1 The concept of correlation . . . . . 6.2 A model for correlation . . . . . . . 6.3 The correlation coeﬃcient, r . . . . 6.4 Improving the prediction for Y . . . 6.5 Calculating ρ(x, y) . . . . . . . . . 6.6 The standard-deviation box . . . . . 6.7 The regression lines . . . . . . . . . 6.8 Two regression examples . . . . . . 6.8.1 Example 1: Retaking a test . 6.8.2 Example 2: Comparing IQ’s 6.9 Least-squares fitting . . . . . . . . . 6.10 Summary . . . . . . . . . . . . . . 6.11 Exercises . . . . . . . . . . . . . . 6.12 Problems . . . . . . . . . . . . . . 6.13 Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277 277 280 285 291 294 297 300 305 305 310 313 317 318 318 320 Appendices 7.1 Appendix A: Subtleties about probability 7.2 Appendix B: Euler’s number, e . . . . . . 7.2.1 Definition of e . . . . . . . . . . 7.2.2 Raising e to a power . . . . . . . 7.2.3 The infinite series for e x . . . . . 7.2.4 The slope of e x . . . . . . . . . . 7.3 Appendix C: Approximations to (1 + a) n 7.4 Appendix D: The slope of e x . . . . . . . 7.4.1 First derivation . . . . . . . . . . 7.4.2 Second derivation . . . . . . . . . 7.5 Appendix E: Important results . . . . . . 7.6 Appendix F: Glossary of notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335 335 339 339 341 344 346 346 350 350 353 356 359 Preface This book is written for high school and college students learning about probability for the first time. Most of the book is very practical, with a large number of concrete examples and worked-out problems. However, there are also parts that are a bit theoretical (at least for an introductory book), with many mathematical derivations. All in all, if you are looking for a book that serves as a quick reference, this may not be the one for you. But if you are looking for a book that starts at the beginning and derives everything from scratch in a comprehensive manner, then you’ve come to the right place. In short, this book will appeal to the reader who has a healthy level of enthusiasm for understanding how and why the standard results of probability come about. Probability is a very accessible (and extremely fun!) subject, packed with challenging problems that don’t require substantial background or serious math. The examples in Chapter 2 are a testament to this. Of course, there are plenty of challenging topics in probability that do require a more formal background and some heavy-duty math. This will become evident in Chapters 4 and 5 (and the latter part of Chapter 3). However, technically the only math prerequisite for this book is a comfort with algebra. Calculus isn’t relied on, although there are a few problems that do involve calculus. These are marked clearly. All of the problems posed at the ends of the chapters have solutions included. The diﬃculty is indicated by stars; most problems have two stars. One star means plug and chug, while three stars mean some serious thinking. Be sure to give a solid eﬀort when solving a problem, and don’t look at the solution too soon. If you can’t solve a problem right away, that’s perfectly fine. Just set it aside and come back to it later. It’s better to solve a problem later than to read the solution now. If you do eventually need to look at a solution, cover it up with a piece of paper and read one line at a time, to get a hint to get started. Then set the book aside and work things out for real. That way, you can still (mostly) solve it on your own. You will learn a great deal this way. If you instead head right to the solution and read it straight through, you will learn very little. For instructors using this book as the assigned textbook for a course, a set of homework exercises is posted at ˜ djmorin/book.html. A solutions manual is available to instructors upon request. When sending a request, please point to a syllabus and/or webpage for the course. The outline of this book is as follows. Chapter 1 covers combinatorics, which is the study of how to count things. Counting is critical in probability, because probabilities often come down to counting the number of ways that something can happen. In Chapter 2 we dive into actual probability. This chapter includes a large number of examples, ranging from coins to cards to four classic problems presented in Section 2.4. Chapter 3 covers expectation values, including the variance and standard deviation. A section on the “sample variance” is included; this is rather mathematical and can be skipped on a first reading. In Chapter 4 we introduce the concept of a continuous distribution and then discuss a number of the more common probability distributions. In Chapter 5 we see how the binomial and Poisson distributions reduce to a Gaussian (or normal) distribution in certain limits. We also discuss the law of large numbers and the central limit theorem. Chapter 6 is somewhat of a stand-alone chapter, covering correlation and regression. Although these topics are usually found in books on statistics, it makes sense to include them here, because all of the framework has been set. Chapter 7 contains six appendices. Appendix C deals with approximations to (1 + a) n which are critical in the calculations in Chapter 5, Appendix E lists all of the main results we derive in the book, and Appendix F contains a glossary of notation; you may want to refer to this when starting each chapter. A few informational odds and ends: This book contains many supplementary remarks that are separated oﬀ from the main text; these end with a shamrock, “♣.” The letters N, n, and k generally denote integers, while x and t generally denote continuous quantities. Upper-case letters like X denote a random variable, while lower-case letters like x denote the value that the random variable takes. We refer to the normal distribution by its other name, the “Gaussian” distribution. The numerical plots were generated with Mathematica. I will sometimes use “they” as a gender-neutral singular pronoun, in protest of the present failing of the English language. And I will often use an “ ’s” to indicate the plural of one-letter items (like 6’s on dice rolls). Lastly, we of course take the frequentist approach to probability in this introductory book. I would particularly like to thank Carey Witkov for meticulously reading through the entire book and oﬀering many valuable suggestions. Joe Swingle provided many helpful comments and sanity checks throughout the writing process. Other friends and colleagues whose input I am grateful for are Jacob Barandes, Sharon Benedict, Joe Blitzstein, Brian Hall, Theresa Morin Hall, Paul Horowitz, Dave Patterson, Alexia Schulz, and Corri Taylor. Despite careful editing, there is essentially zero probability that this book is error free (as you can show in Problem 4.16!). If anything looks amiss, please check the webpage ˜ djmorin/book.html for a list of typos, updates, additional material, etc. And please let me know if you discover something that isn’t already posted. Suggestions are always welcome. David Morin Cambridge, MA Chapter 1 Combinatorics TO THE READER: This book is available as both a paperback and an eBook. I have made a few chapters available on the web, but it is possible (based on past experience) that a pirated version of the complete book will eventually appear on file-sharing sites. In the event that you are reading such a version, I have a request: If you don’t find this book useful (in which case you probably would have returned it, if you had bought it), or if you do find it useful but aren’t able to aﬀord it, then no worries; carry on. However, if you do find it useful and are able to aﬀord the Kindle eBook (priced below \$10), then please consider purchasing it (available on Amazon). If you don’t already have the Kindle reading app for your computer, you can download it free from Amazon. I chose to self-publish this book so that I could keep the cost low. The resulting eBook price of around \$10, which is very inexpensive for a 350-page math book, is less than a movie and a bag of popcorn, with the added bonus that the book lasts for more than two hours and has zero calories (if used properly!). – David Morin Combinatorics is the study of how to count things. By “things” we mean the various combinations, permutations (diﬀerent orderings), subgroups, and so on, that can be formed from a given set of objects/people/etc. For example, how many diﬀerent outcomes are possible if you flip a coin four times? How many diﬀerent full-house hands are there in poker? How many diﬀerent committees of three people can be chosen from five people? What if we additionally designate one person as the committee’s president? Knowing how to count these types of things is critical for an understanding of probability, because when calculating the probability of a given event, we often need to count the number of ways that the event can happen. The outline of this chapter is as follows. In Section 1.1 we introduce the concept of factorials, which are ubiquitous in the study of probability. In Section 1.2 we learn how to count the number of possible permutations (orderings) of a set of objects. Section 1.3 covers the number of possible combined outcomes of a repeated experiment, where each repetition has an identical set of possible results. Examples 2 Chapter 1. Combinatorics include rolling dice and flipping coins. In Section 1.4 we learn how to count the number of subgroups that can be formed from a given set of objects, where the order within the subgroup matters. An example is choosing a committee of people in which all of the positions are distinct. Section 1.5 covers the related question of the number of subgroups that can be formed from a given set of objects, where the order within the subgroup doesn’t matter. An example is a poker hand; the order of the cards in the hand is irrelevant. We find that the answer takes the form of a binomial coeﬃcient. In Section 1.6 we summarize the various results we have found so far. We discover that one result is missing from our counting repertoire, and we remedy this in Section 1.7. In Section 1.8 we look at the binomial coeﬃcients in more detail. After learning in this chapter how to count all sorts of things, we’ll see in Chapter 2 how the counting can be used to calculate probabilities. It’s usually a trivial step to obtain a probability once you’ve counted the relevant things, so the work we do here will prove well worth it. 1.1 Factorials Before getting into the discussion of actual combinatorics, we first need to look at a certain quantity that comes up again and again. This quantity is called the factorial. We’ll see throughout this chapter that when dealing with a situation that involves an integer N, we often need to consider the product of the first N integers. This product is called “N factorial,” and it is denoted by “N!”.1 For the first few integers, we have: 1! = 1, 2! = 1 · 2 = 2, 3! = 1 · 2 · 3 = 6, 4! = 1 · 2 · 3 · 4 = 24, 5! = 1 · 2 · 3 · 4 · 5 = 120, 6! = 1 · 2 · 3 · 4 · 5 · 6 = 720. (1.1) As N increases, N! gets very large very fast. For example, 10! = 3, 628, 800, and 20! ≈ 2.43 · 1018 . In Chapter 2 we will introduce an approximation to N! called Stirling’s formula. This formula makes it clear what we mean by the statement, “N! gets very large very fast.” We should add that 0! is defined to be 1. Of course, 0! doesn’t make much sense, because when we talk about the product of the first N integers, it is understood that we start with 1. Since 0 is below this starting point, it is unclear what 0! actually means. However, there is no need to try too hard to make sense of it, because as we’ll see below, if we simply define 0! to be 1, then a number of formulas turn out to be very nice. 1I don’t know why someone long ago picked the exclamation mark for this notation. But just remember that it has nothing to do with the more common grammatical use of the exclamation mark for emphasis. So try not to get too excited when you see “N!”! 1.2. Permutations 3 Having defined N!, we can now start counting things. With the exception of the result in Section 1.3, all of the main results in this chapter involve factorials. 1.2 Permutations A permutation of a set of objects is a way of ordering them. For example, if we have three people – Alice, Bob, and Carol – then one permutation of them is Alice, Bob, Carol. Another permutation is Carol, Alice, Bob. Another is Bob, Alice, Carol. It turns out that there are six permutations in all, as we will see below. The goal of this section is to learn how to count the number of possible permutations. We’ll do this by starting oﬀ with the very simple case where we have only one object. Then we’ll consider two objects, then three, and so on, until we see a pattern. The route we take here will be a common one throughout this book: Although many of the results can be derived in a few lines of reasoning, we’ll take the longer route where we start with a few...
View Full Document

• Fall '13

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern

Stuck? We have tutors online 24/7 who can help you get unstuck.
Ask Expert Tutors You can ask You can ask You can ask (will expire )
Answers in as fast as 15 minutes