Paninski, Intro. Math. Stats., October 5, 200529Probability inequalities11There is an adage in probability that says that behind every limit theorem liesa probability inequality (i.e., a bound on the probability of some undesiredevent happening). Since a large part of probability theory is about provinglimit theorems, people have developed a bewildering number of inequalities.Luckily, we’ll only need a few key inequalities. Even better, three of themare really just versions of one another.Exercise 29: Can you think of exam-ple distributions for which each of the following inequalities are tight (thatis, the inequalities may be replaced by equalities)?Are these “extremal”distributions unique?Markov’s inequalityFigure 5: Markov.For a nonnegative r.v.X,P(X > u)≤E(X)u.So ifE(X) is small and we knowX≥0, thenXmust be near zero withhigh probability. (Note that the inequality isnottrue ifXcan be negative.)11HMC 1.10Liam Paninski, Intro Math Stats, 20102Prob_Stat.Liam_Paninski.2010.pdf