100%(1)1 out of 1 people found this document helpful
This preview shows page 1 - 4 out of 55 pages.
The preview shows page 2 - 4 out of 55 pages.
Statistics 4109: Probability and StatisticsLiam Paninski, [email protected]Fall 2010The course covers the material of two other courses, STAT4105 and STAT4107, in a single semester.The pace,therefore, is fast, and not all students will be able to keep up. Furthermore, the material is cumulative, that is, almostevery lecture builds on previously discussed concepts, and students unable to keep up will find themselves in a veryuncomfortable position. Students who doubt their preparation or who are concerned that they will not be able toconsistently devote time to the course would be well advised to consider taking STAT4105 this semester followed bySTAT4107 the next. However, if you're thinking of taking 4105 and 4107 in the same semester, I strongly recommendyou take 4109 instead; 4109 offers the big advantage of covering the material in the proper sequence.Also see--------Probability and Statistics, 3rd Ed., by DeGroot and Schervish (ISBN 0-201-52488-0A First Course in Probability (by S. Ross)Statistical Inference (by Casella and Berger).Table of Contents-----------------Book I:Introduction, sample spaces, probability axiomsConditional probability, Bayes rule. Independent events.continuous r.v.'s; multivariate distributionsMoment-generating functions; Covariance and correlation; sample meansSpecial continuous distributionsBook IIInequalities, LLN, SpecialDiscrete, Central limit theoremEstimation Theory:Order statistics; basic simulation theory: Monte Carlo integration, importance samplingDecision theory: admissibility; minimax and Bayes decision rules; Bias/variance of estimatorsCramer-Rao bound;Hypothesis Testing:Simple hypothesis testing; likelihood ratio tests; Neyman-Pearson lemmaLiam Paninski, Intro Math Stats, 20101Prob_Stat.Liam_Paninski.2010.pdf
Paninski, Intro. Math. Stats., October 5, 200529Probability inequalities11There is an adage in probability that says that behind every limit theorem liesa probability inequality (i.e., a bound on the probability of some undesiredevent happening). Since a large part of probability theory is about provinglimit theorems, people have developed a bewildering number of inequalities.Luckily, we’ll only need a few key inequalities. Even better, three of themare really just versions of one another.Exercise 29: Can you think of exam-ple distributions for which each of the following inequalities are tight (thatis, the inequalities may be replaced by equalities)?Are these “extremal”distributions unique?Markov’s inequalityFigure 5: Markov.For a nonnegative r.v.X,P(X > u)≤E(X)u.So ifE(X) is small and we knowX≥0, thenXmust be near zero withhigh probability. (Note that the inequality isnottrue ifXcan be negative.)11HMC 1.10Liam Paninski, Intro Math Stats, 20102Prob_Stat.Liam_Paninski.2010.pdf
Paninski, Intro. Math. Stats., October 5, 200530The proof is really simple:uP(X > u)≤∞utpX(t)dt≤∞0tpX(t)dt=E(X).Chebyshev’s inequalityFigure 6: Chebyshev.P(|X-E(X)|> u)≤V(X)u2,akaP|X-E(X)|σ(X)> u≤1u2.Proof: just look at the (nonnegative) r.v. (X-E(X))2, and apply Markov.So if the variance ofXis really small,X