Lecture Notes 13
Plug-In Estimators and The Bootstrap
This is mostly not in the text.
Can we estimate the mean of a distribution without using a parametric model? Yes. The
key idea is to rst estimate the distribution function nonparametrica
Lecture Notes 9
Asymptotic (Large Sample) Theory
Review of o, O, etc.
1. an = o(1) mean an 0 as n .
2. A random sequence An is op (1) if An 0 as n .
3. A random sequence An is op (bn ) if An /bn 0 as n .
4. np op (1) = op (np ), so n op (1/ n) = o
Lecture Notes 15
This is mostly not in the text. Some relevant material is in Chapters 11 and
We observe training data (X1 , Y1 ), . . . , (Xn , Yn ). Given a new pair (X, Y ) we want to predict
Y from X . There are two commo
Lecture Notes 7
Parametric Point Estimation
X1 , . . . , Xn p(x; ). Want to estimate = (1 , . . . , k ). An estimator
= n = w(X1 , . . . , Xn )
is a function of the data.
1. Method of Moments (MOM)
2. Maximum likelihood (MLE)
3. Bayesian estim
Lecture Notes 12
This is not in the text.
Suppose we want to estimate something without assuming a parametric model. Some
1. Estimate the cdf F .
2. Estimate a density function p(x).
3. Estimate a functional T (P ) of
Lecture Notes 17
Three Bonus Topics
Multiple Testing and Condence Intervals
Suppose we need to test many null hypotheses
H0,1 , . . . , H0,N
where N could be very large. We cannot simply test each hypotheses at level because, if
N is large, we are sure
Lecture Notes 16
Not in the text.
Sometimes we have a set of possible models and we want to choose the best model. Model
selection methods help us choose a good model. Here are some examples.
Example 1 Suppose you use a poly
Lecture Notes 8
Suppose we want to estimate a parameter using data X n = (X1 , . . . , Xn ). What is the
best possible estimator = (X1 , . . . , Xn ) of ? Minimax theory provides a framework for
answering this question.
Lecture Notes 14
Relevant material is scattered throughout the book: see sections 7.2.3, 8.2.2, 9.2.4 and 9.3.3.
We will also cover some material that is not in the book.
So far we have been using frequentist (or classica