Lecture Notes 13
Plug-In Estimators and The Bootstrap
This is mostly not in the text.
1
Introduction
Can we estimate the mean of a distribution without using a parametric model? Yes. The
key idea is to rst estimate the distribution function nonparametrica
Lecture Notes 9
Asymptotic (Large Sample) Theory
1
Review of o, O, etc.
1. an = o(1) mean an 0 as n .
P
2. A random sequence An is op (1) if An 0 as n .
P
3. A random sequence An is op (bn ) if An /bn 0 as n .
P
4. np op (1) = op (np ), so n op (1/ n) = o
Lecture Notes 15
Prediction
This is mostly not in the text. Some relevant material is in Chapters 11 and
12.
1
Introduction
We observe training data (X1 , Y1 ), . . . , (Xn , Yn ). Given a new pair (X, Y ) we want to predict
Y from X . There are two commo
Lecture Notes 7
1
Parametric Point Estimation
X1 , . . . , Xn p(x; ). Want to estimate = (1 , . . . , k ). An estimator
= n = w(X1 , . . . , Xn )
is a function of the data.
Methods:
1. Method of Moments (MOM)
2. Maximum likelihood (MLE)
3. Bayesian estim
Lecture Notes 12
Nonparametric Inference
This is not in the text.
Suppose we want to estimate something without assuming a parametric model. Some
examples are:
1. Estimate the cdf F .
2. Estimate a density function p(x).
3. Estimate a functional T (P ) of
Lecture Notes 17
Three Bonus Topics
1
Multiple Testing and Condence Intervals
Suppose we need to test many null hypotheses
H0,1 , . . . , H0,N
where N could be very large. We cannot simply test each hypotheses at level because, if
N is large, we are sure
Lecture Notes 16
Model Selection
Not in the text.
1
Introduction
Sometimes we have a set of possible models and we want to choose the best model. Model
selection methods help us choose a good model. Here are some examples.
Example 1 Suppose you use a poly
Lecture Notes 8
1
Minimax Theory
Suppose we want to estimate a parameter using data X n = (X1 , . . . , Xn ). What is the
best possible estimator = (X1 , . . . , Xn ) of ? Minimax theory provides a framework for
answering this question.
1.1
Introduction
L
Lecture Notes 14
Bayesian Inference
Relevant material is scattered throughout the book: see sections 7.2.3, 8.2.2, 9.2.4 and 9.3.3.
We will also cover some material that is not in the book.
1
Introduction
So far we have been using frequentist (or classica