Due: Thursday Sept 25 by 3:00
1. Let C = A
B. Show that
sn (C) sn (A) + sn (B).
2. Let C = cfw_A B; A A, B B. Show that
sn (C) sn (A)sn (B).
3. Show that sn+m (A) sn (A)sm (A).
A = [a, b] [c, d] : a b c d .
Find VC dimension of
Lecture Notes 8
Suppose we want to estimate a parameter using data X n = (X1 , . . . , Xn ). What is the
best possible estimator = (X1 , . . . , Xn ) of ? Minimax theory provides a framework for
answering this question.
Lecture Notes 5
(Chapter 6.) A statistical model P is a collection of probability distributions (or a collection of densities). An example of a nonparametric model is
(p (x)2 dx < .
A parametric model has the form
p(x; ) :
Lecture Notes 9
Asymptotic (Large Sample) Theory
Review of o, O, etc.
1. an = o(1) mean an 0 as n .
2. A random sequence An is op (1) if An 0 as n .
3. A random sequence An is op (bn ) if An /bn 0 as n .
4. nb op (1) = op (nb ), so n op (1/ n) = o
Lecture Notes 12
See Chapters 7 and 20.
Suppose we want to estimate something without assuming a parametric model. Some
1. Estimate the cdf F .
2. Estimate a density function p(x).
3. Estimate a functional T (P ) of a
Lecture Notes 14
Relevant material is in Chapter 11.
So far we have been using frequentist (or classical) methods. In the frequentist approach,
probability is interpreted as long run frequencies. The goal of frequentist i
Lecture Notes 15
Chapters 13, 22, 20.4.
We observe training data (X1 , Y1 ), . . . , (Xn , Yn ). Given a new pair (X, Y ) we want to predict
Y from X. There are two common versions:
1. Y cfw_0, 1. This is called classication, or
Lecture Notes 16
Not in the text except for a brief mention in 13.6.
Sometimes we have a set of possible models and we want to choose the best model. Model
selection methods help us choose a good model. Here are some example
Lecture Notes 13
See Chapter 8.
Can we estimate the mean of a distribution without using a parametric model? Yes. The
key idea is to rst estimate the distribution function nonparametrically. Then we can get an
estimate of the
1. Let X1, . . . , Xn be a random sample from a Norma1(0, 62), for unknown 0 > 0.
(a) Show that T 2 (23:1 Xi, 21:1 is sufcient for 0.
(b) Find two different estimators based on T, call them 91 (T) and 92(T), that are each unbiased for
(C) Is T complet
Lecture Notes 4
Let X1 , . . . , Xn F . A statistic is any function T = g(X1 , . . . , Xn ). Recall that the sample
and sample variance is
(Xi X n )2 .
Let = E(Xi ) and = Var(Xi ).
Lecture Notes 7
Parametric Point Estimation
X1 , . . . , Xn p(x; ). Want to estimate = (1 , . . . , k ). An estimator
= n = w(X1 , . . . , Xn )
is a function of the data.
1. Method of Moments (MOM)
2. Maximum likelihood (MLE)
3. Bayesian estim
Due: Thursday October 2 by 3:00
1. Let X1 , . . . , Xn Uniform(, + 1).
(a) Find a minimal sucient statistic.
(b) Show that X3 is not a sucient statistic.
2. Chapter 6, problem 1.
3. Chapter 9, Problem 2(a), 2(b), 2(c).
4. Let X1 , . . .
Due: Thursday October 16 by 3:00
1. Let X1 , . . . , Xn Poisson(). Let = P(Xi = 0). Note that = g() for some
(a) Find the mle for .
(b) Find the limiting distribution for (appropriately normalized).
(c) Show that is consisten
Due: Thursday Sept 18 by 3:00
Do not submit homework by email. Bring a paper copy to Mari-Alice.
1. Let X1 , . . . , Xn be independent random variables. We do not assume that they are
identically distributed. Let i = E(Xi ). Assume that
Due Thursday Nov 6 by 3:00
1. (Relationship between tests and condence sets.) Let X1 , . . . , Xn p(x; ) where
(a) Let Cn be a 1 condence interval for . Consider testing
H0 : = 0
and H1 : = 0 .
Dene a test as follows: reject H0 if 0 Cn . Sh
Due: Thursday October 30 by 3:00
1. Chapter 10, problem 9.
2. Chapter 10, problem 15.
3. Chapter 10, problem 16.
4. In hypothesis testing, we often study the power of tests at local alternatives. For
example, let Y1 , . . . , Yn N (, 1).
Due Thursday Nov 13 by 3:00
1. Let X1 , . . . , Xn Bernoulli(p). Let p =
sample and let p = n n Xi .
Xi . Let X1 , . . . , Xn denote a bootstrap
(a) What is the exact distribution of np , conditional on X1 , . . . , Xn ?
There are 4 questions. CHOOSE 3. Each is worth 33 points. (You get one
bonus point for writing your name.)
Circle the 3 questions you want to be graded:
There are extra blank pages at the end of the test.
(1) Let X1 , . . . , Xn Be
Lecture Notes 1
Brief Review of Basic Probability
I assume you know basic probability. Chapters 1-3 are a review. I will assume you have
read and understood Chapters 1-3. Here is a very quick review.
A random vari
Lecture Notes 2
Inequalities are useful for bounding quantities that might otherwise be hard to compute.
They will also be used in the theory of convergence.
Theorem 1 (The Gaussian Tail Inequality) Let X N (0, 1). Then
Lecture Notes 3
Recall that, if X1 , . . . , Xn Bernoulli(p) and pn = n1
P(|pn p| > ) 2e2n .
Xi then, from Hoedings
Sometimes we want to say more than this.
Example 1 Suppose that X1 , . . . , Xn have cdf F . Let
Lecture Notes 6
The Likelihood Function
Denition. Let X n = (X1 , , Xn ) have joint density p(xn ; ) = p(x1 , . . . , xn ; ) where
. The likelihood function L : [0, ) is dened by
L() L(; xn ) = p(xn ; )
where xn is xed and varies in .
1. The likelihood
HW 1 - Solutions
Intermediate Statistics - 10-705 / 36-705
Problem 1: Wasserman 1.1 [20 points]
Case An A (increasing sequence)
We need to prove: 1) B j Bk = 0, j = k; 2) An = n Ai = n Bi and Bi = Ai .
1. It sufces to prove B j