1
INTRODUCTION
Two key theorems in probability and statistics are the law of large
numbers and the central limit theorem.
Theorem 1.1 (Strong law of large numbers) Let X1 , X2 , . . . be independent and identically distributed random variables with E Xi <
5.9 exercies
5.9
exercies
1. In class we had worked out the asymptotics of the Mann-Whitney
statistic using the functional/nonparametric delta method. There
was one piece of the puzzle that still needed to be worked out
- this question is designed to ll i
40
estimating statistical functionals of the cdf
5.3
metrics on distributions
Our overall goal here is to study the asymptotics of statistical functionals, and to do this, we will need notions of continuity and differentiability (leading to a delta method
32
estimating the cumulative distribution function
4.6
types of confidence intervals/bands
Let F be a class of distribution functions F and let be some quantity
of interest, such as the mean of F or the whole function F . Let Cn be
a set of possible value
20
introduction to stochastic processes
3.5.1
Donskers theorem
We will now prove that the re-scaled random walk converges weakly
to Brownian motion. Let X1 , . . . , Xn be IID random variables where
each is equal to 1 with equal probability. Let Sn = X1 +
46
estimating statistical functionals of the cdf
5.8
exercies
1. Suppose that Xn , Yn is a sequence of real-valued random variables such that Xn X and Yn Y. Assume also that for each
n, (Xn , Yn ) are independent. Show that (Xn , Yn ) (X, Y).
2. Let U(t),
3.4 weak convergence on polish spaces
3.4
weak convergence on polish spaces
Let X be a mapping from a probability space (, F, P) into a Polish
space X . In particular, X is a metric space with some metric, and we
denote it as . If X1 (B) F for all Borel s
3
INTRODUCTION TO STOCHASTIC PROCESSES
3.1
definition and existence
A stochastic process is dened to be a collection of real-valued random variables cfw_Xt , t T . If the set T is nite, then were really
dealing with a random vector, and we know how to han
4
E S T I M AT I N G T H E C U M U L AT I V E D I S T R I B U T I O N
FUNCTION
4.1
the empirical distribution as the nonparametric
mle
The maximum likelihood method has been a cornerstone of statistical methodology for many years. The maximum likelihood e
8
some review of probability
2.4
our favourite, the gaussian
The normal distribution is central to much of statistical analysis, so
lets review some of its main properties. A random vector Y = (Y1 , . . . , Yn )T
is (jointly) normal with mean zero if ther
2
SOME REVIEW OF PROBABILITY
2.1
probability and random variables
A probability space is a triple (, F, P), where is the state space, F
is the associated -algebra, and P is the probability itself. Recall that
a -algebra is a collections of subsets (of ) s
7.7 penalized likelihood and scad: proof of theorems
7.7
penalized likelihood and scad: proof of theorems
7.8
exercises
1. Suppose that we observe a sample X1 , . . . , Xn IID from the normal distribution with mean and 2 (both unknown). Find the
MLE of an
MATH 6621: ADVANCED THEORETICAL STATISTICS - WINTER 2013
This course is a continuation of MATH 6620. The goal will be to go over some theoretical techniques
frequently used in statistics, while considering a variety of topics. The course will begin with s
6
E S T I M AT I N G A D E N S I T Y N O N PA R A M E T R I C A L LY
6.1
the histogram
6.2
exercises
1. Let fn denote the histogram density estimator. Show that for
x Bk
var(fn (x) =
pk (1 pk )
.
nh2
2. Consider Theorem 6.11 in Wasserman (2006, Chapter 6,
54
variable selection using the lasso
7.5
penalized likelihood and scad
7.6
exercises
1. Prove equality (2.2) in Fan and Li (2001, page 1349).
2. Prove (2.5) in Fan and Li (2001, page 1349). Repeat the same
calculation for the LASSO and ridge regression.
50
estimating a density nonparametrically
6.5
exercises
1. Prove Theorem 6.28 on page 133 from Wasserman (2006). You
may assume that K is symmetric like I did! Make sure that
you can clearly and explicitly describe the difference between
this result for t
7
VA R I A B L E S E L E C T I O N U S I N G T H E L A S S O
7.1
introduction to the lasso
Review. Overview based on Tibshirani (1996).
7.2
exercises
1. Show that the Lasso is also a Bayes MAP estimator in the
Gaussian model under an appropriately chosen