Lecture Notes 1
36-705
Brief Review of Basic Probability
I assume you already know basic probability. Chapters 13 are a review. I will assume
you have read and understood Chapters 1-3. If not, you sho
Lecture Notes 8
1 Minimax Theory
Suppose we want to estimate a parameter 0 using data X n = (X1, . . . ,Xn). What is the
best possible estimator 5 = 3(X1, . . . ,Xn) of 0? Minimax theory provides a fr
1
X1,.
Lec_ture _Notes 7 _
Parametric Pomt Estimation
Introduction
,Xn ~ p(:c; 6). Want to estimate 6 = (61, . . . ,6k). A11 estimator
6: 6, =w(X1,.,X,)
is a function of the data. Keep in mind thar th
Lecture Notes 10
Hypothesis Testing (Chapter 10)
1 Introduction
Let X1, . . . ,Xn ~ p(:r; 0). Suppose we we want to know if 0 = 60 or not, where 00 is a specic
value of 0. For example, if we are ippin
Lecture Notes 6
The likelihood function plays an important role in statistical inference. In these notes we
dene the likelihood function. This function Will be used later for many different tasks.
1 T
Lecture Notes 2
1 Probability Inequalities
Inequalities are useful for bounding quantities that might otherwise be hard to compute.
They will also be used in the theory of convergence.
Theorem 1 (The
Lecture Notes 4
Convergence (Chapter 5)
1 Random Samples
Let X1, . . . ,Xn N F. A statistic is any function Tn = g(X1, . . . ,Xn). Recall that the sample
mean is
and sample variance is
Let a = lE(Xi
Lecture Notes 3
Uniform Bounds
1 Introduction
Recall that, if X1,.,X, ~ Bernoulli(p) and fin = n1 Z?=1Xi then, from Hoeffdings
inequality, 2
P016), p| > e) S 2e2E .
Sometimes we want to say more than
Lecture Notes 5
1 Statistical Models
(Chapter 6.) A statistical model 73 is a collection of probability distributions (or a collec
tion of densities). Examples of nonparametric models are
'P = cfw_p :
Lecture Notes 4
Convergence (Chapter 5)
1
Random Samples
Let X1 , . . . , Xn F . A statistic is any function Tn = g(X1 , . . . , Xn ). Recall that the sample
mean is
n
1
Xi
Xn =
n i=1
and sample varia
Lecture Notes 1
Brief Review of Basic Probability
1
Probability Review
I assume you know basic probability. Chapters 1-3 are a review. I will assume you have
read and understood Chapters 1-3. If not,
Lecture Notes 2
1
Probability Inequalities
Inequalities are useful for bounding quantities that might otherwise be hard to compute.
They will also be used in the theory of convergence.
Theorem 1 (The
Lecture Notes 3
1
Uniform Bounds
n
i=1
Recall that, if X1 , . . . , Xn Bernoulli(p) and pn = n1
inequality,
2
P(|pn p| > ) 2e2n .
Xi then, from Hoedings
Sometimes we want to say more than this.
Exampl
Lecture Notes 9
Asymptotic Theory (Chapter 9)
In these notes we look at the large sample properties of estimators, especially the maxi
mum likelihood estimator.
Some Notation: Recall that
mac 2 / g<x>