Lecture Notes 2
1
Probability Inequalities
Inequalities are useful for bounding quantities that might otherwise be hard to compute.
They will also be used in the theory of convergence.
Theorem 1 (The Gaussian Tail Inequality) Let X N (0, 1). Then
2 /2
2e

Lecture Notes 2
1
Probability Inequalities
Inequalities are useful for bounding quantities that might otherwise be hard to compute.
They will also be used in the theory of convergence.
Theorem 1 (The Gaussian Tail Inequality) Let X N (0, 1). Then
P(|X| >

CR12: Statistical Learning & Applications
Johnson-Lindenstrauss theory
Lecturer: Joseph Salmon
1
Scribes: Jordan Frecon and Thomas Sibut-Pinote
Subgaussian random variables
In probability, Gaussian random variables are the easiest and most commonly used d

Math 461 B/C, Spring 2009
Midterm Exam 3 Solutions and Comments
1. Let X have normal distribution with mean 1 and variance 4.
10 pts
(a) Find P (|X| 1).
Solution.
P (|X| 1) = P (1 X 1)
X 1
1 1
11
2
2
2
= (0) (1) = (0) (1 (1)
=P
= 0.5 (1 0.84) = 0.34
10 pt

Mathematics 4255
Midterm 2 with solutions
April 2nd, 2009
Partial credit will be awarded for your answers, so it is to your advantage to explain your
reasoning and what theorems you are using when you write your solutions. Please answer
the questions in t

Homework 10 - Solutions
Intermediate Statistics - 10705/36705
Problem 1. Recall that:
E[|Y g(X)|] = E cfw_E[|Y g(X)| | X]
The idea is to choose c such that E[|Y c| | X = x] is minimized. Now dene:
r(c) = E[|Y c| | X = x] =
|y c|pY |X=x (y)dy
The function

Homework 2 - Solutions
Intermediate Statistics - 36-705
September 21, 2014
Problem 1. For convenience, let Yi = Xi i . We proceed as in the proof of Hoedings
inequality. For all c > 0:
n
Yi nt
P
=P e
n
i=1
Yi
ent =
i=1
= P ec
n
i=1
Yi
ecnt ecnt E ec
n
i

36-705 Intermediate Statistics HW1
Problem 2.31
Since the mgf is dened as MX (t) = EetX , we necessarily have MX (0) = Ee0 = 1. But
t/(1 t) is 0 at t = 0, therefore it cannot be an mgf.
Problem 2.36
Problem 2.38
(a)
r+x1 r
p (1 p)x etx
x
MX (t) =
x=0
= pr

Homework 7 - Solutions
Intermediate Statistics - 10705/36705
Problem 1. [20 points]
The statistic Z can be rewritten as:
Z=
1 1 0
+
se
se
When the true = 1 > 0 we have that for large n, Z N
of the test is given by:
1 0
,1
se
, such that the power
(1 ) =

Homework 3 - Solutions
Intermediate Statistics - 36-705
September 30, 2014
Problem 1. [10 points]
Let F be a nite set of n elements. We have that C = cfw_A : A A or A B, so if C picks
out G F , then either A picks out G or B picks out G. Thus, for any F F

Lecture Notes 4
Convergence (Chapter 5)
1
Random Samples
Let X1 , . . . , Xn F . A statistic is any function Tn = g(X1 , . . . , Xn ). Recall that the sample
mean is
n
1X
Xi
Xn =
n i=1
and sample variance is
n
Sn2
1 X
(Xi X n )2 .
=
n 1 i=1
Let = E(Xi ) a

Lecture Notes 5
1
Statistical Models
(Chapter 6.) A statistical model P is a collection of probability distributions (or a collection of densities). Examples of nonparametric models are
(
)
(
)
Z
P= p:
(p00 (x)2 dx < , and P = all distributions on Rd .
A

Lecture Notes 1
36-705
Brief Review of Basic Probability
I assume you already know basic probability. Chapters 1-3 are a review. I will assume
you have read and understood Chapters 1-3. If not, you should be in 36-700.
1
Random Variables
Let be a sample s

Lecture Notes 3
Uniform Bounds
1
Introduction
Recall that, if X1 , . . . , Xn Bernoulli(p) and pbn = n1
inequality,
2
P(|b
pn p| > ) 2e2n .
Pn
i=1
Xi then, from Hoeffdings
Sometimes we want to say more than this.
Example 1 Suppose that X1 , . . . , Xn hav

Homework 2
36-705
Due: Thursday Sept 15 by 3:00
1. Let X have mean 0. We say that X is sub-Gaussian if there exists > 0 such that
t2 2
log E[etX ]
2
for all t.
(i) Show that X is sub-Gaussian if and only if X is sub-Gaussian.
(ii) Let X have mean . Supp

Lecture Notes 1
Brief Review of Basic Probability
1
Probability Review
I assume you know basic probability. Chapters 1-3 are a review. I will assume you have
read and understood Chapters 1-3. If not, you should be in 36-700.
1.1
Random Variables
A random