TA: Cuicui Qi
Oce: 1335C
Oce Hour: 9 AM - 10:30 AM, TR
Detailed Solution for Dis 1: Ex.4 and Ex.2.(b)
4. A man possesses ve coins, two of which are double-headed, one is double-tailed, and two are
normal. He shuts his eyes, picks a coin at random, and tos
Lecture 2: Measurable space, measure and
probability
Denition 1.2.1
A collection F of subsets of a sample space S is called a -eld (or
-algebra) iff it has the following properties:
(i) The empty set 0 F ;
/
(ii) If A F , then the complement Ac F ;
(iii)
Chapter 1: Probability Theory
Lecture 1: Sample space and subsets
Random experiment
An experiment (in a general sense) that results in one of more than
one outcomes with uncertainty in which outcome will be the realization
Examples of random experiments
T
Lecture 19: Conditional distribution and expectation
Denition 4.2.1 (conditional pmf)
Let (X1 , ., Xn ) be a discrete random vector with joint pmf f (x) and k be
an integer satisfying 1 k n 1. The conditional pmf of (Xk +1 , ., Xn )
given that (X1 , ., Xk
Lecture 18: Multivariate expectation and covariance
Multivariate expectation
The expectation of a random vector X = (X1 , ., Xn ) is dened as
E (X ) = (E (X1 ), ., E (Xn ), provided that E (Xi ) exists for any i.
When M is a matrix whose (i, j)th element
Lecture 17: Joint distributions and mgfs
Bivariate normal distribution
The bivariate normal distribution has the following pdf on R 2 :
(x 1 )
(y
)
exp 2 2 (1 2 ) + (x1 )(y 22 ) 2 22 2 )
(1 )
(1
2
f (x, y) =
1
2
1 2
21 2
2
1 2
(x, y) R 2
where 1 R, 2 R,
Lecture 14: Exponential and location-scale families
Families of Distributions
In statistics we are interested in some families of distributions, i.e.,
some collections of distributions.
For example, the family of binomial distributions with p (0, 1) and a
Lecture 15: Inequalities
Inequalities are useful for statistical theory.
Theorem 3.6.1 (Chebychevs inequality)
Let X be a random variable and let g(x) be a nonnegative function.
For any r > 0,
E [g(X )]
P(g(X ) r )
r
Proof.
Assuming that X has a pdf (if
Lecture 3: Conditional probability
Conditional probability
Sometimes we want to know the probability of event A given that
another event B has occurred.
Denition 1.3.2.
If A and B are events with P(B) > 0, then the conditional probability of
A given B is
Lecture 4: Independence
Denition 1.3.7.
Two events A and B are (statistically) independent iff
P(A B) = P(A)P(B).
This denition is still good if P(A) = 0 or P(B) = 0.
A and B are independent iff P(A|B) = P(A) (or P(B|A) = P(B).
That is, the probability of
Lecture 11: Differentiating under an integral sign
When can we switch the differentiation and integration?
If the range of the integral is nite, this switch is usually valid.
Theorem 2.4.1 (Leibnitzs rule)
If f (x, ), a( ), and b( ) are differentiable wit
Lecture 9: Moments and moment generating functions
The various moments of a random variable are an important class of
expectations.
Denition 2.3.1.
For each integer n, the nth moment of a random variable X (or FX ) is
n = E (X n ),
= 1 .
The nth central
Lecture 10: Characterizing distributions
Can the moments determine a distribution?
Can two random variables with different distributions have the same
moments of any order?
Example 2.3.10.
f1 (x) =
X1 has pdf
X2 has pdf
2
1 e (logx) /2 ,
2 x
f2 (x) = f1
Chapter 2: Transformations and Expectations
Lecture 7: Transformations
For a random variable X , we are often interested in a transformation,
Y = g(X ), which is also a random variable.
Here g is a function from the space for X to a new space.
For an even
Lecture 6: Density and mass functions
The cdfs can be used to calculate various probabilities related to
random variables.
It may be more convenient to use another function to calculate
probabilities.
Denition 1.6.1.
The probability mass function (pmf) of
Lecture 8: Expectations
The expected value, sometimes it is also called the expectation or
mean, of a random variable is its average value weighted
according to its probability distribution.
The expected value is a number that summarizes a typical, middle
Lecture 5: Random variables and their distributions
Denition 1.4.1.
A random variable is a function from S into a set of real numbers (a
subset of R).
It is convenient to deal with a summary variable than the original
sample space.
The range of a random v
Chapter 3: Common Families of Distributions
Lecture 12: Useful distributions (part I)
Binomial distribution binomial(n, p)
Let n be a positive integer and p [0, 1].
The binomial pmf with size n and probability p is
f (x) =
n
x
0
p x (1 p)nx
x = 0, 1, ., n
Lecture 26: Sums based on a random sample
Sums formed from a random sample are useful statistics.
We now study their properties.
Lemma 5.2.5.
Let X1 , ., Xn be a random sample from a population and let g(x) be a
function such that E [g(X1 ) and Var(g(X1 )
Lecture 28: Order statistics
Besides the sample mean and variance, order statistics of a random
sample form the next set of important statistics, which provide
summary information from the sample in addition to the center and
spread.
Denition 5.4.1.
The o
Chapter 6. Principles of Data Reduction
Lecture 34: Sufciency
Data reduction
We consider a sample X = (X1 , ., Xn ), n > 1, from a population of
interest (each Xi may be a vector and X may not be a random sample,
although most of the time we consider a ra
Lecture 29: Convergence concepts
Asymptotic approach
In statistical analysis or inference, a key to the success of nding a
good procedure is being able to nd some moments and/or
distributions of various statistics.
In many complicated problems we are not
Lecture 31: Central Limit Theorem
The Central Limit Theorem (CLT) is one of the most important
theorems in probability and statistics.
It derives the limiting distribution of a sequence of normalized random
variables/vectors.
Theorem 5.5.15 (Central Limit
Lecture 33: Generating a random sample
In this lecture, we study how to generate random variables from a
given distribution.
This may be useful in applications, or in statistical research when we
carry out simulation studies, or to approximate integrals t
Lecture 32: Convergence of transformations
The WLLN, SLLN, and CLT deals with sum of random
variables/vectors.
If Zn converges to Z in some sense, we often need to check whether
g(Zn ) converges to g(Z ) in the same sense.
Continuous mapping theorem.
If a