Problems: Assignment 3
3. Suppose cfw_Xij ; j = 1, . . . , ni ; i = 1, . . . , p are independent N(i , 2 )
random variables. (This is the usual set-up for the one-way layout.)
(a) Find the MLEs for i and .
The log like
Notes on an introduction to inference
Denition: A model is a family cfw_P ; of possible distributions for
some random variable X. (Our data set is X, so X will generally be a big
vector or matrix or even more complicated object.)
We will assume
Independence, conditional distributions and
When analyzing data statisticians need to specify a statistical model for
the data. That is, we regard the data as random variables and specify possible joint distributions for the data. Somet
Convergence in Distribution
In the previous chapter I showed you examples in which we worked out
precisely the distribution of some statistics. Usually this is not possible.
Instead we are reduced to approximation. One method, nowadays likely
Comparing modes of convergence
Think about a sequence Xn and a possible limit X:
Xn converges in distribution to X depends only on the marginal distributions of individual Xn and X.
Convergence in probability and pth mean depends only on the seque
Denition: A level condence set for a parameter () is a random subset,
C, of the set of possible values of such that for each
P () C)
Condence sets are very closely connected with hypothesis tests:
From condence sets to tests
Suppose C is a level = 1 con
Bayesian Point Estimation
In this section I will focus on the problem of estimation of a 1 dimensional parameter, . Earlier we discussed comparing estimators in terms of
Mean Squared Error. In the language of decision theory Mean Squared Error
Decision Theory and Bayesian Methods
Example: Decide between 4 modes of transportation to work:
B = Ride my bike.
C = Take the car.
T = Use public transit.
H = Stay home.
Costs depend on weather: R = Rain or S = Sun.
Ingredients of Decision P
Intuition about convergences
I want to contrast two statements:
1. X and Y are close together.
2. X and Y have similar distributions.
The truth of the rst statement depends on the joint distribution of X
and Y . On the other hand, the truth of the s
Hypothesis testing is a statistical problem where you must choose, on the
basis of data X, between two alternatives. We formalize this as the problem
of choosing between two hypotheses: Ho : 0 or H1 : 1 where 0
and 1 are a part
Statistics versus Probability
I want to begin this course by discussing the dierence between Probability Theory and Statistics. Statisticians use the tools of Probability but
reason from eects to causes rather than from causes to eects. I want to
The ideas of the previous sections can be used to prove the basic sampling
theory results for the normal family. Here is the theorem which describes the
distribution theory of the most important statistics.
Theorem 1 Suppose X1 , . . . , Xn are independen
Problems: Assignment 4
1. From the text page 147 # 6.
The parameter of interest is
= P (X > 0) = P (X > ) = P (N(0, 1) < ) = ()
so the mle is
The estimated standard error is
| |/ n = (X)/ n.
Averages converge to their expected values and
Likelihood Methods of Inference
Imagine we toss a coin 6 times and get Heads twice. Let p be the probability of getting H on an individual toss and suppose the tosses are independent.
Then the probability of getting exactly 2 heads is
15p2 (1 p)4
Moment Generating Functions
There are many uses of generating functions in mathematics. We often study
the properties of a sequence an of numbers by creating the function
an s n
In statistics the most commonly used generating functions are the p
Likelihood Ratio tests
For general composite hypotheses optimality theory is not usually successful in producing an optimal test. instead we look for heuristics to guide
our choices. The simplest approach is to consider the likelihood ratio
The Multivariate Normal Distribution
In this section I present the basics of the multivariate normal distribution as
an example to illustrate our distribution theory ideas.
Denition: A random variable Z R1 has a standard normal distribution
Large Sample Theory
We can study the approximate behaviour of by studying the function U .
Notice rst that U is a sum of independent random variables and remember
the law of large numbers:
Theorem 1 If Y1 , Y2 , . . . are iid with mean then
The basics of nonparametric models
The Empirical Distribution Function EDF
The most common interpretation of probability is that the probability of
an event is the long run relative frequency of that event when the basic
experiment is repeated ov
In this section I want to dene the basic objects. I am going to give full
precise denitions and make lists of various properties even prove some
things rigorously but then I am going to give examples. In dierent versions of thi
The basic problem of distribution is to compute the distribution of statistics when the data come from some model. You start with assumptions about
the density f or the cumulative distribution function F of some random vector X = (X1 , . . . , Xp ); typic
Expectation and Moments
I begin by reviewing the usual undergraduate denitions of expected
value. For absolutely continuous random variables X we usually say:
Denition: If X has density f then
g(x)f (x) dx .
For discrete random variab