Likelihood Methods of Inference
Suppose you toss a coin 6 times and get Heads twice. If p is the probability of getting
H then the probability of getting 2 heads is
15p2(1p)4
This probability, thought of as a function of p, is the likelihood function for
STAT 445 Chapter 2.2 The Shape of Bivariate Normal
Distribution
Multivariate normal random vector: X
1
f (x) =
2
1
=
(2)
p/2

exp[
1/2
1
exp[
1
2
2
1
(x )
1
(x )
For the bivariate case, x
=
. The pdf is
Np (, )
p
(x )],
x R
(x )],
x1
( x )
2
and
=
Chapter 1: Multivariate Random Variables
This chapter aims to prepare basic concepts, notation and software information used for the subsequent modules.
1.1 RANDOM VARIABLE VECTOR
Multivariate random variable
X = (X1 , X2 , . . . , Xp )0
where p is the nu
Chapter 2.2 The Shape of Bivariate Normal Distribution
Probability density contour:
cfw_x : (x )0 1 (x ) = c2 = ellipsoid centered at
[
]
1 0.6
0.6 1
=
[
]
1 0.9
0.9 1
2
2
=
4
4
6
Probability Density Contour
6
Probability Density Contour
0.1
0.2
0.3
0
Chapter 2: Multivariate Normal Distribution
This chapter covers basic concepts and properties of multivariate normal
(MV) distribution, a core distribution used in the theory and methods of
multivariate analysis.
2.1 MULTIVARIATE NORMAL DENSITY
Univariate
Finding (good) preliminary Point Estimates
Method of Moments
Basic strategy: set sample moments equal to population moments and solve for the
parameters.
Gamma Example
The Gamma(
) density is
and has
and
This gives the equations
or
Divide the second equat
Monte Carlo
Suppose you have some random variables
whose joint density f (or
distribution) is specified and a statistic
to know. To compute something like P(T > t):
whose distribution you want
1.
Generate
from the density f.
Compute
.
2.
3.
Repeat N times
Last time defined moments, moment generating functions, cumulants and cumulant
generating functions. We established the following relations between moments and
cumulants:
Example: I am having you derive the moment and cumulant generating function and
all
If
are independent then the log likelihood is of the form
The score function is
The mle
maximizes . If the maximum occurs at the interior of the parameter space
and the log likelihood continuously differentiable then
equations
solves the likelihood
Exampl
Hypothesis Testing
Jargon defined so far: hypothesis, power function, level, critical region, null
hypothesis, simple hypothesis, Type I error, Type II error.
Simple versus Simple testing
Theorem: For each fixed
the quantity
is minimized by any
which has
Large Sample Theory of the MLE
Application of the law of large numbers to the likelihood function:
The log likelihood ratio for
where
to
is
. We proved that
. Then
so
Theorem: In regular problems the mle
is consistent.
Now let us study the shape of the lo
Last time derived the Fourier inversion formula
and the moment generating function inversion formula
(where the limits of integration indicate a contour integral up the imaginary axis.) The
methods of complex variables permit this path to be replaced by a
Large Sample Theory of the MLE
Theorem: Under suitable regularity conditions there is a unique consistent root
the likelihood equations. This root has the property
of
In general (not just iid cases)
where
is the socalled observed information, the negativ
Framework:
are iid with mean 0 and variance 1
is the characteristic function of a single X
We concluded
We differentiated
to obtain
It now follows that
Apply the Fourier inversion formula to deduce
which is the standard normal random density.
This proof o