University Washington Math5061 (Fall 2006)
1
Review
1. Basic probability theory (Chapter 1, LC) pdf, cdf, marginal, conditional, joint Bayes theorem mgf change-of-variable formula how to obtain moments from mgf properties: mgf for ax + b and x + y
convergence in distribution, convergence in probability (1.8, LC) in probability in distribution hold for continuous function For an estimator, in probability consistent MSE 0 consistent Slutskys theorem
law of large numbers, central limit theorem exponential family (1.5 LC) canonical form, natural parameter, natural sucient statistics full-rank exponential family, curved exponential family mean and variance of the natural sucient statistics, E = A , V = A applications of Basus theorem
location-scale family (1.4 LC)
some property of normal distribution
2 2 2 2 N (1 , 1 ) + N (2 , 2 ) = N (1 + 2 , 1 + 2 + 21 2 ) N (0, 1)2 = 2 X Np (, ) AX + b (A + b, AAT )
2. (Minimal) sucient, ancillary, and complete statistics (1.6 LC) Factorization theorem
How to nd minimal sucient statistics? complete statistic
minimal sucient statistics of full-rank exponential family complete sucient statistics of full-rank exponential family
Basus theorem: complete sucient stat ancillary stat
Washington University Math5061 (Fall 2006) 3. Unbiased estimation (Chapter 2) UMVU estimator (2.1, LC)
2
Rao-Blackwell theorem: improve unbiased estimators by conditioning on sucient statistic (2.1, LC) UMVUE must be a function of a complete sucient statistic if such a statistic exists (full-rank exponential family) (2.1, LC) How to nd a UMVUE when a complete sucient statistic exists? (2.1, LC) Directly solve the equation E ((T )) = g(). Conditioning an unbiased estimator on T application of Basus theorem for location-scale family (page 94, LC)
Cramer-Rao lower bound inequality) (information (2.5)
Fisher information I() = V ar(l ()) = E(l ()2 ) = E(l ()) What is the information for h()? What is the information in n i.i.d. observations? Fisher information for location-scale family Fisher information for = E(T ) in exponential family lower bound: var() ()2 /I(). If is unbiased, var() 1/I() Attainment of the bound for exponential family Another way to nd a UMVUE: an unbiased estimator that attains the bound
4. Equivariant estimation Pitmans estimator 5. Method of moments estimator easy to construct Often consistent 6. Least square estimators 7. Maximum likelihood estimator Consistency (6.3, LC)
Delta method (6.3, LC) Computation
Asymptotically normal and ecient (6.3, LC)
Newton-Ralphson (page 453, LC)
Washington University Math5061 (Fall 2006) Fisher scoring EM algorithm (page 457, LC) one-step estimator (page 454, LC) n-consistent 8. M-estimator (estimating equation) (page 483, LC) Fisher-consistent
3
How to derive the asymptotic normality?
Taylor expansion Inuence function asymptotic normality of sample quantiles (e.g. ...