This preview shows pages 1–8. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Stat 5102 Lecture Slides Deck 6 Charles J. Geyer School of Statistics University of Minnesota 1 The GaussMarkov Theorem Suppose we do not want to assume the response vector is normal (conditionally given covariates that are random). What then? One justification for still using least squares estimators (LSE), no longer MLE when normality is not assumed, is the following. Theorem (GaussMarkov). Suppose Y has mean vector and variance matrix 2 I , and suppose = M , where M has full rank. Then the LSE = ( M T M ) 1 M T Y is the best linear unbiased estimator (BLUE) of , where best means var( a T ) var( a T ) , for all a R p where is any other linear and unbiased estimator. 2 The GaussMarkov Theorem (cont.) We do not assume normality. We do assume the same first and second moments of Y as in the linear model. We get the conclusion that the LSE are BLUE, rather than MLE. They cant be MLE because we dont have a statistical model, having specified only moments, not distributions, so there is no likelihood. By the definition of best all linear functions of are also BLUE. This includes = M and new = M new . 3 The GaussMarkov Theorem (cont.) Proof of GaussMarkov Theorem. The condition that be linear and unbiased is = AY for some matrix A satisfying E ( ) = A = AM = for all . Hence, if AM is full rank, then AM = I . It simplifies the proof if we define B = A ( M T M ) 1 M T so = + BY and BM = 0. 4 The GaussMarkov Theorem (cont.) For any vector a var( a T ) = var( a T ) + var( a T BY ) + 2cov( a T , a T BY ) If the covariance here is zero, that proves the theorem. Hence it only remains to prove that. cov( a T , a T BY ) = a T ( M T M ) 1 M T var( Y ) B T a = 2 a T ( M T M ) 1 M T B T a is zero because BM = 0 hence M T B T = 0. And that finishes the proof of the theorem. 5 The GaussMarkov Theorem (cont.) Criticism of the theorem. The conclusion that LSE are BLUE can seem to say more than it actually says. It doesnt say the LSE are the best estimators. It only says they are best among linear and unbiased estimates. Presumably there are better esti mators that are either biased or nonlinear. Otherwise a stronger theorem could be proved. The GaussMarkov theorem drops the assumption of exact nor mality, but it keeps the assumption that the mean specification = M is correct. When this assumption is false, the LSE are not unbiased. More on this later. Not specifying a model, the assumptions of the GaussMarkov theorem do not lead to confidence intervals or hypothesis tests. 6 Bernoulli Response Suppose the data vector Y has independent Bernoulli compo nents....
View
Full
Document
This note was uploaded on 02/07/2012 for the course STAT 5102 taught by Professor Staff during the Spring '03 term at Minnesota.
 Spring '03
 Staff
 Statistics

Click to edit the document details