{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

# s6 - Stat 5102 Lecture Slides Deck 6 Charles J Geyer School...

This preview shows pages 1–8. Sign up to view the full content.

Stat 5102 Lecture Slides Deck 6 Charles J. Geyer School of Statistics University of Minnesota 1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
The Gauss-Markov Theorem Suppose we do not want to assume the response vector is normal (conditionally given covariates that are random). What then? One justification for still using least squares estimators (LSE), no longer MLE when normality is not assumed, is the following. Theorem (Gauss-Markov). Suppose Y has mean vector μ and variance matrix σ 2 I , and suppose μ = M β , where M has full rank. Then the LSE ˆ β = ( M T M ) - 1 M T Y is the best linear unbiased estimator (BLUE) of β , where “best” means var( a T ˆ β ) var( a T ˜ β ) , for all a R p where ˜ β is any other linear and unbiased estimator. 2
The Gauss-Markov Theorem (cont.) We do not assume normality. We do assume the same first and second moments of Y as in the linear model. We get the conclusion that the LSE are BLUE, rather than MLE. They can’t be MLE because we don’t have a statistical model, having specified only moments, not distributions, so there is no likelihood. By the definition of “best” all linear functions of ˆ β are also BLUE. This includes ˆ μ = M ˆ β and ˆ μ new = M new ˆ β . 3

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
The Gauss-Markov Theorem (cont.) Proof of Gauss-Markov Theorem. The condition that ˜ β be linear and unbiased is ˜ β = AY for some matrix A satisfying E ( ˜ β ) = A μ = AM β = β for all β . Hence, if AM is full rank, then AM = I . It simplifies the proof if we define B = A - ( M T M ) - 1 M T so ˜ β = ˆ β + BY and BM = 0. 4
The Gauss-Markov Theorem (cont.) For any vector a var( a T ˜ β ) = var( a T ˆ β ) + var( a T BY ) + 2 cov( a T ˆ β , a T BY ) If the covariance here is zero, that proves the theorem. Hence it only remains to prove that. cov( a T ˆ β , a T BY ) = a T ( M T M ) - 1 M T var( Y ) B T a = σ 2 a T ( M T M ) - 1 M T B T a is zero because BM = 0 hence M T B T = 0. And that finishes the proof of the theorem. 5

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
The Gauss-Markov Theorem (cont.) Criticism of the theorem. The conclusion that LSE are BLUE can seem to say more than it actually says. It doesn’t say the LSE are the best estimators. It only says they are best among linear and unbiased estimates. Presumably there are better esti- mators that are either biased or nonlinear. Otherwise a stronger theorem could be proved. The Gauss-Markov theorem drops the assumption of exact nor- mality, but it keeps the assumption that the mean specification μ = M β is correct. When this assumption is false, the LSE are not unbiased. More on this later. Not specifying a model, the assumptions of the Gauss-Markov theorem do not lead to confidence intervals or hypothesis tests. 6
Bernoulli Response Suppose the data vector Y has independent Bernoulli compo- nents. The assumption μ = M β now seems absurd, because E ( Y i ) = Pr( Y i = 1) is between zero and one, and linear functions are not constrained this way. Moreover var( Y i ) = Pr( Y i = 1) Pr( Y i = 0) so we cannot have constant variance var( Y ) = σ 2 I .

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}