Sampling Distributions Recall the general mean-variance speci cation E(Y |x) = f (x, ), var(Y |x) = 2g( , , x)2. Closed form estimators with exactly known sampling distributions exist only in special cases, principally the linear model f (x, ) = xT with Gaussian errors and known variances. Otherwise, we must use large sample approximations. 1 Issues: Analogs of the unbiasedness and minimum variance properties. Large sample approximations that do not depend on speci c distributions for the errors. Consequences of mis-speci cation of the variances. 2 Review of Large Sample Tools a.s. De nition: Almost sure convergence: Yn Y i P lim Yn = Y n = 1. De nition: Convergence in probability: Yn Y i &gt; 0, n p lim P(|Yn Y | &lt; ) = 1. Yn Y Yn Y but not conversely. a.s. p 3 If h( ) is continuous, then Yn Y h(Yn) h(Y ) Yn Y h(Yn) h(Y ) p p a.s. a.s. Terminology: if n is an estimator from a sample of size n and 0 is the true value of the parameter, then we have two de nitions of consistency: Strong consistency: n 0; Weak consistency: n 0. 4 a.s. p Op and op: Yn is bounded in probability i : &gt; 0 M such that n, P(||Yn|| &lt; M ) &gt; 1 . De nitions: If an is a sequence of positive constants: Yn = Op(an) ( of order at most an in probability ) i : Yn is bounded in probability. an Yn = op(an) ( of smaller order than an in probability ) i : Yn p 0. an 5 Notes: Yn is bounded in probability (i.e., Op(1)) i we can nd a proper cumulative distribution function F ( ) such that n, P(||Yn|| &lt; y ) &gt; F (y). an is often, but not always, of the form nk , typically with k &lt; 0. 6 Convergence: L De nition: Convergence in distribution (or law ): Yn Y i for each continuity point y of F ( ), n lim Fn(y) = F (y). Note: Yn Y Yn Y but, in general, not conversely. L p L Special (and trivial) case: if Yn y where y is a constant, p then Yn y. 7 <a href="/keyword/asymptotic-normality/" >asymptotic normality</a> : If we can nd sequences {an} and {cn &gt; 0} such that cn (Yn an) N (0, 1) we say that Yn is asymptotically normal with asymptotic mean an and asymptotic variance c 2. n L We write 1 Yn N an, 2 . cn 8 Central Limit Theorem: Zj are independent with E Zj = j , var Zj = j . The variance matrices satisfy 1 lim ( 1 + 2 + + n) = . n n The tails of the distributions of Zj satisfy the Lindeberg condition: &gt; 0, 1 n E 1{||Z || n}||Zj j ||2 0 as n . j j n j=1 9 Then 1 n L Zj j N (0, ). n j=1 In terms of <a href="/keyword/asymptotic-normality/" >asymptotic normality</a> : if 1 n Zn = Zj n j=1 and 1 n j , n = n j=1 then 1 Zn N n, . n 10 More general CLT: If we also write 1 n n = j , n j=1 the variance matrix condition can be written n lim n = . A more general CLT does not require this convergence. 11 The Lindeberg condition becomes: &gt; 0, 1 n E 1{||Z || n n}||Zj j ||2 0 as n , j j n n j=1 where n is the smallest eigenvalue of n. Under only this modi ed Lindeberg condition, 1 n N n, n . Z n 12 In terms of convergence in distribution: L Cn Zn n N (0, I) 1 where Cn is any inverse square root of n n: Cn 1 n CT = I. n n 13 Slutsky s Theorem If Yn Y and Vn c, a constant, then: Yn + Vn Y + c YnVn cY Yn/Vn Y /c. Multivariate version: If Yn Y and Vn C, a constant matrix, then: L p L L L L p Yn + Vn Y + C VnYn CY 1 Vn Yn C 1Y. 14 L L L Weak Law of Large Numbers {Zj } are uncorrelated and {aj } are constants. If 1 1 n 2 var aj Zj = 2 a var Zj 0 as n , n j=1 n j=1 j then 1 n 1 n p aj Zj aj E Zj 0. n j=1 n j=1 n 15 How do we use all this? Suppose that some estimator satis es n ( 0) = A 1Cn + op(1), n where: An satis es the WLLN, and An C; Cn satis es the CLT, and is asymptotically normal with zero mean. p Then N ( 0, n) for some asymptotic variance matrix n, typically of the form n 1 . 16 Comparing es...