This preview shows page 1. Sign up to view the full content.
Unformatted text preview: SUMMARY OF WEEK 3 [STAT4610 Applied Regression Analysis] Cont’d on CHAPTER 1 D) Normal Error Regression Model: If in the simple linear regression model εi ~NID(0, σ2 ), the simple regression model is called normal error regression model. (NID stands for Normally Independently Distributed) E) Maximum Likelihood Estimation: In OLS method the random errors are assumed to be distributed from any probability distribution. The OLS estimators would provide estimators that are unbiased and have minimum variance regardless of the type of probability distribution of errors. However the ML method requires a specific distribution to get estimators that have unbiasedness, minimum variance and other nice statistical properties such as sufficiency, consistency. ML: uses the product of the densities as the measure of consistency of the parameter value with the sample data. The product: called the likelihood value of the parameter values β0 , β1 and σ2 , is denoted by L(β0 , β1 , σ2 ). Data: (Yi,Xi), i=1,2,…,n, Errors ~ NID( 0, σ2 ) then Yi ~ NID( β0+ β1xi, σ2). Likelihood function from the joint distribution of the observations Yi n ⎡1 ⎤ L(β0 ,β1, σ 2 ) = ∏ (2πσ2 )−1 / 2 exp⎢− 2 (yi − β0 − β1xi )2 ⎥ ⎣ 2σ ⎦ i=1 Find the parameters that maximizes the likelihood function L or the Natural Logarithm of likelihood function (ln L). ∂ lnL ~ ~ ~2 = 0 ∂β0 β0 ,β1 ,σ The solution of the following system of equations : ∂ lnL ∂β1 ∂ lnL ∂σ 2
~ ~ ~2 β0 ,β1 ,σ = 0 =0 ~ ~ ~2 β0 ,β1 ,σ 1 FALL 10, DR. NEDRET BILLOR| Auburn University SUMMARY OF WEEK 3 [STAT4610 Applied Regression Analysis]
~ ~ β0 = y − β1x give the maximum likelihood estimators of β0 , β1 and σ2 : ~ Sxy
β1 = Sxx ~~
0 2 ~ σ=
2 ∑(y − β − β x )
i i=1 1i n n Statistical properties of MLEs better than LSE: • Unbiased estimators (even MLE of σ2 asymptotically unbiased!) • Minimum variance • Consistent (It means that the distributions of the estimators become more and more concentrated near the true value of the parameter being estimated, so that the probability of the estimator being arbitrarily close to the true parameter converges to 1.) • Sufficient (meaning that "no other statistic which can be calculated from the same sample provides any additional information as to the value of the parameter") . Ch 2. Inferences in Regression and Correlation Analysis Assumption: Normal Error Regression Model: Errors (εi) ~ NID(0,σ2) in Yi = β0 + β 1Xi + εi , i=1,2,…,n A) Inferences in Regression Two Important Inferential approaches on β0 & β 1 parameters 1. Confidence Intervals, 2. Hypothesis Testing ˆ ˆ For these inferential methods, we need “Sampling Distributions of β 0 and β1 . Sampling distributions of the OLS estimators of β0 & β 1 ⎛ ⎛ ⎛ 1 x2 ⎞ ⎞ 1⎞ ⎟ ˆ ˆ ⎟ and β 0 ~ N⎜ β 0 , σ ⎜ + β1 ~ N⎜ β1, σ ⎜n S ⎟⎟ ⎟ ⎜ ⎟ ⎜ S xx ⎠ xx ⎠ ⎝ ⎝ ⎝ ⎠ Sampling distributions of the standardized forms of the LSEs of the model parameters, β0, β1 are: ˆ ˆ β0 − β0 β 1 − β1 ~ N(0,1) ~ N(0,1) & 1 x2 σ 2 / S xx σ2 ( + ) n S xx 2 FALL 10, DR. NEDRET BILLOR| Auburn University SUMMARY OF WEEK 3 [STAT4610 Applied Regression Analysis] Since the σ is unknown therefore it needs to be replaced with its estimated value and the sample size is mostly small then these standardized forms of the OLS estimators become the Studentized Statistics ˆ β1 − β1 s ˆ , (Student t distributed with n‐2 degrees of freedom). ~ t (n - 2), s(β1 ) = ˆ 1) s(β S xx
2 ˆ β0 − β0 ˆ0 ) = s (1 + x ) ~ t(n − 2), s(β ˆ n S xx s(β 0 ) n SSE ( = σ) , S xx = ∑ ( x i − x)2 where s = MSE, MSE = ˆ n−2 i=1 1. 100(1‐α)% Confidence Intervals β0 & β 1 parameters ˆ ˆ ˆ ˆ β1 − t(1 - α/2; n - 2) s(β1 ) ≤ β1 ≤ β1 + t(1 - α/2; n - 2)s(β1 ) ˆ ˆ ˆ ˆ β 0 − t(1 - α/2; n - 2)s(β 0 ) ≤ β 0 ≤ β 0 + t(1 - α/2; n - 2)s(β 0 ) 2) Hypothesis Testing for β0 & β 1: a) To test H0: β1= β1*(specific value) H1: not H0 at α significance level Test statistic: ˆ ˆ β −β * β −β * If the null is true. t 0 = 1 1 = 1 1 ~ t α / 2 with df = n − 2 ˆ s(β1 ) s Sxx If p‐value=2*P(t (1 − α/2; n − 2) > ⎜t0 ⎜ ) < α (Two‐sided) conclude H1 , Otherwise conclude H0.) 3 FALL 10, DR. NEDRET BILLOR| Auburn University ...
View Full Document
This note was uploaded on 10/12/2010 for the course STAT 4630 taught by Professor Billor during the Spring '10 term at Auburn University.
- Spring '10