sum_w7 - SUMMARY OF WEEK 7 [STAT4610 Applied Regression...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: SUMMARY OF WEEK 7 [STAT4610 Applied Regression Analysis] Cont’d on CHAPTER 3 Cont’d on Inferences in Regression 4) Tests involving Residuals Tests for Randomness (Nonindependence of errors) RUNS Test & Durbin‐Watson Test (later!) Tests for Constancy of Variance Brown‐Forsythe & Breusch‐Pagan Tests Tests for Outliers Tests for Normality Goodness of fit tests can be used for examining the normality of the error terms (i.e., Chi‐Square or Kolmogorov‐Smirnov, Lilliefors tests) A) Brown‐ForsytheTest: not based on normality assumption of errors, but based on the variability of the residuals. To test H0: Constant Variance H1: Non‐constant variance Step 1: Divide the data set into two groups, according to the level of X so that one group consists of cases where the X level is comparatively low, the other group consists of cases where the X level is comparatively high. Step 2: Regress Y on X in the first and second group and then obtain ei1: residuals for group 1 (G1) , n1 sample size ei2: residuals for group 2 (G2), n2 sample size Step 3: Find emed1 & emed2 :medians of residuals in G1 & G2 Step 4: Calculate these distances di1=|ei1‐ emed1 | & di2=|ei2‐ emed2 |, n= n1+n2 Step 5: Calculate Test statistic : t BF * = d1 − d2 1 1 + s n1 n2 where ,pooled var iance n−2 Under H0 and n1 & n2 are NOT seriously small tBF * ~ t n-2 2 s ∑ (d = i1 − d1 ) 2 + ∑ (di2 − d2 ) 2 1 FALL 10, DR. NEDRET BILLOR| Auburn University SUMMARY OF WEEK 7 [STAT4610 Applied Regression Analysis] Step 6. Decision rule: t BF * ≤ t(1 − α / 2,n − 2) ⇒ Constant variance (Accept H0 ) t BF * > t(1 − α / 2,n − 2) ⇒ Non − Constant variance (Reject H0 ) That is, Large absolute values of t BF * then nonconstant variance B) F Test for Lack of fit of the Regression Model Assume that all the assumptions on Normality, Independence of errors, Constant variance Hold. We have Only a doubt on linearity! Requirement: replicate observations on the response Y for at least one level of X. H0: Linear Regression function is adequate. H1: Not H0 To test these hypotheses: F* = SSLOF / (m ‐ 2) ~ F 1−α ,m‐2,n‐m SSPE /(n − m) Critical region approach : If F* > F1−α ,m‐2,n‐m ⇒ Reject H0. That is, regression function is NOT linear. P ‐ value approach : P ‐ value = Pr(F > F*) < α then reject H0 . 5)Transformations to Correct Model Inadequacies Transformations are applied to accomplish certain objectives to ensure linearity, to achieve normality to stabilize the variance A) Transformations to Achieve Linearity Linearizable Simple Regression functions with corresponding Transformations Function Transformation Linear Form y= αxβ y’=logy, x’=logx y’=log(α) + βx’ y= αeβx y’=lny y’=ln(α) + βx y= α+βlog(x) x’=logx y=α + βx’ y= x / (α x – β) y’=1/y, x’=1/x y’=α – βx’ y= eα+βx /(1+ eα+βx) y’=ln(y / (1 – y)) y’=α + βx 2 FALL 10, DR. NEDRET BILLOR| Auburn University SUMMARY OF WEEK 7 [STAT4610 Applied Regression Analysis] y β>1 β =1 y y -1<β<0 β<-1 β=-1 β >0 y β <0 0<β x x i) Y=αXβ x y β >0 xx ii)xY=αeXβ α < 0 ,β <0 β >0, α > 0, x > β/ α x y 1 y β <0 y β >0, α >0, α < 0 ,β < 0 x> β/ α x iii) Y=α+βlog(x) x x iv) y= x / (α x –β) 1x v) y= eα+βx /(1+ eα+βx) Graphs for Linearizable Functions B) Transformations to stabilize variance Relationship of σ2 to E(y) Transformation Variance(σ2) ∝ constant y’=y (no transformation) σ2 ∝ E(y) y’=y1/2 ( square root: Poisson data) 2 y’=arcsin(y1/2) σ ∝ E(y) [ 1 ‐ E(y) ] (arcsin: Binomial data, 0≤yi≤1) 2 σ ∝ [E(y)]2 y’=ln(y) (logarithmic) 2 σ ∝ [E(y)]3 y’=y‐1/2 (reciprocal square root) 2 y’=y‐1 (reciprocal) σ ∝ [E(y)]4 C) Analytic Methods for Selecting Transformation Transformations ( 1/Y, square root, lnY ) so far have been chosen based on theoretical or empirical evidence to obtain linearity of the model, to achieve normality and/or stabilize error variance! All these transformation can be thought of as a general case of power transformation (more formal and more objective ) techniques. In power transformation we raise the response variable y and/or some of the predictor variables to a power. For example; use Yλ, (Y>0) 3 FALL 10, DR. NEDRET BILLOR| Auburn University SUMMARY OF WEEK 7 [STAT4610 Applied Regression Analysis] How do we choose λ? There two methods for the choice of λ: 1)Theoretical evidence 2) Empirical evidence If theoretical considerations do not provide a value for λ we use the data to determine the appropriate value of λ. Maximum Likelihood Method for choosing λ Data: (Yiλ ,Xi), i=1,2,…,n Errors ~ NID( 0, σ2 ) then Yi ~ NID( β0+ β1Xi, σ2) Likelihood function from the joint distribution of the observations Yiλ n ⎡1 ⎤ L(yiλ , x i ,β0 ,β1 , σ2 ) = ∏ (2πσ2 )−1 / 2 exp⎢− 2 (yiλ − β0 − β1x i )2 ⎥ ⎣ 2σ ⎦ i=1 2 Find the values of the parameters (β0,, β1, σ ,λ) that maximizes the likelihood function L or the Natural Logarithm of likelihood function (ln L). Steps for the numerical search for finding the value for λ: Step1: Standardize Yiλ so that the magnitude of the error sum of squares does not depend on the value of λ by using 1/ n ⎧K (Y λ − 1), λ ≠ 0 1 ⎛n ⎞ * where K 2 = ⎜ ∏ Yi ⎟ ,K 1 = Yi = ⎨ 1 i λ −1 λK 2 ⎝ i=1 ⎠ K 2 (ln Yi ), λ = 0 ⎩ Step 2: Step 2: Regress Y* (obtained for every value of λ) on the predictor , get SSE Step 3: Plot SSE(λ) or Log Likelihood value vs. λ Step 4: Read the value that minimizes SS E(λ) or that maximizes the likelihood function. For example for the following plot, the appropriate λ is ‐1/2. Scatterplot of Log vs Lambda 0 -10 -20 Log -30 -40 -50 -3 -2 -1 0 Lambda 1 2 3 4 FALL 10, DR. NEDRET BILLOR| Auburn University ...
View Full Document

Ask a homework question - tutors are online