# ie_Slide03 - Introductory Econometrics ECON2206/ECON3209...

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Introductory Econometrics ECON2206/ECON3209 Slides03 Lecturer: Minxian Yang ie_Slides03 my, School of Economics, UNSW 1 3. Multiple Regression Model: Estimation (Ch3) 3. Multiple Regression Model: Estimation • Lecture plan (largely parallel to Ch2) – – – – – – – – Motivation and definitions ZCM assumption Estimation method: OLS Mechanics of OLS Underlying assumptions of multiple regression model Expected values and variances of OLS estimators Omitted and irrelevant variables Gauss-Markov theorem ie_Slides03 my, School of Economics, UNSW 2 3. Multiple Regression Model: Estimation (Ch3) • Motivation – Example 1. (control observable factors) wage = β0 + β1educ + error, where error represents (or contains) exper. • exper is likely related to educ. • The ceteris paribus effect of educ on wage cannot be properly estimated in this model. Why? If exper is available, then we can “hold expr fixed” in wage = β0 + β1educ + β2exper + u, where wage is explained by both educ and expr. • β1 and β2 measure ceteris paribus effects, properly estimable if u is not “related” to educ and expr. ie_Slides03 my, School of Economics, UNSW 3 3. Multiple Regression Model: Estimation (Ch3) • Motivation – Example 2. (flexible functional form) wage = β0 + β1age + β2age2 +...+ u, where wage may increase initially and decrease eventually as age increases. – In general, regression models with multiple x’s have the following merits. They • allow us to explicitly control for (hold fixed) many factors that affect the dependent variable, in order to draw ceteris paribus conclusions; • provide better explanation of the dependent variable by accommodating flexible functional forms. ie_Slides03 my, School of Economics, UNSW 4 3. Multiple Regression Model: Estimation (Ch3) • Multiple regression model – Definition y = β0 + β1x1 +...+ βkxk + u , • y : dependent variable (observable) • x1, ..., xk : independent variables (observable) • β1, ..., βk : slope parameters, “partial effect”, (to be estimated) • β0 : intercept parameter (to be estimated) • u : error term or disturbance (unobservable) • k : the number of independent variables – The disturbance u represents factors other than x’s. – With the intercept β0, the unconditional mean of u can always be set to zero: E(u) = 0 . ie_Slides03 my, School of Economics, UNSW 5 3. Multiple Regression Model: Estimation (Ch3) • Zero conditional mean assumption – The zero-conditional-mean (ZCM) assumption is E(u | x1, ..., xk) = 0 , for the multiple regression model. • It requires the average of u to be the same irrespective of the values of x’s. • It implies that the factors in u are uncorrelated with x1, ..., xk. • It is a key condition for the OLS estimators being unbiased. • It defines the population regression function (PRF) E(y | x1, ..., xk) = β0 + β1x1 +...+ βkxk . ie_Slides03 my, School of Economics, UNSW 6 3. Multiple Regression Model: Estimation (Ch3) • Observations on (x1, ..., xk, y) – A random sample is a set of independent observations {(xi1, xi2 , ... , xik , yi), i = 1,2,...,n}. the i-th observation on the 2nd independent variable – At observation level, the model may be written as yi = β0 + β1xi1 + ... + βkxik + ui , i = 1, 2, ..., n where i is the observation index. – Or collectively, y1 1 x11 x1k β 0 u1 y 1 x x β u 2k 1 2 = 21 + 2 . y n 1 x n1 x nk β k un – Matrix notation: ie_Slides03 Y= X my, School of Economics, UNSW B + U. 7 3. Multiple Regression Model: Estimation (Ch3) • Estimate multiple regression – The model: yi = β0 + β1xi1 + ... + βkxik + ui , i = 1, 2, ..., n. ˆˆ ˆ – Let ( β 0 , β1,..., β k ) estimate (β0 , β1, ..., βk). – Corresponding residual is ˆ ˆ ˆ ˆ ui = y i − β 0 − β1 x i 1 − − β k x ik , i = 1,2,..., n. – The sum of squared residuals (SSR) n n i =1 i =1 ˆ ˆ ˆ ˆ SSR = ∑ ui2 = ∑ ( y i − β 0 − β1 x i 1 − − β k x ik )2 indicates the goodness of the estimates. – Good estimates should make SSR small. ie_Slides03 my, School of Economics, UNSW 8 3. Multiple Regression Model: Estimation (Ch3) • Ordinary least squares (OLS) ˆˆ ˆ – The OLS estimates( β 0 , β1,..., β k ) minimise the SSR: ˆˆ ˆ ( β 0 , β1,..., β k ) = minimiser of SSR. – As the estimates minimise SSR, the first order conditions lead to n ∑ (y i =1 i n ∑ (y i =1 i ˆ ˆ ˆ − β 0 − β1 x i 1 − − β k x ik ) = 0, ˆ ˆ ˆ − β 0 − β1 x i 1 − − β k x ik )x ij = 0, j = 1,..., k . the i-th observation on the j-th independent variable ie_Slides03 my, School of Economics, UNSW 9 3. Multiple Regression Model: Estimation (Ch3) • Ordinary least squares (OLS) – Solving the k+1 equations with k+1 unknowns gives ˆ B = ( X ' X )−1 X 'Y in a matrix notation. (linear in Y) 1st order conditions in a matrix notation : ˆ X ' (Y − XB ) = 0 – OLS only requires the existence of ( X ' X )−1 . It can be carried out for almost any data set. (X′X) is a form of sum-of-squared x’s. – Don’t worry, matrix will NOT be tested in this course. ie_Slides03 my, School of Economics, UNSW 10 3. Multiple Regression Model: Estimation (Ch3) • Sample regression function (SRF) – Once OLS estimates are obtained, ˆ ˆ ˆ ˆ y i = β 0 + β1 x i 1 + + β k x ik is the fitted value of y when (x1, ..., xk)= (xi1, ..., xik). – The OLS regression line or sample regression function (SRF) is ˆ ˆ ˆˆ y = β 0 + β1 x1 + + β k x k Unless indicated otherwise, which is an estimate of the PRF. an intercept is included. – “Run a regression of y on x1 , ..., xk” : use OLS to estimate the multiple regression model y = β0 + β1x1 +...+ βkxk + u. ie_Slides03 my, School of Economics, UNSW 11 3. Multiple Regression Model: Estimation (Ch3) • Interpretation of SRF – The OLS regression line or SRF can be written in the form of changes, holding u fixed: ˆ ˆˆ ∆y = β1∆x1 + + β k ∆x k . • the coefficient on x1 is the partial effect of x1 on y ˆˆ holding other x’s fixed: ∆y = β1∆x1. • we are able to control for (or hold fixed) {x2 , ..., xk} when considering the effect of x1 on y. ˆ • β1 has a ceteris paribus interpretation when ZCM holds or the factors in u are not “related” to x1. • other coefficients can be similarly interpreted. – If possible, include relevant independent variables. ie_Slides03 my, School of Economics, UNSW 12 3. Multiple Regression Model: Estimation (Ch3) • Interpretation of SRF – Example 3.2. Use educ, exper, tenure (years with current employer) to explain hourly wage: log(wage) = .284 + .092 educ + .004 exper + .022 tenure • the coefficient on educ : holding exper and tenure fixed, an extra year of education is predicted to increase log(wage) by 0.092 (or 9.2% increase in wage), which is the ceteris paribus effect under ZCM. • holding educ fixed, the effect of an individual staying at the same firm for an extra year on log(wage) : Δlog(wage) = .004 + .022 = .026 ie_Slides03 my, School of Economics, UNSW 13 3. Multiple Regression Model: Estimation (Ch3) • Predicted value and residual – The fitted value ˆ ˆ ˆ ˆ y i = β 0 + β1 x i 1 + + β k x ik is also known as predicted value. ˆ ˆ – The residual ui = y i − y i can be regarded as prediction error. – Important properties • the sample mean of the residuals is 0. • the sample covariance between the residuals and each independent variable is 0. • the sample mean point ( x1,..., x k , y ) is on the SRF, ie, ˆ ˆ ˆ y = β 0 + β1 x1 + + β k x k . ie_Slides03 my, School of Economics, UNSW 14 3. Multiple Regression Model: Estimation (Ch3) • Sums of squares ˆˆ – Each yi may be decomposed into y i = y i + ui . We measure variations from y : • Total sum of squares (total variation in yi ): SST = ∑i =1 ( y i − y )2 , n ˆ • Explained sum of squares (variation in y i ): ˆ SSE = ∑i =1 ( y i − y )2 , n • sum of squared Residuals (variation in ui ): ˆ ˆ SSR = ∑i =1 ui2 . n • It can be shown that ie_Slides02 SST = SSE + SSR . my, School of Economics, UNSW 15 3. Multiple Regression Model: Estimation (Ch3) • R-squared: goodness-of-fit – We may use the fraction of variation in y that is explained by x’s (or by the SRF) to measure the goodness-of-fit. – R-squared (coefficient of determination): SSR SSE 2 Note: R2 never decreases R= = 1− . when additional explanatory SST SST R2, variables are added to the model. • larger better fit; • 0 ≤ R2 ≤ 1. ˆ • R2 is the squared correlation between y and y . eg. log(wage) = .284 + .092educ + .004exper + .022tenure R2 = 0.316. ie_Slides03 my, School of Economics, UNSW 16 3. Multiple Regression Model: Estimation (Ch3) • Assumptions about multiple regression model (MLR1 to MLR4) 1. (linear in parameters) In the population model, y is related to x’s by y = β0 + β1x1 +...+ βkxk + u, where (β0, β1,..., βk) are population parameters and u is disturbance. 2. (random sample) {(xi1, xi2,..., xik , yi), i = 1,2,...,n} with n > k+1 is a random sample from the population. 3. (no perfect collinearity) None of x’s is constant and there is no perfect linear relationship among x’s. 4. (zero conditional mean) The disturbance u satisfies E(u | x1, ..., xk) = 0 for any given value of (x1, ..., xk). ie_Slides03 my, School of Economics, UNSW 17 3. Multiple Regression Model: Estimation (Ch3) • Unbiasedness of OLS estimators Theorem 3.1 Under MLR1 to MLR4, the OLS estimators are ˆ unbiased: E ( β j ) = β j , j = 0,1,..., k . Unbiased estimators – are “centred” around (β0, β1, ...,βk). – correctly estimate (β0, β1, ...,βk) on average. – will be “near” (β0, β1, ...,βk) for a “typical” sample. ie_Slides03 my, School of Economics, UNSW 18 3. Multiple Regression Model: Estimation (Ch3) • Irrelevant “explanatory” variables – What if an irrelevant x is included? • “irrelevant” means the population coefficient of that variable is 0. • as the OLS estimators are unbiased, the estimate of that coefficient will “typically” be near 0. • the inclusion of irrelevant variables has undesirable effects on the variances of the OLS estimators. eg. x3 is irrelevant in the model y = β0 + β1x1 + β2x2 + 0x3 + u, ˆ the OLS estimator is unbiased: E ( β j ) = β j , ˆ in particular, E ( β 3 ) = β 3 = 0. ie_Slides03 my, School of Economics, UNSW j = 0,1,..., k . 19 3. Multiple Regression Model: Estimation (Ch3) • Omitted explanatory variables – What if a relevant x is omitted? • the OLS estimators will generally be biased. • the direction and size of bias depend on how the omitted is related to the included. ie_Slides03 my, School of Economics, UNSW δ β β β – Example. When x2 is omitted from the true model y = β0 + β1x1 + β2x2 + u, it becomes: y = β0 + β1x1 + v with v = β2x2 + u. ~~~ The estimated model is y = β 0 + β1 x1. It can be shown that OLS is biased: E( ~1 ) = 1 + 2 ~, ~ ~ where β 2δ is known as omitted variable bias and δ is the coefficient of regressing x2 on x1. 20 3. Multiple Regression Model: Estimation (Ch3) • Omitted variable bias – Example. (continued) the omitted variable bias is zero in two special cases • when β2 = 0; or (the case of irrelevant variable) ~ • when δ = 0. (the case of uncorrelated x variables) In general, cov(x1, x2) > 0 cov(x1, x2) < 0 β2 > 0 + ve bias − ve bias β2 < 0 − ve bias + ve bias In practice, the knowledge about the signs of β2 and cov(x1, x2) is useful for interpreting estimation result. ie_Slides03 my, School of Economics, UNSW 21 3. Multiple Regression Model: Estimation (Ch3) ~ 2, δ 1+ β 1)= β E( ~ β • Omitted variable bias eg. Suppose the true model is log(wage) = β0 + β1educ + β2ability + u. We do not have data on ability but know that ability is positively affects wage and positively correlated with educ. The estimated equation is ~ log(wag e ) = .584 + .083educ, n = 526, R 2 = .186. ~ – β1 = .083 is from one sample and it can be either above or below the population parameter β1. ~ – The average of estimates β1 across many random samples would be too large (positive bias) ie_Slides03 my, School of Economics, UNSW 22 3. Multiple Regression Model: Estimation (Ch3) • Variance of OLS estimators 5. (MLR5, homoskedasticity) Var(ui | xi1, ..., xik) = σ2 for i = 1,2,...,n. (It implies Var(ui) = σ2.) – MLR5 requires that the conditional variance of u be unrelated to x’s. – MLR1-5 are collectively known as the Gauss-Markov assumptions. – MLR5 is needed to derive a “simple” formula for the variances of the OLS estimators. ie_Slides03 my, School of Economics, UNSW 23 3. Multiple Regression Model: Estimation (Ch3) • Variance of OLS estimators Strictly, Theorem 3.2 is about the variances of Theorem 3.2 OLS estimators, conditional on given x. Under MLR1 to MLR5, the variances of the OLS estimators are given by: σ2 ˆ Var ( β j ) = , j = 1,..., k, 2 SST j (1 − R j ) where SST j = ∑i =1 ( x ij − x j )2 , x j = n −1 ∑i =1 x ij and R 2 is the R-squared from regressing xj on all j other independent variables. ˆ – the larger is σ2, the greater is Var ( β j ). ˆ – the larger is R 2 , the greater is Var ( β j ) . j ˆ – the larger the variation in xj, the smaller Var ( β j ) . n ie_Slides03 my, School of Economics, UNSW n 24 3. Multiple Regression Model: Estimation (Ch3) • Multicollinearity ˆ – The larger is R 2, the greater is Var ( β j ) . j R 2 is the R-squared from regressing xj on all other x’s. j The larger is R 2, the stronger is xj “associated” with j other x’s, the less informative is xj. “R 2 = 1”, which is ruled out by MLR3, implies that j there is a perfect linear relationship between xj and other x’s (therefore, xj is redundant). – High (but not perfect) correlation between two or more independent variables is known as multicollinearity, which does not violate MLR3. eg. In the model y = β0 + β1x1 + β2x2 + β3x3 + u, if β1 is of interest, a high correlation between x2 and x3 may not be a major concern. ie_Slides03 my, School of Economics, UNSW 25 3. Multiple Regression Model: Estimation (Ch3) • Omitted variables again – Example. When x2 is omitted from the true model y = β0 + β1x1 + β2x2 + u, it becomes: y = β0 + β1x1 + v with v = β2x2 + u. ~ ˆ Let β1 and β1 be the OLS estimators from the true and the mis-specified. We find that σ2 σ 2 + β 22 var( x 2 ) ~ ˆ , Var ( β1 ) = . Var ( β1 ) = 2 SST1 (1 − R1 ) SST1 ~ ˆ 1. When β2 = 0 and R12 > 0, bothβ1 and β1are unbiased and the latter is more efficient. (Irrelevant x2 is not harmless.) ~ ˆ 2. When β2 ≠ 0 and R12 = 0, bothβ1 and β1are unbiased and the former is more efficient. (Relevant, uncorrelated x2 is beneficial.) ~ ˆ 3. When β2 ≠ 0 and R12 > 0, β1 is unbiased but β1is biased. ie_Slides03 my, School of Economics, UNSW 26 3. Multiple Regression Model: Estimation (Ch3) • Omitted variables again – Point 1: the inclusion of an irrelevant variable in the regression generally leads to inefficient estimator of β1. – Point 2: the exclusion of a relevant variable from the regression, when it is uncorrelated with x1, leads to inefficient estimator of β1. – Point 3: the exclusion of a relevant variable from the regression generally leads to biased estimator of β1. When bias is a major concern, researchers tend to include many x-variables as controls in practice. ie_Slides03 my, School of Economics, UNSW 27 3. Multiple Regression Model: Estimation (Ch3) • Estimation of σ2 – As the residual approximates u, the estimator of σ2 is ui2 ∑i =1 ˆ n SSR . σ= ˆ = n − (k + 1) n − (k + 1) 2 the number of estimated coefficients – σ = σ 2 is known as the standard error of the ˆ ˆ regression. – The degrees of freedom (df) for the regression is n – (k+1) = # of obs – # of estimated coefs. Theorem 3.3 (unbiased estimator of σ2) Under MLR1 to MLR5, E (σ 2 ) = σ 2 . ˆ ie_Slides03 my, School of Economics, UNSW 28 3. Multiple Regression Model: Estimation (Ch3) • Efficiency of OLS estimation Theorem 3.4 (Gauss-Markov theorem) ˆˆ ˆ Under MLR1 to MLR5, ( β 0 , β1,..., β k ) are the best linear unbiased estimators (BLUEs) of (β0, β1, ...,βk). – Linear estimator : a linear combination of y’s. – Linear unbiased estimators : a class of linear estimator that are unbiased. – BLUEs : ones with smallest variance. – OLS estimators under MLR1-MLR5: • they are linear; • they are unbiased; • they are efficient (have smallest variances). ie_Slides03 my, School of Economics, UNSW 29 3. Multiple Regression Model: Estimation (Ch3) • Summary – Multiple regression allows us to hold other (observable) factors fixed. – It allows for nonlinear relationships between y and x’s. – It is linear in the β parameters so that simple OLS method is applicable. – Under MRL1-MRL4, the OLS estimators are unbiased. – Under MRL1-MRL5, the OLS estimators are BLUEs and the estimator of σ2 is unbiased. – Omitting relevant variable or including irrelevant variable are generally undesirable. ie_Slides03 my, School of Economics, UNSW 30 ...
View Full Document

## This note was uploaded on 06/12/2011 for the course ECONOMICS 3291 taught by Professor Professorsnamespublishedtheyarethesoleowners during the Three '11 term at University of New South Wales.

Ask a homework question - tutors are online