3300: PAM Regression Analysis
Omitted Variable Bias
In an OLS regression of Y on X, bOLS measures the true partial effect of X on Y () only if X is uncorrelated with the omitted variables that also affect Y (d/dX = 0). In observational studies, this is unlikely to be true and so in general, bOLS will be biased.
Effect of X on Z
dY bols = = " + #$ dX
Effect of Z on Y
If we know something about the signs of the effect of omitted variables on Y () and the relationship between ! X and the omitted variables () then we can sign the bias: guess whether it is positive or negative.
OVB - some examples
For each of the following examples, think about what biases, if any, might be important in interpreting the results of a simple regression of Y on X. Is the simple regression likely to capture the causal effect of X on Y? What omitted variables do you think are important? What are the signs of the biases, if any, due to omitting each variable (what are you assuming in signing the bias)? On net, can you say whether the simple bivariate regression is likely to overestimate or underestimate the true causal effect (I.e., is the bias positive or negative)?
OVB - examples
1. Does Hormone Replacement Therapy (X) reduce the risk of heart disease (Y)? A longitudinal study follows 10,000 nurses over time, some who elect to receive HRT, some who do not. The data include measures on whether each individual contracts heart disease. 2. Does having access to cheaper health insurance lead to higher expenditures on health care? A large survey asks respondents how much they are required to pay for a variety of medical procedures: X measures the percent of total medical costs that must be paid by the individual (X ranges from O insurance provides total coverage to 100 theres no insurance coverage). The survey also asks about total yearly expenditures (both those paid by the individual and those paid by insurance) incurred by each person. 3. Does living in a high poverty neighborhood adversely affect the academic achievement of younger students? The test scores (Y) from students in the 3rd grade in NYC are linked to information about the neighborhoods in which they reside. In particular, data are available on the fraction of poor (incomes below the poverty line) individuals in each students neighborhood (census tract).
OVB - examples (cont.)
4. Does regular exercise reduce the prevalence of obesity? A survey asks a large sample of individuals about their height and weight, used to determine whether they are obese (Y), along with how many times per week they typically exercise (X). 5. Does immigration adversely affect the wages of low-skill native workers? Census data is used to compute the share of immigrants in the workforce (X) in each county in the United States. The same data is also used to compute the average wages (Y) of all individuals with a high-school degree or less. 6. Does remedial summer school education improve student achievement? Administrative data from a school district records which students attended summer school for remedial education (X) before the school year. The same data also includes the performance of these students on end-of-year exams (Y) for the year after the students were in summer school.
Solutions to Omitted Variable Bias
There are two basic approaches for dealing with omitted variables bias:
Statistical Control: use multiple regression or a similar technique to control for sources of bias, or Randomization control: rely on an actual randomized experiment or a natural experiment to create independence of key treatment and sources of bias. (well get to this later)
Motivation - A familiar example
Consider an example: Whats the effect of education (X) on earnings (Y)? We suspect education is correlated with , so if we regress only Y on X we will have omitted variables bias. One omitted variable might be race (Z=1 if white, 0 otherwise) - for a variety of factors, race may be related to educational attainment and have an independent association with earnings.
Results of a regression ignoring Z
Do we estimate the true (partial) effect of X on Y? If we omit Z, we worry that bOLS might be biased.
. reg lnw coll Source | SS df MS -------------+-----------------------------Model | 84.4818412 1 84.4818412 Residual | 399.954623 998 .400756136 -------------+-----------------------------Total | 484.436465 999 .484921386 Number of obs F( 1, 998) Prob > F R-squared Adj R-squared Root MSE = = = = = = 1000 210.81 0.0000 0.1744 0.1736 .63305
-----------------------------------------------------------------------------lnw | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------coll | .6412066 .0441628 14.52 0.000 .554544 .7278691 _cons | 2.553667 .0237413 107.56 0.000 2.507079 2.600256 ------------------------------------------------------------------------------
Diagnosing an OVB Problem (1)
1) Is X related to Z? If education is related to other variables that affect wages, bOLS may pick up indirect effects of those variables.
. reg white coll Source | SS df MS -------------+-----------------------------Model | 4.74167091 1 4.74167091 Residual | 214.982329 998 .215413155 -------------+-----------------------------Total | 219.724 999 .219943944 Number of obs F( 1, 998) Prob > F R-squared Adj R-squared Root MSE = = = = = = 1000 22.01 0.0000 0.0216 0.0206 .46413
-----------------------------------------------------------------------------white | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------coll | .1519085 .0323782 4.69 0.000 .0883714 .2154456 _cons | .6300985 .0174061 36.20 0.000 .5959417 .6642552 ------------------------------------------------------------------------------
Diagnosing an OVB Problem (2)
2) Is Z related to Y? If we omit something that doesnt affect Y, no problem. Otherwise, weve got trouble.
. reg lnw white Source | SS df MS -------------+-----------------------------Model | 19.8714435 1 19.8714435 Residual | 464.565021 998 .465496013 -------------+-----------------------------Total | 484.436465 999 .484921386 Number of obs F( 1, 998) Prob > F R-squared Adj R-squared Root MSE = = = = = = 1000 42.69 0.0000 0.0410 0.0401 .68227
-----------------------------------------------------------------------------lnw | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------white | .3007295 .0460277 6.53 0.000 .2104073 .3910516 _cons | 2.536285 .0377876 67.12 0.000 2.462132 2.610437 ------------------------------------------------------------------------------
Whats the effect of OVB in this case?
It looks like Z belongs in our model for Y. A better model of earnings might look like: = Yi + Xi + Zi + I Regressing only Y on X will yield a biased bOLS
Effect of X on Z
dY bols = = " + #$ dX
Whats the sign of the bias?
Effect of Z on Y
Mutiple Regression approach: controlling for sources of OVB
If we have data on Z, then we can include Z in our regression model and estimate the partial effect of X on Y (holding Z constant). (Also the partial effect of Z on Y, holding X constant). We now estimate , , and from: Yi = + Xi + Zi + I
. reg lnw coll white Source | SS df MS -------------+-----------------------------Model | 94.3514277 2 47.1757138 Residual | 390.085037 997 .391258813 -------------+-----------------------------Total | 484.436465 999 .484921386 Number of obs F( 2, 997) Prob > F R-squared Adj R-squared Root MSE = = = = = = 1000 120.57 0.0000 0.1948 0.1932 .62551
Est() Est() -----------------------------------------------------------------------------lnw | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------coll | .6086581 .0441149 13.80 0.000 .5220893 .6952269 white | .2142635 .042661 5.02 0.000 .1305479 .297979 _cons | 2.41866 .0356772 67.79 0.000 2.348649 2.488671 -----------------------------------------------------------------------------Est()
What just happened?
(1) reg y x: E[earningsi|colli] = 2.554 + .641 colli
(2) reg z x: E[whitei|colli] = .630 + .152 colli (3) reg y z: E[earningsi|whitei] = 2.536 + .301 whitei
(4) reg y x z: E[earningsi|colli,whitei] = 2.419 + .609 colli + .214 whitei NOTICE: bOLS in (4) is lower than in (1): .641-.609 = .032 lower In (1): bOLS = + , so bias =
bias = .152*.214 = .032
What does multiple regression do?
Multiple regression estimates the change in the average value of Y due to a one unit change in each independent variable, holding all other independent variables constant. (bOLS will measure Y/X, not dY/dX) Compared to reg y x, using reg y x z gets rid of bias due to differences in race (z) between those who have a college degree (x=1) and those who dont. You can view the regression as looking at the change in earnings between people with and without a college degree within the same race category - in this sense, the regression controls for race. (n.b. -
this is conceptually correct, but not quite right in terms of the math when z is continuous).
What doesnt multiple regression do?
bOLS from reg y x z is NOT necessarily an unbiased estimate of . If X is correlated with other things that may affect Y (even after controlling for Z) then omitting those variables will cause bias for bOLS.
Controlling for Another Variable (age)
Human capital theory in economics suggest that experience is another important determinant of wages - it may also be related to age among people of the same race. Lets assume experience is well proxied by age (A). Then the true model should be: Yi = + Xi + Zi + Ai+ I
. reg lnw coll white age Source | SS df MS -------------+-----------------------------Model | 118.182301 3 39.3941005 Residual | 366.254163 996 .367725064 -------------+-----------------------------Total | 484.436465 999 .484921386 Number of obs F( 3, 996) Prob > F R-squared Adj R-squared Root MSE = = = = = = 1000 107.13 0.0000 0.2440 0.2417 .6064
-----------------------------------------------------------------------------lnw | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------coll | .5518004 .0433469 12.73 0.000 .4667386 .6368621 white | .2093039 .0413627 5.06 0.000 .1281359 .2904718 age | .0130286 .0016184 8.05 0.000 .0098527 .0162045 _cons | 1.925929 .0703037 27.39 0.000 1.787968 2.063889 ------------------------------------------------------------------------------
What just happened?
(1) reg y x: E[earningsi|colli] = 2.554 + .641 colli
(4) reg y x z: E[earningsi|colli,whitei] = 2.419 + .609 colli + .214 whitei
(5) Reg y x z a: E[earningsi|colli,whitei] = 1.926 + .552 colli + .209 whitei +.013 agei NOTICE: The coefficient on college changes as you add more variables because they are correlated with both x and y - here you are controlling for the bias due to the correlation of college attendance with race and age. If you add variables not correlated with college completion, then the coefficient wont change
Controlling for a variable that is not related to education
. reg female coll -----------------------------------------------------------------------------female | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------coll | .0221239 .034895 0.63 0.526 -.0463521 .0905999 _cons | .4796062 .0187591 25.57 0.000 .4427944 .516418 -----------------------------------------------------------------------------. reg lnw coll white age female Source | SS df MS -------------+-----------------------------Model | 131.287682 4 32.8219205 Residual | 353.148783 995 .3549234 -------------+-----------------------------Total | 484.436465 999 .484921386 Number of obs F( 4, 995) Prob > F R-squared Adj R-squared Root MSE = = = = = = 1000 92.48 0.0000 0.2710 0.2681 .59575
-----------------------------------------------------------------------------lnw | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------coll | .5588561 .0426015 13.12 0.000 .4752569 .6424552 white | .192968 .0407251 4.74 0.000 .113051 .272885 age | .013143 .0015901 8.27 0.000 .0100226 .0162633 female | -.2296076 .0377858 -6.08 0.000 -.3037567 -.1554586 _cons | 2.041991 .0716614 28.49 0.000 1.901366 2.182616 ------------------------------------------------------------------------------
When can multiple regression coefficients be given causal interpretations?
bOLS represents the true partial effect of X on Y (holding all other factors constant) if X is not correlated with the error term even after accounting for the effects of the control variables (i.e., Z, A, etc.). Ask yourself:
What things besides the control variables affect Y? Among individuals who are similar in terms of the control variables (same race, same age, and same sex), do you expect the omitted variable to be correlated with X? If so, then there is an omitted variable bias for bOLS If data exists, then include the variable in the regression If not, at least think about the sign of the bias - is bOLS an overestimate or underestimate?
E[earningsi|colli,whitei] = 1.926 + .552 colli + .209 whitei +.013 agei Question: Is .552 the causal effect of college attendance on earnings?
next
Problem Set 2: Due in-class February 19th Next week - Randomized Trials: an overview
Avorn, Chapter 2; Shadish, Cook, and Campbell: Chapter 8; Burtless
And then:
Manning et al. random evaluation of cost-sharing in health insurance plans