This preview shows pages 1–9. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: + Evaluating Regression Models Political Science 102 Introduction to Political Inquiry Lecture 22 + Good Models and “Explaining” Variance What do we mean when we say a model is “good” or “explains” the dependent variable? Explanation exists in our theory, not in any data we might observe. Thus “explained” variance cannot be measured with a statistic (or with data). What do we mean when we say we have a “good model” of our data? Changes in X have a large impact on changes in Y ( That is, b is large) Remaining error terms are small (That is, σ u 2 is small) + A Composite Measure of Model Quality Developed an overall measure of these concepts that is insensitive to the scale measuring X & Y Measure is based on fundamental goal of the OLS estimator – minimize squared errors. Take the ratio of “Explained”Sum of Squares (ESS) to Total Sum of Squares (TSS) This ratio is known as the “coefficient of determination” or R 2 R 2 is literally just the correlation between Y and Yhat, squared TSS USS TSS ESS R = = 1 2 + What IS R 2 Really? The R 2 Stew Want a measure of model quality to compare across samples Like correlation coefficients – R 2 cannot generalize It depends on the variance of X and the variance of the errors R2 for bivariate regression can be written as: Which is: 2 2 2 2 2 2 * * u x x b b R σ σ σ + = ( causal _ strength ) 2 * s x 2 ( causal _ strength ) 2 * s x 2 + good _ fit USS ESS ESS TSS ESS R + = = 2 + Don’t Drink the R 2 Kool Aid Thus R 2 conflates our two aspects of a “good model,” combines them with σ x 2 , and places them on a dimensionless scale The resulting value is nearly uninterpretable Basically measures the shape of the cloud of observations around our regression line. R 2 tells us little we want to know – pay it no heed Also avoid standardized regression coefficients We must make substantive evaluations of our models: Size of Coefficients (hypothesis tests) Size of Substantive Effects Ability to forecast outofsample + Standards of Model Evaluation Statistical Significance of Coefficients Ttests Substantive Size of Effects Generate predictions from the model Size of Residuals Compare to substantive effects Forecasting OutofSample + Statistical Significance of Regression Coefficients In general, our theories give us hypotheses that B>0 or B<0 We can estimate b, but we need a way to assess the validity of statements that B is positive or negative We can rely on our estimate of b and its variance to use probability theory to test such statements. Ttests (or Zscores) give us confidence that relationships we observe generalize to the population Don’t get too focused on .05 as a “magical” threshold for significance + Z – Scores & Hypothesis Tests Recall that b ~ N( B , σ b ) Subtracting B from both sides, we can see that...
View
Full
Document
This note was uploaded on 10/21/2011 for the course POL SCI 102 taught by Professor Gelpi during the Spring '11 term at Duke.
 Spring '11
 Gelpi
 Political Science

Click to edit the document details