Populism in America
In recent years populism has been a growing global phenomenon. Concerns have been raised that
democracy might be threatened in the most praised democracy of all: the United States
Mikhail Khodorkovsky, once the richest man in Russia with an estimated net
worth of fifteen billion dollars, now lives in Switzerland after serving a 10-year prison
sentence with his fortune mostly go
30 CHAPTER 3 Least Squares
where
u=yXdzc is the vector of residuals when y is regressed on X and z. Note that unless Xz = 0,
dwillnotequalb= (XX)1Xy.(SeeSection8.2.1.)Moreover,unlessc =0,uwillnot equa
y=X + =X11 +X22 +. What is the algebraic solution for b2? Thenormal equations are (1) (2) X 1X1 X
1X2 X 2X1 X 2X2
b1 b2
=
X 1y X 2y
. (3-17) A solution can be obtained by using the partitioned inverse
Then
q =vv=
n i=1
v2 i , where v=Xc.
Unlesseveryelementofviszero,q ispositive.Butifvcouldbezero,thenvwouldbea
linearcombinationofthecolumnsofXthatequals0,whichcontradictstheassumption that X has full
a set of two equations: b2
i(T i T )2 + b3
i(T i T )(Gi G) =
i(T i T )(Gi G) + b3 i(Gi G)2 = i(Gi G)(Y i Y).
i(T i T )(Y i Y), b2
(3-8)
This result shows the nature of the solution for the slopes, whi
This result comes at a cost, however. The parameter estimates become progressively less precise as we
do so. We will pursue this result in Chapter 4. 4This measure is sometimes advocated on the basis
LEAST SQUARES REGRESSION The unknown parameters of the stochastic relation yi =x i + i are the
objects of estimation.Itisnecessarytodistinguishbetweenpopulationquantities,suchas andi,
andsampleestimat
THEOREM 3.7 Change in R2 When a Variable Is Added to a Regression Inamultipleregression, R2
willfall(rise)whenthevariable x isdeletedfromthe regression if the t ratio associated with this variable is
CHAPTER 3 Least Squares
y
e
a bx
E(y) x
x
x
y a bx
FIGURE 3.1 Population and Sample Regression.
Although numerous candidates have been suggested, the one used most frequently is least squares.1
3.2.1
noting is the signs of the coefcients. The signs of the partial correlation coefcients are the same as the
signs of the respective regression coefcients, three of which are negative. All the simple co
easily show that M is both symmetric (M=M) and idempotent (M=M2). In view of (313),wecaninterpretMasamatrixthatproducesthevectorofleastsquaresresiduals in the regression of y on X
when it premultiplie
in lny. The latter R2 will typically be larger, but this does not imply that the loglinear model is a better t
in some absolute sense. It is worth emphasizing that R2 is a measure of linear associatio
COROLLARY 3.3.2 Regression with a Constant Term The slopes in a multiple regression that contains a
constant term are obtained by transforming the data to deviations from their means and then regressi
CHAPTER 3 Least Squares 29
controlling for the effect of age, is obtained as follows: 1. y =the residuals in a regression of income on
a constant and age. 2. z =the residuals in a regression of educat
CHAPTER 3 Least Squares 25
the vector y into the column space of X. (See Sections A3.5 and A3.7.) By multiplying
itout,youwillndthat,likeM,Pissymmetricandidempotent.Giventheearlierresults, it also fol
or
S(b0) =yy2yXb0 +b0XXb0. The necessary condition for a minimum is S(b0) b0 = 2Xy+2XXb0 =0. (3-4)
1We shall have to establish that the practical approach of tting the line as closely as possible to t
uu=ee (z
y)2 (z z) =ee
1r2 yz
, (3-28)
where r yz is the partial correlation between y and z, controlling for X. Now divide
throughbothsidesoftheequalitybyyM0y.From(3-26),uu/yM0yis (1R2 Xz) forthe re
CHAPTER 3 Least Squares 33
or
SST=SSR+SSE. (3-25) (Note that this is precisely the partitioning that appears at the end of Section 3.2.4.)
We can now obtain a measure of how well the regression line t
0s and add a variable to the model that takes the value 1 for that one observation and 0 for all other
observations. Show that this strategy is equivalent to discarding the observation as regards the
bXM0Xb= yM0 y,
but y = Xb,y = y+e,M0e = e, and Xe = 0, so yM0 y = yM0y. Multiply R2 = yM0 y/yM0y=
yM0y/yM0y by 1= yM0y/ yM0 y to obtain R2 = [
i(yi y)( yi y)]2 [ i(yi y)2][
i( yi
y)2] , (3-27) whic
The solution is b= (XX)1Xy= (0.50907,0.01658,0.67038,0.002326,0.00009401).
3.2.3 ALGEBRAIC ASPECTS OF THE LEAST SQUARES SOLUTION The normal equations are
XXbXy=X(yXb) =Xe=0. (3-12) Hence, for every co
CHAPTER 3 Least Squares 37
bypass these difculties by reporting a third R2, the squared sample correlation between the actual
values of y and the tted values from the regression. This approach could b
THEOREM 3.1 Orthogonal Regression If the variables in a multiple regression are not correlated (i.e., are
orthogonal), then the multiple regression slopes are the same as the slopes in the individual
CHAPTER 3 Least Squares 23
Now divide both the numerator and denominator in the expression for b3 by
it2 i
ig2 i . By
manipulating it a bit and using the denition of the sample correlation between G a
3.6 SUMMARY AND CONCLUSIONS
Thischapterhasdescribedthepurelyalgebraicexerciseofttingaline(hyperplane)to a set of points using the
method of least squares. We considered the primary problem rst,usingad
TABLE 3.4 Analysis of Variance for the Investment Equation Source Degrees of Freedom Mean Square
Regression 0.0159025 4 0.003976 Residual 0.0004508 10 0.00004508 Total 0.016353 14 0.0011681 R2
=0.0159
R2 1,2 = R2 1 +
1 R2 1
r2 y21. [This is the multivariate counterpart to (3-29).] Therefore, it is possible to push R2 as high as
desired just by adding regressors. This possibility motivates the use o
IntermsofExample2.2,wecouldobtainthecoefcientoneducationinthemultiple regression by rst
regressing earnings and education on age (or age and age squared) and then using the residuals from
these regres