620 Econ Why GLS? Recall the assumptions of the classical multiple regression model - especially the assumption on the distribution of the disturbance terms; y = X + E () = 0 E ( ) = I
2
(1) (2)
The zero mean assumption is not so severe that we can easily accommodate the non-zero mean by dening the constant term dierently. However, the assumption on the second moment matrix of the disturbance terms are very restrictive; the homoskedasticity & uncorrelatedness assumption (or, indeed, sometimes the stronger i.i.d. assumption) represented by (2) is too stringent to be applied to most economic data. Alternative specication of the error term is given by; E () = 0 E ( ) = V (3) where V is an arbitrary positive denite symmetric matrix. The specication can nest both heteroskedasticity and serial correlation in disturbance terms. To see the argument in detail, consider the explicit form of the matrix V ; E 2 E (1 2 ) E (1 N 1 ) E (1 N ) 1 E (2 1 ) E 2 E (2 N ) E (2 N 1 ) 2 (4) V = E (N 1 1 ) E (N 1 2 ) E 2 1 E (N 1 N ) N E (N 1 ) E (N 2 ) E (N N 1 ) E 2 N What is the consequence of the OLS estimation with error structure (3)? It is still unbiased; OLS = (X X)
1
X y = + (X X)
1
1
X
E OLS = + (X X) It has dierent variance matrix; V ar OLS = E OLS
1
X E () =
OLS
1
= E (X X)
X X (X X)
= (X X)
1
1
X V X (X X)
1
(5)
Note that under classical assumptions; V ar OLS = 2 (X X)
.
It is not BLUE. - immediate consequence of Gauss-Markov theorem. Since we have dierent variance formula as in (5), the usual t-test and F test statistics are invalid. It is still consistent as long as plim XNX = Q and plim X = 0; N plimOLS = plim + (X X) = + plim XX N
1
X plim X = + Q1 0 = N (6)
1
The asymptotic variance matrix is dierent from what we used to have in classical cases. The asymptotic distribution of OLS is now given by; N OLS N
d
0,
XX N
1
X VX N
XX N
1
(7)
as long as the probability limits of three arguments of the asymptotic variance matrix exist. 1
Now, what to do?
First of all, we will reparameterize the matrix V in slightly dierent way; V = 2 we lose no generality in this reparameterization. But the reparameterization will deliver a convenient comparison between OLS and GLS. Suppose that we know the complete structure of , which, of course, is highly unlikely. Anyway, then we can always nd a decomposition of 1 such that L L = 1 where L is an (N N ) non-singular matrix. Multiplying both sides of (1) with L, we have; Ly = LX + L E (L) = LE () = 0 V ar (L) = LV ar () L = LV L = L 2 L = 2 L (L L) Regressing Ly on LX gives; GLS = (LX) (LX) = [X L LX] = X 2
1 1 1 1
(8)
(9)
We can treat Ly as dependent variable, LX as independent variables, and L as error terms. Then, L = 2 I (10)
Note that the error terms now satises the assumptions of the classical regression model;
(LX) Ly
1
(X L Ly) = X 1 X X
1
X 1 y
1
(11) X V 1 y (12)
X 2
1
y = X V 1 X
Lets check the characteristics of GLS estimator; GLS = X V 1 X Hence, It is unbiased; E GLS = + X V 1 X Its variance is given by; V ar GLS = E GLS E GLS =E =E
1 1 1
X V 1 y = X V 1 X
1
1
X V 1 [X + ]
= + X V 1 X
X V 1
X V 1 E () =
(13)
GLS E GLS GLS GLS
1 1 1 1
X V 1 X
1
1
X V 1 V 1 X X V 1 X
1
= X V 1 X = X V 1 X
X V 1 E ( ) V 1 X X V 1 X = X V 1 X = 2 X 1 X
X V 1 V V 1 X X V 1 X
(14)
1 N
It is BLUE; It is consistent under the usual conditions; the crucial condition is again plim X Asymptotic distribution is given by; N GLS N
d
= 0;
0, 2
X 1 X N
1
(15)
2
Feasible Generalized Least Squares (FGLS)
The theory for GLS is nice. How useful is it? The answer is that it is virtually useless. The truth is that we dont know V or at least . Then, what are we supposed to do? One universally true maxim in econometrics is that when you have something you dont know, estimate it!. There are a lot of way to estimate depending on the model we consider. For the moment, just assume that we have a consistent estimator of . We can replace with in our procedure. The procedure is naturally called FGLS. We can derive the asymptotic distribution of FGLS estimator under some conditions. Suppose that plim X 1 X = Q where Q is positive denite and nite N plim then, F GLS = X 1 X Suppose that plim X X 1 1 X N 1 1 N
d 1
X 1 =0 N
X 1 y is consistent. - prove it.
=0 =0
1
plim then,
N F GLS N
0, 2
X 1 X N
(16)
The proof is in the lecture note and you have to redo the exercise with your own pencil and paper. The above conditions are sucient and they are satised when
p
Examples
Grouping of the observations; In some cases, statistical sources group observations and publish only average values for each group in order mainly to protect the identity of the survey subjects. However, most economic models are usually based on individual decision making. How can we solve the problem? Surely, we cannot solve the whole problem, but there is a lot better way to analyze the data set than simple OLS with grouped data. Suppose the true model is y = X + E () = 0 E ( ) = 2 I
But, we have G group-averaged observations on yi , Xi where i = 1, 2, , G. Suppose that we have ni individuals in each group so that n1 + n2 + + nG = N. Due to the data requirement, we have to consider the model; y = X +
3
Clearly, we can infer that 1 E () = E 2 = 0 G 2 1 2 1 G 1 1 2 2 2 G 2 V ar ( ) = E 1 G 2 G 2 G 1 0 0 n1 1 0 0 n2 = 2 0 0 n1 G
=
2 0 n2 0 0 0
2 n1
0 0
2 nG
If we know the number of individuals in each group, which is usually available, we can construct 2 . We know exact structure of . The L matrix in this case is; n1 0 0 0 n2 0 L= nG 0 0 It is sometimes not reasonable to assume that the type of heteroskadasticity depends on one or a combination of independent variables. Suppose that, for simplicity, the pattern of heteroskadasticity is determined by js independent variable. Then; y = X + and; x2 1j 0 2 V ar () = 0 E () = 0 0 x2 2j 0 0 0 x2 j N
where xij is the ith observation on the j th independent variable. Then, the GLS estimate is obtained from; yi 1 xi2 = j + 1 + 2 + xij xij xij xij1 xij+1 + j1 + j+1 + + 2 xij xij
xiK xij
+
i xij
We can also assume that the pattern of heteroskadasticity is governed by a combination of some variables - which may include independent variables or other variables-. The specication is then; yi = xi + i i = 1, 2, , N
where is a (k 1) vector of parameters.. We cam specify; E 2 = 2 zi ( ) and E (i j ) = 0 when i = j i where zi is an (h 1) vector. We still hold the independence assumption but give up homoskadasticity. In the variance specication, both 2 and are unknown parameters and zi is the vector of observation on variables z s. We can estimate the model using GLS. The problem is that we dont know the
2
4
parameter so that we dont know . If we can somehow consistently estimate , therefore, , we can do FGLS. More appealing approach is MLE. If we assume that i N 0, 2 ( zi )
2
with serial independence. The the log likelihood function is; L , 2 , = N N log 2 log 2 2 2
N
zi
i=1
1 2 2
N i=1
(yi xi ) ( zi )
2
2
we can estimate , 2 ,and by dierentiating the log-likelihood function. We know that the MLE are consistent and asymptotically ecient. The asymptotic variance matrix is obtained by the inverse of information matrix as usual. Another quite popular specication is that i N 0, 2 exp ( zi ) We now turn to the example where we keep the homoskadasticity assumption but weaken dependence structure of error terms. If we allow some correlations in error terms, our variance matrix of error terms is not a diagonal matrix anymore. Do you see why? Look at the matrix (4). One of the most popular specication of disturbance terms with serial dependence is AR(1) model; yt = xt + ut ut = ut1 + t E (t ) = 0, E Under the specication, we know that E (ut ) = 0, V ar (ut ) = Cov (ut , ut+h ) =
2 for all t = 1, 2, .T 1 2
|| < 1
2 , E
2 t
=
(t s ) = 0 when t = s
2 h , Corr (ut , uth ) = h 1 2
Hence, in vector notation, the variance matrix of error terms are; 1 T 2 T 1 1 T 3 T 2 2 V ar (uu ) = 2 1 T 2 T 3 1 T 1 T 1 1 T 2 T 1 1 1 T 3 T 2 = 2 = 2 T 2 3 T 1 1 T 1 T 1 where 2 =
2 12 .
It is know that 1 = 1 2 1 0 0 0 0 1 + 2 0 0 0 0 1 + 2 0 0 5 0 0 1 + 2 0 0 0 0 0 1 + 2 0 0 0 0 0 1
1
and L= 1
1 2
1 2 0 0
0 0 1 0 1 0
0 0 0 1
The estimation of the model will be discussed later.
Seemingly Unrelated Regression Estimator (SURE)
Consider a typical utility maximization problem a consumer solves; max U (x) s.t. p x w where x is a (M 1) vector of quantity demanded and p is the price vector. The solution to the max problem will be given as; x1 = f (p1 , p2 , , pM , w) x2 = f (p1 , p2 , , pM , w) xM = f (p1 , p2 , , pM , w) Econometrically, we would specify the model as; x1i = f (p1t , p2t , , pMt , wt ) + 1i x2i = f (p1t , p2t , , pMt , wt ) + 2i xMi = f (p1t , p2t , , pMt , wt ) + N i where i = 1, 2, , N. We may estimate each equation by OLS to get the estimates of parameters. However, we may lose some information doing that. It is highly likely that the demand equations are interdpendent since consumers determine the quantity demanded simultaneously, not separately. In statistical notation, it is natural to assume that E (ji li ) = 0 when j = l (17)
We can achieve some improvement in eciency by incorporating the information on the inter-equation dependence into estimation procedure. The seemingly unrelated regression estimator will give us the answer to the question of how to do that. Suppose that we have M system of equations; y 1 = X 1 1 + 1 y 2 = X 1 1 + 2 y M = X 1 1 + M where yj is (N 1) matrix of observations on the dependent variable of the j th equation, Xj is (N K) is (N Kj ) matrix of observations on the independent variables of the j th equation, and j is (N 1) matrix of the disturbances of the j th equation. For the notational simplicity, we will assume that each equation has the same number of regressors, K, i.e. K1 = K2 = = KM = K. We assume that error terms are independent across observations but dependent across equations; E (ji hi ) = jh for all i = 1, 2, , N E (ji jl ) = 0 when i = l E (jr hs ) = 0...