{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

01MESMET - 1 CHAPTER Elementary Regression Analysis In this...

This preview shows pages 1–3. Sign up to view the full content.

1 : CHAPTER Elementary Regression Analysis In this chapter, we shall study three methods which are capable of gener- ating estimates of statistical parameters in a wide variety of contexts. These are the method of moments, the method of least squares and the principle of maximum likelihood. We shall study the methods only in relation to the simple linear regression model; and we shall show that each entails assumptions which may be more or less appropriate to the context in which we wish to apply the model. In the case of the regression model, the three methods generate estimating equations which are formally identical; but this does not justify us in taking a casual approach to the statistical assumptions which sustain the model. To be casual in making our assumptions is to invite the danger of misinterpretation when the results of the estimation are in hand. We begin with the method of moments, we shall proceed to the method of least squares, and we shall conclude with a brief treatment of the method of maximum likelihood. Conditional Expectations Let y be a continuously distributed random variable whose probability density function is f ( y ). If we wish to predict the value of y without the help of any other information, then we might take its expected value which is defined by E ( y ) = Z y yf ( y ) dy. The expected value is a so-called minimum-mean-square-error (m.m.s.e.) predictor. If π is the value of a prediction, then the mean-square error is given by (1) M = Z y ( y - π ) 2 f ( y ) dy = E ' ( y - π ) 2 = E ( y 2 ) - 2 πE ( y ) + π 2 ; 1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
D.S.G. POLLOCK : ECONOMETRIC THEORY and, using the methods of calculus, it is easy to show that this quantity is minimised by taking π = E ( y ). Now let us imagine that y is statistically related to another random variable x whose value we have already observed. For the sake of argument, let us assume that we know the form of the joint distribution of x and y which is f ( x, y ). Then the minimum-mean-square-error prediction of y is given by the conditional expectation (2) E ( y | x ) = Z y y f ( x, y ) f ( x ) dy wherein (3) f ( x ) = Z y f ( x, y ) dy is the so-called marginal distribution of x . We may state this proposition formally in a way which will assist us in proving it: (4) Let ˆ y = ˆ y ( x ) be the conditional expectation of y given x which is also expressed as ˆ y = E ( y | x ). Then we have E { ( y - ˆ y ) 2 } ≤ E { ( y - π ) 2 } , where π = π ( x ) is any other function of x . Proof. Consider (5) E ' ( y - π ) 2 = E h ' ( y - ˆ y ) + (ˆ y - π ) 2 i = E ' ( y - ˆ y ) 2 + 2 E ' ( y - ˆ y )(ˆ y - π ) + E ' y - π ) 2 . In the second term, there is (6) E ' ( y - ˆ y )(ˆ y - π ) = Z x Z y ( y - ˆ y )(ˆ y - π ) f ( x, y ) ∂y∂x = Z x ‰ Z y ( y - ˆ y ) f ( y | x ) ∂y y - π ) f ( x ) ∂x = 0 . Here the second equality depends upon the factorisation f ( x, y ) = f ( y | x ) f ( x ) which expresses the joint probability density function of x and y as the product of the conditional density function of y given x and the marginal density func- tion of x . The final equality depends upon the fact that R ( y - ˆ y ) f ( y | x ) ∂y = E ( y | x ) - E ( y | x ) = 0. Therefore E { ( y - π ) 2 } = E { ( y - ˆ y ) 2 } + E { y - π ) 2 } ≥ E { ( y - ˆ y ) 2 } , and the assertion is proved.
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern