01MESMET - 1 CHAPTER Elementary Regression Analysis In this chapter we shall study three methods which are capable of generating estimates of

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
1 : CHAPTER Elementary Regression Analysis In this chapter, we shall study three methods which are capable of gener- ating estimates of statistical parameters in a wide variety of contexts. These are the method of moments, the method of least squares and the principle of maximum likelihood. We shall study the methods only in relation to the simple linear regression model; and we shall show that each entails assumptions which may be more or less appropriate to the context in which we wish to apply the model. In the case of the regression model, the three methods generate estimating equations which are formally identical; but this does not justify us in taking a casual approach to the statistical assumptions which sustain the model. To be casual in making our assumptions is to invite the danger of misinterpretation when the results of the estimation are in hand. We begin with the method of moments, we shall proceed to the method of least squares, and we shall conclude with a brief treatment of the method of maximum likelihood. Conditional Expectations Let y be a continuously distributed random variable whose probability density function is f ( y ). If we wish to predict the value of y without the help of any other information, then we might take its expected value which is defined by E ( y )= Z y yf ( y ) dy. The expected value is a so-called minimum-mean-square-error (m.m.s.e.) predictor. If π is the value of a prediction, then the mean-square error is given (1) M = Z y ( y - π ) 2 f ( y ) dy = E ' ( y - π ) 2 = E ( y 2 ) - 2 πE ( y )+ π 2 ; 1
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
D.S.G. POLLOCK : ECONOMETRIC THEORY and, using the methods of calculus, it is easy to show that this quantity is minimised by taking π = E ( y ). Now let us imagine that y is statistically related to another random variable x whose value we have already observed. For the sake of argument, let us assume that we know the form of the joint distribution of x and y which is f ( x, y ). Then the minimum-mean-square-error prediction of y is given by the conditional expectation (2) E ( y | x )= Z y y f ( x, y ) f ( x ) dy wherein (3) f ( x Z y f ( x, y ) dy is the so-called marginal distribution of x . We may state this proposition formally in a way which will assist us in proving it: (4) Let ˆ y y ( x ) be the conditional expectation of y given x which is also expressed as ˆ y = E ( y | x ). Then we have E { ( y - ˆ y ) 2 }≤ E { ( y - π ) 2 } , where π = π ( x ) is any other function of x . Proof. Consider (5) E ' ( y - π ) 2 = E h ' ( y - ˆ y )+(ˆ y - π ) 2 i = E ' ( y - ˆ y ) 2 +2 E ' ( y - ˆ y )(ˆ y - π ) + E ' y - π ) 2 . In the second term, there is (6) E ' ( y - ˆ y y - π ) = Z x Z y ( y - ˆ y y - π ) f ( x, y ) ∂y∂x = Z x ‰Z y ( y - ˆ y ) f ( y | x ) ∂y ± y - π ) f ( x ) ∂x =0 . Here the second equality depends upon the factorisation f ( x, y f ( y | x ) f ( x ) which expresses the joint probability density function of x and y as the product of the conditional density function of y given x and the marginal density func- tion of x . The final equality depends upon the fact that R ( y - ˆ y ) f ( y | x ) = E ( y | x ) - E ( y | x ) = 0. Therefore E { ( y - π ) 2 } = E { ( y - ˆ y ) 2 } + E { y - π ) 2 }≥ E { ( y - ˆ y ) 2 } , and the assertion is proved.
Background image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 10/27/2009 for the course MBA 1918272 taught by Professor Peter during the Fall '09 term at Aberystwyth University.

Page1 / 22

01MESMET - 1 CHAPTER Elementary Regression Analysis In this chapter we shall study three methods which are capable of generating estimates of

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online