L11 Ordinary Least Squares Regression

# L11 Ordinary Least Squares Regression - Ordinary Least...

This preview shows pages 1–3. Sign up to view the full content.

Ordinary Least Squares Regression. Simple Regression. Algebra and Assumptions. In this part of the course we are going to study a technique for analysing the linear relationship between two variables Y and X. We have n pairs of observations (Y i X i ), i = 1, 2, . .,n on the relationship which, because it is not exact, we shall write as: 1,. ..., ii i y xin α β = + +∈ = In this relationship α , β and ε i i = 1,. ., n are fundamentally unobservable and we would like to estimate α and β . Approach: The idea is to select estimates of α and β lets call them α * and β * which yield a straight line (called the regression line) Y = α * + β *X which minimises a measure of the aggregate distance of the points (Y i X i ), i = 1, 2, . .,n to that line in X Y space, where Y is measured on the vertical axis. The measure we use is the sum of squared vertical distances which we shall call the Error Sum of Squares (ERSS) so that α * and β * are solutions to the problem: () 2 , 1 min n ti i ERSS Y X αβ ∗∗ = ⎛⎞ =− + ⎜⎟ ⎝⎠ The multivariate calculus is employed to solve this problem by setting the partial derivatives of ERSS with respect to α * and β * to zero (called the first order conditions) and solving thus: 1 1 20 n i n i ERSS YX ERSS X = = + = + = i The solutions to which are:

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
() 1 1 n ii i n i YX YYX X XX αβ β ∗∗ = = =− = Note the formula for β * has many alternative equivalent versions which may be seen by observing that for the numerator: 11 1 1 nn n n i i i i i i YYX X X X Y X Yn Y X == = = −= = ∑∑ and for the denominator: 2 2 1 n i i i i X X X XX X n X = so that various combinations of numerator and denominator formulae yield 12 alternative representations of β * . Assumptions in the Ordinary Least Squares model. Note that while α , β and ε i , i = 1,. ., n are fundamentally unobservable we only concern ourselves with estimating α and β which define the relationship between Y and X. The ε i i = 1,. ., n are considered “errors” which accommodate all the other influences on Y not accounted for in α + β X as such we assume them to be random and to obey the following four assumptions: 1) E( ε i ) = 0 for all i. This assumption really says that the average or net effect of all the other influences on Y not accounted for in α + β X is constant and zero for each observation (i=1,. .,n). 2) V( ε i ) = σ 2 > 0 for all i.
This is the end of the preview. Sign up to access the rest of the document.

## This note was uploaded on 03/23/2011 for the course ECONOMICS 209 taught by Professor Kambourov during the Spring '11 term at University of Toronto- Toronto.

### Page1 / 9

L11 Ordinary Least Squares Regression - Ordinary Least...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online