Stat841f09 - Wiki Course Notes

# The simple linear regression model has the general

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: can be found at the University of South Florida (http://numericalmethods.eng.usf.edu/topics/linear_regression.html) and this MIT lecture (http://academicearth.org/lectures/applications- to- linear- estimation- least- squares) . For the purpose of classification, the linear regression model assumes that the regression function is linear in the inputs . The simple linear regression model has the general form: where is a vector and is a vector . Given input data and our goal is to find Squares method (http://en.wikipedia.org/wiki/Least_squares) . Note that vectors Denote and such that the linear model fits the data while minimizing sum of squared errors using the Least could be numerical inputs, transformations of the original data, i.e. as a or , or basis expansions, i.e. matrix with each row an input vector (with 1 in the first position), and or . as a vector of outputs. We then try to minimize the residual sum- of- squares This is a quadratic function in the parameters. Differentiating with respect to wikicour senote.com/w/index.php?title= Stat841&pr intable= yes we obtain 25/74 10/09/2013 Stat841 - Wiki Cour se Notes Set the first derivative to zero we obtain the solution Thus the fitted values at the inputs are where is called the hat matrix (http://en.wikipedia.org/wiki/Hat_matrix) . Note For classification purposes, this is not a correct model. Recall the following application of Bayes classifier: It is clear that to make sense mathematically, must be a value between 0 and 1. If this is estimated with the regression function and is learned as above, then there is nothing that would restrict to taking values between 0 and 1. This is more direct approach to classification since it do not need to estimate and . This model does not classify Y between 0 and 1, so it is not good and sometimes it can lead to a decent classifier. A line ar re gre s s ion e xample in M atlab We can see how linear regression works through the following example in Matlab. The following is the code an...
View Full Document

## This document was uploaded on 03/07/2014.

Ask a homework question - tutors are online