This preview shows page 1. Sign up to view the full content.
Unformatted text preview: can be found at the University of South Florida
(http://numericalmethods.eng.usf.edu/topics/linear_regression.html) and this MIT lecture (http://academicearth.org/lectures/applications- to- linear- estimation- least- squares) .
For the purpose of classification, the linear regression model assumes that the regression function is linear in the inputs . The simple linear regression model has the general form: where is a vector and is a vector . Given input data
our goal is to find
Squares method (http://en.wikipedia.org/wiki/Least_squares) .
Note that vectors
Denote and such that the linear model fits the data while minimizing sum of squared errors using the Least could be numerical inputs, transformations of the original data, i.e. as a or , or basis expansions, i.e. matrix with each row an input vector (with 1 in the first position), and or . as a vector of outputs. We then try to minimize the residual sum- of- squares This is a quadratic function in the parameters. Differentiating with respect to wikicour senote.com/w/index.php?title= Stat841&pr intable= yes we obtain 25/74 10/09/2013 Stat841 - Wiki Cour se Notes Set the first derivative to zero we obtain the solution Thus the fitted values at the inputs are where is called the hat matrix (http://en.wikipedia.org/wiki/Hat_matrix) . Note For classification purposes, this is not a correct model. Recall the following application of Bayes classifier: It is clear that to make sense mathematically,
must be a value between 0 and 1. If this is estimated with the regression function
learned as above, then there is nothing that would restrict
to taking values between 0 and 1. This is more direct approach to classification since it do not need to
This model does not classify Y between 0 and 1, so it is not good
and sometimes it can lead to a decent classifier.
A line ar re gre s s ion e xample in M atlab
We can see how linear regression works through the following example in Matlab. The following is the code an...
View Full Document
This document was uploaded on 03/07/2014.
- Winter '13