Lecture5b.slides.pdf - CS4487 Machine Learning Lecture 5b...

This preview shows 1 out of 6 pages.

CS4487 - Machine Learning Lecture 5b - Supervised Learning - Regression Dr. Antoni B. Chan Dept. of Computer Science, City University of Hong Kong Outline 1. Linear Regression 2. Selecting Features 3. Removing Outliers 4. Non-linear regression Outliers Too many outliers in the data can a ff ect the squared-error term. regression function will try to reduce the large prediction error for outliers, at the expense of worse prediction for other points In [3]: outfig RANSAC Out[3]:
Image of page 1

Subscribe to view the full document.

RAN dom SA mple C onsensus attempt to robustly fit a regression model in the presence of corrupted data (outliers). works with any regression model. Idea: split the data into inliers (good data) and outliers (bad data). learn the model only from the inliers Random sampling Repeat many times... randomly sample a subset of points from the data. Typically just enough to learn the regression model fit a model to the subset. classify all data as inlier or outlier by calculating the residuals (prediction errors) and comparing to a threshold. The set of inliers is called the consensus set . save the model with the highest number of inliers. Finally, use the largest consensus set to learn the final model. In [5]: ransacfig RANSAC More iterations increases the probability of finding the correct function. higher probability to select a subset of points contains all inliers. Threshold typically set as the median absolute deviation of . y Out[5]:
Image of page 2
In [7]: # use RANSAC model (defaults to linear regression) rlin = linear_model . RANSACRegressor(random_state =1234 ) rlin . fit(outlinX, outlinY) inlier_mask = rlin . inlier_mask_ outlier_mask = logical_not(inlier_mask) plt . figure() plot_regr_trans_1d(rlin, axbox, outlinX, outlinY) plt . plot(outlinX[inlier_mask], outlinY[inlier_mask], 'b.' , label = 'inliers' ) plt . plot(outlinX[outlier_mask], outlinY[outlier_mask], 'co' , label = 'outliers' ) leg = plt . legend(fontsize =8 , loc = 'upper left' ) plt . title( 'estimated (w,b) = ( %0.4g , %0.4g ) \n true (w,b) = ( %0.4g , %0.4g )' % (rlin . estimator_ . coef_, rlin . estimator_ . intercept_, lincoefs, linbias) ); Non-linear regression So far we have only considered linear regression: Similar to classifcation, we can do non-linear regression by forming a feature vector of and then performing linear regression on the feature vector. Polynomial regression p-th order Polynomial function Collect the terms into a vector f ( x ) = x + b w T x f ( x ) = + x + + + + w 0 w 1 w 2 x 2 w 3 x 3 w p x p
Image of page 3

Subscribe to view the full document.

weight vector ; polynomial feature vector: Now it's a linear function, so we can use the same linear regression! Example 1st to 6th order polynomials f ( x ) = [ ] = ϕ ( x ) w 0 w 1 w 2 w p 1 x x 2 x p w T w = w 0 w 1 w 2 w p ϕ ( x ) = 1 x x 2 x p
Image of page 4
In [8]: # example data polyX = random . normal(size =200 ) polyY = sin(polyX) + 0.1* random . normal(size =200 ) polyX = polyX[:,newaxis] plt . figure(figsize = ( 9 , 6 )) axbox = [ -4 , 4 , -2 , 2 ] for d in [ 1 , 2 , 3 , 4 , 5 , 6 ]: # extract polynomial features with degree d polyfeats = preprocessing . PolynomialFeatures(degree = d) polyXf = polyfeats . fit_transform(polyX)
Image of page 5

Subscribe to view the full document.

Image of page 6
You've reached the end of this preview.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern