Lec05-LinearRegression.pdf - Introduction Maximum Likelihood and Least Squares Solving Regression Linear Models for Regression Jiayu Zhou 1 Department

# Lec05-LinearRegression.pdf - Introduction Maximum...

• 55

This preview shows page 1 - 12 out of 55 pages.

Introduction Maximum Likelihood and Least Squares Solving Regression Linear Models for Regression Jiayu Zhou 1 Department of Computer Science and Engineering Michigan State University East Lansing, MI USA Jiayu Zhou CSE 491 Intro. to Machine Learning 1 / 30 Introduction Maximum Likelihood and Least Squares Solving Regression Table of contents 1 Introduction 2 Maximum Likelihood and Least Squares 3 Solving Regression Jiayu Zhou CSE 491 Intro. to Machine Learning 2 / 30 Introduction Maximum Likelihood and Least Squares Solving Regression Polynomial Curve Fitting Suppose we observe a real-valued input variable x and we wish to use this observation to predict the value of a real-valued target variable t . Jiayu Zhou CSE 491 Intro. to Machine Learning 3 / 30 Introduction Maximum Likelihood and Least Squares Solving Regression Polynomial Curve Fitting Suppose we observe a real-valued input variable x and we wish to use this observation to predict the value of a real-valued target variable t . Polynomial function y ( x , w ) = w 0 + w 1 x + w 2 x 2 + · · · + w M x M Jiayu Zhou CSE 491 Intro. to Machine Learning 3 / 30 Introduction Maximum Likelihood and Least Squares Solving Regression Sum-of-Squares Error Function The values of the coefficients will be determined by fitting the polynomial to the training data. This can be done by minimizing an error function that measures the misfit between the function y ( x , w ), for any given value of w , and the training set data points. The sum of the squares error function: 1 2 N X n =1 ( y ( x n , w ) - t n ) 2 Jiayu Zhou CSE 491 Intro. to Machine Learning 4 / 30 Introduction Maximum Likelihood and Least Squares Solving Regression Sum-of-Squares Error Function The values of the coefficients will be determined by fitting the polynomial to the training data. This can be done by minimizing an error function that measures the misfit between the function y ( x , w ), for any given value of w , and the training set data points. The sum of the squares error function: 1 2 N X n =1 ( y ( x n , w ) - t n ) 2 Let us say we can find w * that minimizes the objective. Jiayu Zhou CSE 491 Intro. to Machine Learning 4 / 30 Introduction Maximum Likelihood and Least Squares Solving Regression How to choose the order M ? Jiayu Zhou CSE 491 Intro. to Machine Learning 5 / 30 Introduction Maximum Likelihood and Least Squares Solving Regression How to choose the order M ? Jiayu Zhou CSE 491 Intro. to Machine Learning 6 / 30 Introduction Maximum Likelihood and Least Squares Solving Regression How to choose the order M ? Jiayu Zhou CSE 491 Intro. to Machine Learning 7 / 30 Introduction Maximum Likelihood and Least Squares Solving Regression How to choose the order M ? Jiayu Zhou CSE 491 Intro. to Machine Learning 8 / 30 Introduction Maximum Likelihood and Least Squares Solving Regression How to choose the order M ?  #### You've reached the end of your free preview.

Want to read all 55 pages?

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern 