Lec05-LinearRegression.pdf - Introduction Maximum Likelihood and Least Squares Solving Regression Linear Models for Regression Jiayu Zhou 1 Department

Lec05-LinearRegression.pdf - Introduction Maximum...

This preview shows page 1 - 12 out of 55 pages.

Introduction Maximum Likelihood and Least Squares Solving Regression Linear Models for Regression Jiayu Zhou 1 Department of Computer Science and Engineering Michigan State University East Lansing, MI USA Jiayu Zhou CSE 491 Intro. to Machine Learning 1 / 30
Image of page 1
Introduction Maximum Likelihood and Least Squares Solving Regression Table of contents 1 Introduction 2 Maximum Likelihood and Least Squares 3 Solving Regression Jiayu Zhou CSE 491 Intro. to Machine Learning 2 / 30
Image of page 2
Introduction Maximum Likelihood and Least Squares Solving Regression Polynomial Curve Fitting Suppose we observe a real-valued input variable x and we wish to use this observation to predict the value of a real-valued target variable t . Jiayu Zhou CSE 491 Intro. to Machine Learning 3 / 30
Image of page 3
Introduction Maximum Likelihood and Least Squares Solving Regression Polynomial Curve Fitting Suppose we observe a real-valued input variable x and we wish to use this observation to predict the value of a real-valued target variable t . Polynomial function y ( x , w ) = w 0 + w 1 x + w 2 x 2 + · · · + w M x M Jiayu Zhou CSE 491 Intro. to Machine Learning 3 / 30
Image of page 4
Introduction Maximum Likelihood and Least Squares Solving Regression Sum-of-Squares Error Function The values of the coefficients will be determined by fitting the polynomial to the training data. This can be done by minimizing an error function that measures the misfit between the function y ( x , w ), for any given value of w , and the training set data points. The sum of the squares error function: 1 2 N X n =1 ( y ( x n , w ) - t n ) 2 Jiayu Zhou CSE 491 Intro. to Machine Learning 4 / 30
Image of page 5
Introduction Maximum Likelihood and Least Squares Solving Regression Sum-of-Squares Error Function The values of the coefficients will be determined by fitting the polynomial to the training data. This can be done by minimizing an error function that measures the misfit between the function y ( x , w ), for any given value of w , and the training set data points. The sum of the squares error function: 1 2 N X n =1 ( y ( x n , w ) - t n ) 2 Let us say we can find w * that minimizes the objective. Jiayu Zhou CSE 491 Intro. to Machine Learning 4 / 30
Image of page 6
Introduction Maximum Likelihood and Least Squares Solving Regression How to choose the order M ? Jiayu Zhou CSE 491 Intro. to Machine Learning 5 / 30
Image of page 7
Introduction Maximum Likelihood and Least Squares Solving Regression How to choose the order M ? Jiayu Zhou CSE 491 Intro. to Machine Learning 6 / 30
Image of page 8
Introduction Maximum Likelihood and Least Squares Solving Regression How to choose the order M ? Jiayu Zhou CSE 491 Intro. to Machine Learning 7 / 30
Image of page 9
Introduction Maximum Likelihood and Least Squares Solving Regression How to choose the order M ? Jiayu Zhou CSE 491 Intro. to Machine Learning 8 / 30
Image of page 10
Introduction Maximum Likelihood and Least Squares Solving Regression How to choose the order M ?
Image of page 11
Image of page 12

You've reached the end of your free preview.

Want to read all 55 pages?

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern

Stuck? We have tutors online 24/7 who can help you get unstuck.
A+ icon
Ask Expert Tutors You can ask 0 bonus questions You can ask 10 questions (10 expire soon) You can ask 10 questions (will expire )
Answers in as fast as 15 minutes