ECE3040 Chapter 6.pdf - Chapter 6 The lecture notes are selected from the public Powerpoint lecture documents provided by Authors Autar Kaw Sri Harsha

ECE3040 Chapter 6.pdf - Chapter 6 The lecture notes are...

This preview shows page 1 - 13 out of 68 pages.

Chapter 6 The lecture notes are selected from the public Powerpoint lecture documents provided by Authors: Autar Kaw, Sri Harsha Garapati
Image of page 1

Subscribe to view the full document.

Linear Regression
Image of page 2
What is Regression? What is regression? Given n data points best fit ) ( x f y to the data. Residual at each point is ) ( x f y Figure. Basic model for regression ) , ( ), ...... , , ( ), , ( 2 2 1 1 n n y x y x y x ) ( i i i x f y E y x ) , ( 1 1 y x ) , ( n n y x ) , ( i i y x ) ( i i i x f y E
Image of page 3

Subscribe to view the full document.

Linear Regression-Criterion#1 Given n data points best fit x a a y 1 0 to the data. Does minimizing n i i E 1 work as a criterion? x x a a y 1 0 ) , ( 1 1 y x ) , ( 2 2 y x ) , ( 3 3 y x ) , ( n n y x ) , ( i i y x i i i x a a y E 1 0 y Figure. Linear regression of y vs x data showing residuals at a typical point, x i . ) , ( ), ...... , , ( ), , ( 2 2 1 1 n n y x y x y x
Image of page 4
Example for Criterion#1 x y 2.0 4.0 3.0 6.0 2.0 6.0 3.0 8.0 Example: Given the data points (2,4), (3,6), (2,6) and (3,8), best fit the data to a straight line using Criterion#1 Figure. Data points for y vs x data. Table. Data Points 0 2 4 6 8 10 0 1 2 3 4 y x Minimize n i i E 1
Image of page 5

Subscribe to view the full document.

Linear Regression-Criteria#1 0 4 1 i i E x y y predicted E = y - y predicted 2.0 4.0 4.0 0.0 3.0 6.0 8.0 -2.0 2.0 6.0 4.0 2.0 3.0 8.0 8.0 0.0 Table. Residuals at each point for regression model y =4 x − 4 Figure. Regression curve y =4 x − 4 and y vs x data 0 2 4 6 8 10 0 1 2 3 4 y x Using y =4 x − 4 as the regression curve
Image of page 6
Linear Regression-Criterion#1 x y y predicted E = y - y predicted 2.0 4.0 6.0 -2.0 3.0 6.0 6.0 0.0 2.0 6.0 6.0 0.0 3.0 8.0 6.0 2.0 0 4 1 i i E 0 2 4 6 8 10 0 1 2 3 4 y x Table. Residuals at each point for regression model y =6 Figure. Regression curve y =6 and y vs x data Using y =6 as a regression curve
Image of page 7

Subscribe to view the full document.

Linear Regression Criterion #1 0 4 1 i i E for both regression models of y= 4 x- 4 and y= 6 The sum of the residuals is minimized, in this case it is zero, but the regression model is not unique. Hence the criterion of minimizing the sum of the residuals is a bad criterion. 0 2 4 6 8 10 0 1 2 3 4 y x
Image of page 8
Linear Regression-Criterion#1 0 4 1 i i E x y y predicted E = y - y predicted 2.0 4.0 4.0 0.0 3.0 6.0 8.0 -2.0 2.0 6.0 4.0 2.0 3.0 8.0 8.0 0.0 Table. Residuals at each point for regression model y =4 x − 4 Figure. Regression curve y= 4 x- 4 and y vs x data 0 2 4 6 8 10 0 1 2 3 4 y x Using y =4 x − 4 as the regression curve
Image of page 9

Subscribe to view the full document.

Linear Regression-Criterion#2 x ) , ( 1 1 y x ) , ( 2 2 y x ) , ( 3 3 y x ) , ( n n y x ) , ( i i y x i i i x a a y E 1 0 y Figure. Linear regression of y vs. x data showing residuals at a typical point, x i . Will minimizing | | 1 n i i E work any better? x a a y 1 0
Image of page 10
Example for Criterion#2 x y 2.0 4.0 3.0 6.0 2.0 6.0 3.0 8.0 Example: Given the data points (2,4), (3,6), (2,6) and (3,8), best fit the data to a straight line using Criterion#2 Figure. Data points for y vs. x data. Table. Data Points 0 2 4 6 8 10 0 1 2 3 4 y x Minimize n i i E 1 | |
Image of page 11

Subscribe to view the full document.

Linear Regression-Criterion#2 4 | | 4 1 i i E x y y predicted E = y - y predicted 2.0 4.0 4.0 0.0 3.0 6.0 8.0 -2.0 2.0 6.0 4.0 2.0 3.0 8.0 8.0 0.0 Table. Residuals at each point for regression model y =4 x − 4 Figure. Regression curve y= y =4 x − 4 and y vs.
Image of page 12
Image of page 13
  • Fall '17
  • le yi wang

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern

Ask Expert Tutors You can ask 0 bonus questions You can ask 0 questions (0 expire soon) You can ask 0 questions (will expire )
Answers in as fast as 15 minutes