Lecture 5 - 5 Least Squares Problems Consider the solution...

Info icon This preview shows pages 1–2. Sign up to view the full content.

5 Least Squares Problems Consider the solution of A x = b , where A C m × n with m > n . In general, this system is overdetermined and no exact solution is possible. Example Fit a straight line to 10 measurements. If we represent the line by f ( x ) = mx + c and the 10 pieces of data are { ( x 1 , y 1 ) , . . . , ( x 10 , y 10 ) } , then the constraints can be sumamrized in the linear system x 1 1 x 2 1 . . . . . . x 10 1 A m c x = y 1 y 2 . . . y 10 b . This type of problem is known as linear regression or (linear) least squares fitting . The basic idea (due to Gauss) is to minimize the 2-norm of the residual vector , i.e., b - A x 2 . In other words, we want to find x C n such that m i =1 [ b i - ( A x ) i ] 2 is minimized. For the notation used in the example above we want to find m and c such that 10 i =1 [ y i - ( mx i + c ))] 2 is minimized. Example We can generalize the previous example to polynomial least squares fitting of arbitrary degree. To this end we assume that p ( x ) = n i =0 c i x i , where n is the degree of the polynomial. We can fit a polynomial of degree n to m > n data points ( x i , y i ), i = 1 , . . . , m , using the least squares approach, i.e., min m i =1 [ y i - p ( x i )] 2 is used as constraint for the overdetermined linear system A x = b with A = 1 x 1 x 2 1 . . . x n 1 1 x 2 x 2 2 . . . x n 2 . . . . . . . . . . . . 1 x m x 2 m . . . x n m , x = c 0 c 1 c 2 . . . c n , b = y 1 y 2 . . . y m . Remark The special case n = m - 1 is called interpolation and is known to have a unique solutions if the conditions are independent, i.e., the points x i are distinct. How- ever, for large degrees n we frequently observe severe oscillations which is undesirable.
Image of page 1

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern