cs357-slides-ls - least-squares L Olson Department of...

Info icon This preview shows pages 1–8. Sign up to view the full content.

View Full Document Right Arrow Icon
least-squares L. Olson Department of Computer Science University of Illinois at Urbana-Champaign 1
Image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
polling data Suppose we are given the data { ( x 1 , y 1 ) , ..., ( x n , y n ) } and we want to find a curve that best fits the data. 2
Image of page 2
fitting curves 3
Image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
fitting a line Given n data points { ( x 1 , y i ) , ..., ( x n , y n ) } find a and b such that y i = ax i + b i [ 1 , n ] . In matrix form, find a and b that solves x 1 1 . . . . . . x n 1 " a b # = y 1 . . . y n Systems with more equations than unknowns are called overdetermined 4
Image of page 4
overdetermined systems If A is an m × n matrix, then in general, an m × 1 vector b may not lie in the column space of A . Hence Ax = b may not have an exact solution. Definition The residual vector is r = b - Ax . The least squares solution is given by minimizing the square of the residual in the 2-norm. 5
Image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
normal equations Writing r = ( b - Ax ) and substituting, we want to find an x that minimizes the following function φ ( x ) = || r || 2 2 = r T r = ( b - Ax ) T ( b - Ax ) = b T b - 2 x T A T b + x T A T Ax From calculus we know that the minimizer occurs where φ ( x ) = 0. The derivative is given by φ ( x ) = - 2 A T b + 2 A T Ax = 0 Definition The system of normal equations is given by A T Ax = A T b . 6
Image of page 6
solving normal equations Since the normal equations forms a symmetric system, we can solve by computing the Cholesky factorization A T A = LL T and solving Ly = A T b and L T x = y .
Image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 8
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern