cs357-slides-ls

# cs357-slides-ls - least-squares L Olson Department of...

• 21

This preview shows pages 1–8. Sign up to view the full content.

least-squares L. Olson Department of Computer Science University of Illinois at Urbana-Champaign 1

This preview has intentionally blurred sections. Sign up to view the full version.

polling data Suppose we are given the data { ( x 1 , y 1 ) , ..., ( x n , y n ) } and we want to find a curve that best fits the data. 2
fitting curves 3

This preview has intentionally blurred sections. Sign up to view the full version.

fitting a line Given n data points { ( x 1 , y i ) , ..., ( x n , y n ) } find a and b such that y i = ax i + b i [ 1 , n ] . In matrix form, find a and b that solves x 1 1 . . . . . . x n 1 " a b # = y 1 . . . y n Systems with more equations than unknowns are called overdetermined 4
overdetermined systems If A is an m × n matrix, then in general, an m × 1 vector b may not lie in the column space of A . Hence Ax = b may not have an exact solution. Definition The residual vector is r = b - Ax . The least squares solution is given by minimizing the square of the residual in the 2-norm. 5

This preview has intentionally blurred sections. Sign up to view the full version.

normal equations Writing r = ( b - Ax ) and substituting, we want to find an x that minimizes the following function φ ( x ) = || r || 2 2 = r T r = ( b - Ax ) T ( b - Ax ) = b T b - 2 x T A T b + x T A T Ax From calculus we know that the minimizer occurs where φ ( x ) = 0. The derivative is given by φ ( x ) = - 2 A T b + 2 A T Ax = 0 Definition The system of normal equations is given by A T Ax = A T b . 6
solving normal equations Since the normal equations forms a symmetric system, we can solve by computing the Cholesky factorization A T A = LL T and solving Ly = A T b and L T x = y .

This preview has intentionally blurred sections. Sign up to view the full version.

This is the end of the preview. Sign up to access the rest of the document.
• Fall '12
• Zilles
• Computer Architecture, Singular value decomposition, Linear least squares, normal equations, gram-schmidt orthogonalization, overdetermined systems

{[ snackBarMessage ]}

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern