chap03_8up - Least Squares Data Fitting Existence...

Info icon This preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
Least Squares Data Fitting Existence, Uniqueness, and Conditioning Solving Linear Least Squares Problems Scientific Computing: An Introductory Survey Chapter 3 – Linear Least Squares Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction permitted for noncommercial, educational use only. Michael T. Heath Scientific Computing 1 / 61 Least Squares Data Fitting Existence, Uniqueness, and Conditioning Solving Linear Least Squares Problems Outline 1 Least Squares Data Fitting 2 Existence, Uniqueness, and Conditioning 3 Solving Linear Least Squares Problems Michael T. Heath Scientific Computing 2 / 61 Least Squares Data Fitting Existence, Uniqueness, and Conditioning Solving Linear Least Squares Problems Least Squares Data Fitting Method of Least Squares Measurement errors are inevitable in observational and experimental sciences Errors can be smoothed out by averaging over many cases, i.e., taking more measurements than are strictly necessary to determine parameters of system Resulting system is overdetermined , so usually there is no exact solution In effect, higher dimensional data are projected into lower dimensional space to suppress irrelevant detail Such projection is most conveniently accomplished by method of least squares Michael T. Heath Scientific Computing 3 / 61 Least Squares Data Fitting Existence, Uniqueness, and Conditioning Solving Linear Least Squares Problems Least Squares Data Fitting Linear Least Squares For linear problems, we obtain overdetermined linear system Ax = b , with m × n matrix A , m > n System is better written Ax = b , since equality is usually not exactly satisfiable when m > n Least squares solution x minimizes squared Euclidean norm of residual vector r = b - Ax , min x k r k 2 2 = min x k b - Ax k 2 2 Michael T. Heath Scientific Computing 4 / 61 Least Squares Data Fitting Existence, Uniqueness, and Conditioning Solving Linear Least Squares Problems Least Squares Data Fitting Data Fitting Given m data points ( t i , y i ) , find n -vector x of parameters that gives “best fit” to model function f ( t, x ) , min x m X i =1 ( y i - f ( t i , x )) 2 Problem is linear if function f is linear in components of x , f ( t, x ) = x 1 φ 1 ( t ) + x 2 φ 2 ( t ) + · · · + x n φ n ( t ) where functions φ j depend only on t Problem can be written in matrix form as Ax = b , with a ij = φ j ( t i ) and b i = y i Michael T. Heath Scientific Computing 5 / 61 Least Squares Data Fitting Existence, Uniqueness, and Conditioning Solving Linear Least Squares Problems Least Squares Data Fitting Data Fitting Polynomial fitting f ( t, x ) = x 1 + x 2 t + x 3 t 2 + · · · + x n t n - 1 is linear, since polynomial linear in coefficients, though nonlinear in independent variable t Fitting sum of exponentials f ( t, x ) = x 1 e x 2 t + · · · + x n - 1 e x n t is example of nonlinear problem For now, we will consider only linear least squares problems Michael T. Heath Scientific Computing 6 / 61 Least Squares Data Fitting Existence, Uniqueness, and Conditioning Solving Linear Least Squares Problems Least Squares
Image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern