chap03_8up

# chap03_8up - Least Squares Data Fitting Existence...

This preview shows pages 1–2. Sign up to view the full content.

Least Squares Data Fitting Existence, Uniqueness, and Conditioning Solving Linear Least Squares Problems Scientific Computing: An Introductory Survey Chapter 3 – Linear Least Squares Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction permitted for noncommercial, educational use only. Michael T. Heath Scientific Computing 1 / 61 Least Squares Data Fitting Existence, Uniqueness, and Conditioning Solving Linear Least Squares Problems Outline 1 Least Squares Data Fitting 2 Existence, Uniqueness, and Conditioning 3 Solving Linear Least Squares Problems Michael T. Heath Scientific Computing 2 / 61 Least Squares Data Fitting Existence, Uniqueness, and Conditioning Solving Linear Least Squares Problems Least Squares Data Fitting Method of Least Squares Measurement errors are inevitable in observational and experimental sciences Errors can be smoothed out by averaging over many cases, i.e., taking more measurements than are strictly necessary to determine parameters of system Resulting system is overdetermined , so usually there is no exact solution In effect, higher dimensional data are projected into lower dimensional space to suppress irrelevant detail Such projection is most conveniently accomplished by method of least squares Michael T. Heath Scientific Computing 3 / 61 Least Squares Data Fitting Existence, Uniqueness, and Conditioning Solving Linear Least Squares Problems Least Squares Data Fitting Linear Least Squares For linear problems, we obtain overdetermined linear system Ax = b , with m × n matrix A , m > n System is better written Ax = b , since equality is usually not exactly satisfiable when m > n Least squares solution x minimizes squared Euclidean norm of residual vector r = b - Ax , min x k r k 2 2 = min x k b - Ax k 2 2 Michael T. Heath Scientific Computing 4 / 61 Least Squares Data Fitting Existence, Uniqueness, and Conditioning Solving Linear Least Squares Problems Least Squares Data Fitting Data Fitting Given m data points ( t i , y i ) , find n -vector x of parameters that gives “best fit” to model function f ( t, x ) , min x m X i =1 ( y i - f ( t i , x )) 2 Problem is linear if function f is linear in components of x , f ( t, x ) = x 1 φ 1 ( t ) + x 2 φ 2 ( t ) + · · · + x n φ n ( t ) where functions φ j depend only on t Problem can be written in matrix form as Ax = b , with a ij = φ j ( t i ) and b i = y i Michael T. Heath Scientific Computing 5 / 61 Least Squares Data Fitting Existence, Uniqueness, and Conditioning Solving Linear Least Squares Problems Least Squares Data Fitting Data Fitting Polynomial fitting f ( t, x ) = x 1 + x 2 t + x 3 t 2 + · · · + x n t n - 1 is linear, since polynomial linear in coefficients, though nonlinear in independent variable t Fitting sum of exponentials f ( t, x ) = x 1 e x 2 t + · · · + x n - 1 e x n t is example of nonlinear problem For now, we will consider only linear least squares problems Michael T. Heath Scientific Computing 6 / 61 Least Squares Data Fitting Existence, Uniqueness, and Conditioning Solving Linear Least Squares Problems Least Squares

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern