{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

5-leastsquares

# 5-leastsquares - Chapter 5 Least Squares The term least...

This preview shows pages 1–3. Sign up to view the full content.

Chapter 5 Least Squares The term least squares describes a frequently used approach to solving overdeter- mined or inexactly speciﬁed systems of equations in an approximate sense. Instead of solving the equations exactly, we seek only to minimize the sum of the squares of the residuals. The least squares criterion has important statistical interpretations. If ap- propriate probabilistic assumptions about underlying error distributions are made, least squares produces what is known as the maximum-likelihood estimate of the pa- rameters. Even if the probabilistic assumptions are not satisﬁed, years of experience have shown that least squares produces useful results. The computational techniques for linear least squares problems make use of orthogonal matrix factorizations. 5.1 Models and Curve Fitting A very common source of least squares problems is curve ﬁtting. Let t be the independent variable and let y ( t ) denote an unknown function of t that we want to approximate. Assume there are m observations , i.e., values of y measured at speciﬁed values of t : y i = y ( t i ) , i = 1 ,...,m. The idea is to model y ( t ) by a linear combination of n basis functions : y ( t ) β 1 φ 1 ( t ) + ··· + β n φ n ( t ) . The design matrix X is a rectangular matrix of order m by n with elements x i,j = φ j ( t i ) . The design matrix usually has more rows than columns. In matrix-vector notation, the model is y Xβ. February 15, 2008 1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
2 Chapter 5. Least Squares The symbol stands for “is approximately equal to.” We are more precise about this in the next section, but our emphasis is on least squares approximation. The basis functions φ j ( t ) can be nonlinear functions of t , but the unknown parameters, β j , appear in the model linearly. The system of linear equations y is overdetermined if there are more equations than unknowns. The Matlab back- slash operator computes a least squares solution to such a system. beta = X\y The basis functions might also involve some nonlinear parameters, α 1 ,...,α p . The problem is separable if it involves both linear and nonlinear parameters: y ( t ) β 1 φ 1 ( t,α ) + ··· + β n φ n ( t,α ) . The elements of the design matrix depend upon both t and α : x i,j = φ j ( t i ) . Separable problems can be solved by combining backslash with the Matlab func- tion fminsearch or one of the nonlinear minimizers available in the Optimization Toolbox. The new Curve Fitting Toolbox provides a graphical interface for solving nonlinear ﬁtting problems. Some common models include the following: Straight line: If the model is also linear in t , it is a straight line: y ( t ) β 1 t + β 2 .
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 27

5-leastsquares - Chapter 5 Least Squares The term least...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online