p103_6_S10

p103_6_S10 - EE103 Spring 2010 Lecture Notes (SEJ) Section...

Info iconThis preview shows pages 1–4. Sign up to view the full content.

View Full Document Right Arrow Icon
EE103 Spring 2010 Lecture Notes (SEJ) Section 6 i © Copyright Stephen E Jacobsen, 2010 ........................................................................... i SECTION 6: INTRODUCTION TO LEAST SQUARES APPROXIMATION (with Application to Algebraic Polynomial Approximation) .............................................. 112 Linear Least Squares (LLS) ..................................................................................... 112 Example 1 (Algebraic Polynomial linear least squares approximation) .......... 113 Example 2: (Multiple linear regression) ............................................................. 115 Example 3 (Numerical example of algebraic polynomial approximation) ...... 116 Geometric Interpretation and Orthogonality ........................................................ 117 The Gram-Schmidt Process and QR Factorization ............................................... 120 Examples of Classic and Modified Gram-Schmidt, and Choleski Implementations .................................................................................................... 123 The Computational Differences: Gram-Schmidt, and Modified Gram-Schmidt . ....... 129 The Gram-Schmidt process for a matrix A , x mn , () rank A n : . ......................... 129 Gram-Schmidt Process (Normalized) ..................................................................... 129 Modified Gram-Schmidt Process ........................................................................... 131 © Copyright Stephen E Jacobsen, 2010
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
EE103 Spring 2010 Lecture Notes (SEJ) Section 6 112 SECTION 6: INTRODUCTION TO LEAST SQUARES APPROXIMATION (with Application to Algebraic Polynomial Approximation) Consider the system of linear equations A xb where A is mxn, m > n, and the columns of 12 n Aaa a , ,,, , are linearly independent (i.e., rank A = n ). Such a system of equations usually has no solution. We define, for a given n x R , the error vector ex A x b () The minimum norm problem is the problem of choosing an n x R that minimizes the norm of the error, ||() || . For the purposes of these notes, the only norms that are of concern are the following:  1 1 2 2 1 1, , 1. 2. 3. max n i i n t i i i in zz z z  The minimum error problem is, therefore, n xR min ||() || Linear Least Squares (LLS) Least-squares problems are those for which the norm is the Euclidean norm 2 T z || || In this case we may write 22 1 m T i nn n n i ex ex e x   min ( ) ( ) min ( ) That is, if the Euclidean norm is used, we choose an x that minimizes the sum of the squared errors; hence the term “least squares”. Let 2 2 T f xe x e x e x () () () ; we wish to minimize f x and, of course, if x is a minimizer, then x must satisfy the vector equation 0 fx
Background image of page 2
EE103 Spring 2010 Lecture Notes (SEJ) Section 6 113 When this vector equation is linear in the unknowns, x , we have a so-called linear least squares problem. For the remainder of this section, we will focus on the linear least squares problem. Now, 2 22 TT T T t T T f x e x e x Ax b Ax b x A Ax b Ax b b fx xAA bA   () () () ( )( ) () . Therefore, 0 T f xx A A b A  or, by taking the transpose of this latter equation, AA x Ab These equations are called the normal equations , and we'll soon learn of the meaning of that term. T is x nn , nonsingular, and positive definite. Therefore, mathematically 1 x AA Ab (this is a mathematical expression and is not to be used for computation). Since the columns of A are linearly independent, the matrix T is PD (Positive Definite) and therefore Choleski's method may be employed to factor T as AA LL where L is a lower triangular matrix. Given this Choleski factorization, the equation LL x A b
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 4
This is the end of the preview. Sign up to access the rest of the document.

Page1 / 23

p103_6_S10 - EE103 Spring 2010 Lecture Notes (SEJ) Section...

This preview shows document pages 1 - 4. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online