This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: AIMS Lecture Notes 2006 Peter J. Olver 12. Minimization In this part, we will introduce and solve the most basic mathematical optimization problem: minimize a quadratic function depending on several variables. This will require a short introduction to positive definite matrices. Assuming the coefficient matrix of the quadratic terms is positive definite, the minimizer can be found by solving an associated linear algebraic system. With the solution in hand, we are able to treat a wide range of applications, including least squares fitting of data, interpolation, as well as the finite element method for solvilng boundary value problems for differential equations. 12.1. Positive Definite Matrices. Minimization of functions of several variables relies on an extremely important class of symmetric matrices. Definition 12.1. An n × n matrix K is called positive definite if it is symmetric, K T = K , and satisfies the positivity condition x T K x > for all vectors 6 = x ∈ R n . (12 . 1) We will sometimes write K > 0 to mean that K is a symmetric, positive definite matrix. Warning : The condition K > 0 does not mean that all the entries of K are positive. There are many positive definite matrices that have some negative entries; see Example 12.2 below. Conversely, many symmetric matrices with all positive entries are not positive definite! Remark : Although some authors allow non-symmetric matrices to be designated as positive definite, we will only say that a matrix is positive definite when it is symmetric. But, to underscore our convention and remind the casual reader, we will often include the superfluous adjective “symmetric” when speaking of positive definite matrices. Given any symmetric matrix K , the homogeneous quadratic polynomial q ( x ) = x T K x = n X i,j = 1 k ij x i x j , (12 . 2) 3/15/06 203 c ° 2006 Peter J. Olver is known as a quadratic form on R n . The quadratic form is called positive definite if q ( x ) > for all 6 = x ∈ R n . (12 . 3) Thus, a quadratic form is positive definite if and only if its coefficient matrix is. Example 12.2. Even though the symmetric matrix K = µ 4- 2- 2 3 ¶ has two negative entries, it is, nevertheless, a positive definite matrix. Indeed, the corresponding quadratic form q ( x ) = x T K x = 4 x 2 1- 4 x 1 x 2 + 3 x 2 2 = (2 x 1- x 2 ) 2 + 2 x 2 2 ≥ is a sum of two non-negative quantities. Moreover, q ( x ) = 0 if and only if both 2 x 1- x 2 = 0 and x 2 = 0, which implies x 1 = 0 also. This proves q ( x ) > 0 for all x 6 = , and hence K is indeed a positive definite matrix. On the other hand, despite the fact that K = µ 1 2 2 1 ¶ has all positive entries, it is not a positive definite matrix. Indeed, writing out q ( x ) = x T K x = x 2 1 + 4 x 1 x 2 + x 2 2 , we find, for instance, that q (1 ,- 1) =- 2 < 0, violating positivity. These two simple examples should be enough to convince the reader that the problem of determining whether a given symmetric matrix is or is not positive definite is not completely elementary.a given symmetric matrix is or is not positive definite is not completely elementary....
View Full Document
This note was uploaded on 02/10/2012 for the course MATH 5485 taught by Professor Olver during the Fall '09 term at University of Central Florida.
- Fall '09
- Basic Math