ref1_lec4_quad_form

ref1_lec4_quad_form - Quadratic Functions, Optimization,...

Info iconThis preview shows pages 1–6. Sign up to view the full content.

View Full Document Right Arrow Icon
Quadratic Functions, Optimization, and Quadratic Forms Robert M. Freund February, 2004 1 2004 Massachusetts Institute of Technology.
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
2 1 Quadratic Optimization A quadratic optimization problem is an optimization problem of the form: T (QP) : minimize f ( x ):= 1 x T Qx + c x 2 n s.t. x ∈± . Problems of the form QP are natural models that arise in a variety of settings. For example, consider the problem of approximately solving an over-determined linear system Ax = b , where A has more rows than columns. We might want to solve: (P 1 ) : minimize ² Ax b ² n s.t. x . Now notice that ² Ax b ² 2 = x T A T Ax 2 b T Ax + b T b , and so this problem is equivalent to: (P 1 ) : minimize x T A T Ax 2 b T Ax + b T b n s.t. x , which is in the format of QP. A symmetric matrix is a square matrix Q n × n with the property that Q ij = Q ji for all i, j =1 ,...,n .
Background image of page 2
3 We can alternatively define a matrix Q to be symmetric if Q T = Q. denote the identity matrix (i.e., a matrix with all 1’s on the diagonal and 0’s everywhere else) by I , that is, 10 ... 0 01 0 I = . . . . . . . . . . . . , 00 1 and note that I is a symmetric matrix. The gradient vector of a smooth function f ( x ): ± n →± is the vector of first partial derivatives of f ( x ): ∂f ( x ) ∂x 1 . f ( x ):= . . . ( x ) n The Hessian matrix of a smooth function f ( x ± n is the ma- trix of second partial derivatives. Suppose that f ( x ± n is twice differentiable, and let 2 f ( x ) [ H ( x )] ij := i j . Then the matrix H ( x ) is a symmetric matrix, reflecting the fact that 2 f ( x ) = 2 f ( x ) . i j j i A very general optimization problem is:
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
4 (GP) : minimize f ( x ) n s.t. x ∈± , where f ( x ): ± n →± is a function. We often design algorithms for GP by building a local quadratic model of f ( · )a tag iv enpo in t x x .W e form the gradient f x ) (the vector of partial derivatives) and the Hessian H x ) (the matrix of second partial derivatives), and approximate GP by the following problem which uses the Taylor expansion of f ( x )at x x up to the quadratic term. (P 2 ) : minimize f ˜ ( x ):= f x ) T ( x ¯ 2 ( x ¯ x )( x ¯ x )+ f x 1 x ) T H x ) n s.t. x . This problem is also in the format of QP. Notice in the general model QP that we can always presume that Q is a symmetric matrix, because: 1 T x Qx = x T ( Q + Q T ) x 2 ¯ and so we could replace Q by the symmetric matrix Q := 1 2 ( Q + Q T ). Now suppose that f ( x 1 x T Qx + c T x 2 where Q is symmetric. Then it is easy to see that: f ( x )= Qx + c and H ( x Q.
Background image of page 4
5 Before we try to solve QP, we first review some very basic properties of symmetric matrices. 2 Convexity, Definiteness of a Symmetric Matrix, and Optimality Conditions A function f ( x ): ± n →± is a convex function if n f ( λx +(1 λ ) y ) λf ( x )+(1 λ ) f ( y ) for all x, y ∈± , for all λ [0 , 1] . A function f ( x ) as above is called a strictly convex function if the inequality above is strict for all x ² = y and λ (0 , 1).
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 6
This is the end of the preview. Sign up to access the rest of the document.

This document was uploaded on 06/17/2010.

Page1 / 18

ref1_lec4_quad_form - Quadratic Functions, Optimization,...

This preview shows document pages 1 - 6. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online