This preview shows page 1. Sign up to view the full content.
Unformatted text preview: (t f(x)) = t xf(x) Two important notes Af(A) is always the same as the size of A the gradient of f is defined only if f is a real-valued function e.g. we can't take the gradient of f=2A with respect to A 14 The Hessian Definition Function f: Rn R x: an nx1 vector The Hessian matrix with respect to x (written as 2f(x)) x is an n n matrix and each element of the matrix is a partial derivative defined by 15 The Hessian Example x: a 2x1 vector f(x)= calculate each element of 2f(x) x 16 The Hessian Example The Hessian matrix of f In general, if f(x) = xTAx and A Sn, 17 The Hessian Some notes The Hessian is defined only when f(x) is real-valued. Hessian is always symmetric. We will only consider taking the Hessian with respect to a vector. The Hessian is not the gradient of the gradient. Some useful results T xb x = b T xx Ax = 2Ax (if A symmetric) 2xTAx = 2A (if A symmetric) x However, the gradient of the ith entry of row) of 2f(x). x xf(x) is the ith column (or 18 Applica*on in Least Squares Op*miza*on The problem Given a full-ranked matrix A Rmn and a vector b Rm Suppose there is no x such that Ax=b. Find a vector x Rn, such that the square of the Euclidean norm ||Ax - b||2 is minimized. 2 Solve the problem Take the gradient with respect to x Set the gradient to zero (vector) and we get the solution 19 20...
View Full Document
This note was uploaded on 01/22/2013 for the course EE 103 taught by Professor Vandenberghe,lieven during the Fall '08 term at UCLA.
- Fall '08