Chap5.4-Hessian

# Chap5.4-Hessian - Machine Learning Srihari The Hessian...

This preview shows pages 1–4. Sign up to view the full content.

Machine Learning Srihari The Hessian Matrix Sargur Srihari 1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Machine Learning Srihari Hessian of Neural Network Error Function Backpropagation can be used to obtain ±rst derivatives of error function wrt weights in network It can also be used to derive second derivatives If all weights and bias parameters are elements w i of single vector w then the second derivatives form the elements H ij of Hessian matrix H where i,j ε { 1,. .W } 2 E w ji w lk
Machine Learning Srihari Hessian Matrix Defnition If f is a real-valued function f(x 1 ,..,x n ) And if all real-valued derivatives exist then the Hessian matrix of f is the matrix H(f) ij ( x )=D i D j f( x) where x=( x 1 ,..x n ) and D i is the differential operator wrt the i th variable Hessian matrices are used in large-scale optimization problems within Newton-type methods because they are the coefficient of the quadratic term of a local Taylor expansion of a function. That is,

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

## This document was uploaded on 02/25/2012.

### Page1 / 12

Chap5.4-Hessian - Machine Learning Srihari The Hessian...

This preview shows document pages 1 - 4. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online