Chap5.4-Hessian

Chap5.4-Hessian - Machine Learning Srihari The Hessian...

Info iconThis preview shows pages 1–4. Sign up to view the full content.

View Full Document Right Arrow Icon
Machine Learning Srihari The Hessian Matrix Sargur Srihari 1
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Machine Learning Srihari Hessian of Neural Network Error Function Backpropagation can be used to obtain ±rst derivatives of error function wrt weights in network It can also be used to derive second derivatives If all weights and bias parameters are elements w i of single vector w then the second derivatives form the elements H ij of Hessian matrix H where i,j ε { 1,. .W } 2 E w ji w lk
Background image of page 2
Machine Learning Srihari Hessian Matrix Defnition If f is a real-valued function f(x 1 ,..,x n ) And if all real-valued derivatives exist then the Hessian matrix of f is the matrix H(f) ij ( x )=D i D j f( x) where x=( x 1 ,..x n ) and D i is the differential operator wrt the i th variable Hessian matrices are used in large-scale optimization problems within Newton-type methods because they are the coefficient of the quadratic term of a local Taylor expansion of a function. That is,
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 4
This is the end of the preview. Sign up to access the rest of the document.

This document was uploaded on 02/25/2012.

Page1 / 12

Chap5.4-Hessian - Machine Learning Srihari The Hessian...

This preview shows document pages 1 - 4. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online