hw6_solutions_update

hw6_solutions_update - MS&E 211 Fall 2007 Linear and...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: MS&E 211 Fall 2007 Linear and Nonlinear Optimization Dec 4, 2007 Prof. Yinyu Ye Homework Assignment 6 - Solutions Problem 1 : (a) Since log( x ) is a strictly increasing function, maximizing log( f ( x )) is equivalent to maximizing f ( x ). Thus, the maximum likelihood problem is equivalent to the following maximum log-likelihood problem. maximize x 1 ,x 2 g ( x 1 , x 2 ) = log parenleftBigg M productdisplay i =1 f ( c i x 1 + x 2 ) y i (1 f ( c i x 1 + x 2 )) (1 y i ) parenrightBigg After some manipulation looks like: g ( x 1 , x 2 ) = M summationdisplay i =1 y i log( f ( c 1 x 1 + x 2 )) + (1 y i ) log(1 f ( c 1 x 1 + x 2 )) To show that this problem is a convex problem, we will demonstrate that g ( x 1 , x 2 ) is a concave function by showing that its Hessian is negative semi-definite. Lets first find expressions for the gradient and Hessian matrix of g ( x 1 , x 2 ). Using the fact that: f ( y ) y = exp( y ) (1 + exp( y )) 2 = f ( y )(1 f ( y )) , we can formulate the partial derivatives with respect to x 1 and x 2 as: g ( x 1 , x 2 ) x 1 = M summationdisplay i =1 y i f ( c i x 1 + x 2 )(1 f ( c i x 1 + x 2 )) c i f ( c i x 1 + x 2 ) + (1 y i ) f ( c i x 1 + x 2 )(1 f ( c i x 1 + x 2 )) c i 1 f ( c i x 1 + x 2 ) = M summationdisplay i =1 y i c i (1 f ( c i x 1 + x 2 )) + ( y i 1) c i f ( c i x 1 + x 2 ) = M summationdisplay i =1 c i ( y i f ( c i x 1 + x 2 )) Similarly: g ( x 1 , x 2 ) x 2 = M summationdisplay i =1 ( y i f ( c i x 1 + x 2 )) We can then formulate the second order partial derivatives as: 2 g ( x 1 , x 2 ) x 2 1 = M summationdisplay i =1 c 2 i f ( c i x 1 + x 2 )(1 f ( c i x 1 + x 2 )) 2 g ( x 1 , x 2 ) x 2 2 = M summationdisplay i =1 f ( c i x 1 + x 2 )(1 f ( c i x 1 + x 2 )) 2 g ( x 1 , x 2 ) x 1 x 1 = M summationdisplay i =1 c i f ( c i x 1 + x 2 )(1 f ( c i x 1 + x 2 )) As mentioned in the hint, to show that g ( x 1 , x 2 ) is concave, we need to demonstrate that for all z 1 and z 2 : z 2 1 2 g ( x 1 , x 2 ) x 2 1 + 2 z 1 z 2 2 g ( x 1 , x 2 ) x 1 x 2 + z 2 2 2 g ( x 1 , x 2 ) x 2 2 Using the expressions that we just derived: z 2 1 2 g ( x 1 , x 2 ) x 2 1 + 2 z 1 z 2 2...
View Full Document

This note was uploaded on 06/16/2010 for the course MS&E 211 taught by Professor Yinyuye during the Fall '07 term at Stanford.

Page1 / 7

hw6_solutions_update - MS&E 211 Fall 2007 Linear and...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online