{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

hw6_solutions_update

hw6_solutions_update - MS&E 211 Linear and Nonlinear...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
MS&E 211 Fall 2007 Linear and Nonlinear Optimization Dec 4, 2007 Prof. Yinyu Ye Homework Assignment 6 - Solutions Problem 1 : (a) Since log( x ) is a strictly increasing function, maximizing log( f ( x )) is equivalent to maximizing f ( x ). Thus, the maximum likelihood problem is equivalent to the following maximum log-likelihood problem. maximize x 1 ,x 2 g ( x 1 , x 2 ) = log parenleftBigg M productdisplay i =1 f ( c i x 1 + x 2 ) y i (1 f ( c i x 1 + x 2 )) (1 y i ) parenrightBigg After some manipulation looks like: g ( x 1 , x 2 ) = M summationdisplay i =1 y i log( f ( c 1 x 1 + x 2 )) + (1 y i ) log(1 f ( c 1 x 1 + x 2 )) To show that this problem is a convex problem, we will demonstrate that g ( x 1 , x 2 ) is a concave function by showing that its Hessian is negative semi-definite. Let’s first find expressions for the gradient and Hessian matrix of g ( x 1 , x 2 ). Using the fact that: ∂f ( y ) ∂y = exp( y ) (1 + exp( y )) 2 = f ( y )(1 f ( y )) , we can formulate the partial derivatives with respect to x 1 and x 2 as: ∂g ( x 1 , x 2 ) ∂x 1 = M summationdisplay i =1 y i f ( c i x 1 + x 2 )(1 f ( c i x 1 + x 2 )) c i f ( c i x 1 + x 2 ) + (1 y i ) f ( c i x 1 + x 2 )(1 f ( c i x 1 + x 2 )) c i 1 f ( c i x 1 + x 2 ) = M summationdisplay i =1 y i c i (1 f ( c i x 1 + x 2 )) + ( y i 1) c i f ( c i x 1 + x 2 ) = M summationdisplay i =1 c i ( y i f ( c i x 1 + x 2 ))
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Similarly: ∂g ( x 1 , x 2 ) ∂x 2 = M summationdisplay i =1 ( y i f ( c i x 1 + x 2 )) We can then formulate the second order partial derivatives as: 2 g ( x 1 , x 2 ) ∂x 2 1 = M summationdisplay i =1 c 2 i f ( c i x 1 + x 2 )(1 f ( c i x 1 + x 2 )) 2 g ( x 1 , x 2 ) ∂x 2 2 = M summationdisplay i =1 f ( c i x 1 + x 2 )(1 f ( c i x 1 + x 2 )) 2 g ( x 1 , x 2 ) ∂x 1 ∂x 1 = M summationdisplay i =1 c i f ( c i x 1 + x 2 )(1 f ( c i x 1 + x 2 )) As mentioned in the hint, to show that g ( x 1 , x 2 ) is concave, we need to demonstrate that for all z 1 and z 2 : z 2 1 2 g ( x 1 , x 2 ) ∂x 2 1 + 2 z 1 z 2 2 g ( x 1 , x 2 ) ∂x 1 ∂x 2 + z 2 2 2 g ( x 1 , x 2 ) ∂x 2 2 0 Using the expressions that we just derived: z 2 1 2 g ( x 1 , x 2 ) ∂x 2 1 + 2 z 1 z 2 2 g ( x 1 , x 2 ) ∂x 1 ∂x 2 + z 2 2 2 g ( x 1 , x 2 ) ∂x 2 2 = M summationdisplay i =1 f ( c i x 1 + x 2 )(1 f ( c i x 1 + x 2 ))( z 2 1 c 2 i 2 z 1 z 2 c i z 2 2 ) = M summationdisplay i =1 f ( c i x 1 + x 2 )(1 f ( c i x 1 + x 2 ))( z 1 c i z 2 ) 2 0 , since 0 f ( y ) 1.
Background image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}