This preview shows pages 1–3. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: 10725 Optimization, Spring 2008: Homework 3 Solutions April 2, 2008 1 Convexity [Han, 30 points] (Acknowledgement: The solutions for question 1 are adapted from the submitted solution of Ajit Paul Singh) 1.1 Linear Maps between convex sets Assume C 1 ⊂ R n and C 2 ⊂ R m are both convex for n, m ∈ Z + . Define S to be the set of all matrices which correspond to a linear map from C 1 to C 2 , i.e. S = { A ∈ R m × n : s . t . ∀ x ∈ C 1 ,Ax ∈ C 2 } 1. [3 pts] Prove that S is a convex set. F SOLUTION: S is convex if any convex combination of A 1 ,A 2 ∈ S is a linear map from C 1 to C 2 . For θ ∈ [0 , 1] ( θA 1 + (1 θ ) A 2 ) x = θA 1 x + (1 θ ) A 2 x, where A 1 x ∈ C 2 and A 2 x ∈ C 2 . Since C 2 is a convex set it must also contain the point θA 1 x +(1 θ ) A 2 x , and so θA 1 + (1 θ ) A 2 is a linear map from C 1 to C 2 . 2. [10 pts] Let H be a separation oracle for the set C 2 . Thus, for a point y ∈ R m , y / ∈ C 2 , H ( y ) returns a hyperplane { z  h ( z ) = 0 } , where the linear function h ( z ) is defined as h ( z ) = a T z + b , where a,z ∈ R m and b ∈ R . The returned hyperplane H ( y ) is such that h ( y ) ≥ 0 and h ( y ) < , ∀ y ∈ C 2 . Note that if y ∈ C 2 , H ( y ) = ∅ . Let C 1 be the convex hull of k points { x 1 ,. .. ,x k } in R n , where k ≥ 3. Using this definition of C 1 and the separation oracle H for the set C 2 , explicitly construct a separation oracle for the convex set S . F SOLUTION: To define a hyperplane in R m × n we need an inner product h· , ·i on the space. For two matrices U,V ∈ R m × n the matrix inner product is h U,V i = trace( U T V ) . For consistency with between matrix and vector inner products we denote h a,x i = trace( a T x ) = a T x . All points in the convex hull are convex combinations of the vertices. Since linear maps preserve convexity this means any A that maps ( x 1 ,. .. ,x k ) ∈ C 1 into C 2 maps all of C 1 onto C 2 . Let ( a i ,b i ) denote the separation oracle for hull vertex x i such that Ax i / ∈ C 2 . For any such point the separation oracle H ( x i ) yields the constraint a T i Ax i + b ≥ , which is equivalent to trace( a T i Ax i ) + b i ≥ . Using trace equivalence it is easy to show that trace( a T i Ax i ) = trace( a i x i A ) where ¯ A = a i x i is n × m outer product of a i and x i . By the definition of a hyperplane in R m × n it follows that ( ¯ A,b i ) is a separation oracle for A . 1 1.2 Convex Sets and Convex Functions In the following, you will be asked to prove certain functions or sets are convex 1. [3 pts] Let C ⊂ R n be a convex set, with x 1 ,. .. ,x k ∈ C , and let θ 1 ,. .. ,θ k ∈ { } ∪ R + satisfy θ 1 + .. . + θ k = 1. Show that θ 1 x 1 + .. . + θ k x k ∈ C . (Hint: use mathematical induction) F SOLUTION: We prove by induction on k . For k = 1 the statement is trivially true since θ 1 = 1 ....
View
Full
Document
This note was uploaded on 05/25/2008 for the course MACHINE LE 10708 taught by Professor Carlosgustin during the Spring '07 term at Carnegie Mellon.
 Spring '07
 CarlosGustin

Click to edit the document details