Spring
10-725 Optimization, Spring 2010: Final Exam
Due: Tuesday, May 4th by noon
Instructions There are 6 questions on this exam. Two questions involve coding. Do not attach your code
to the writeup. Instead, copy your implementation to
/afs/andrew.cmu.e

The geometry of LP
solutions
Optimization - 10725
Carlos Guestrin
Carnegie Mellon University
2008-2010 Carlos Guestrin
January 25th, 2008
1
The Simplex alg walks from
vertex to vertex. Why?
2008-2010 Carlos Guestrin
2
1
Understanding the Geometry of LPs
T

Spring
10-725 Optimization, Spring 2010: Homework 1 Solutions, Yis part
February 22, 2010
1
Convex Sets [Yi, 12 points]
1.1
A slab, i.e., a set of the form cfw_x Rn | aT x .
Solution: A slab is convex, since it is the intersection of two half-spaces cfw_

Spring
10-725 Optimization, Spring 2010: Homework 1 Solutions,
Sivaramans part
April 12, 2010
1
Linear Algebra and Convex Sets [Sivaraman, 8 points]
Solution: The result is known as Carathodorys theorem. While wikipedia has a rigorous proof of the
e
theor

Spring
10-725 Optimization, Spring 2010: Homework 3 Solutions
April 30, 2010
1
Convexity [Sivaraman, 15 points]
1.1
Linear Maps between convex sets
Assume C1 Rn and C2 Rm are both convex for n, m Z+ . Dene S to be the set of all matrices which
correspond

Spring
10-725 Optimization, Spring 2010: Homework 2
Due: Wednesday, February 17, beginning of class
Instructions There are 4 questions on this assignment. The last question involves coding. Do not attach
your code to the writeup. Instead, copy your implem

Spring
10-725 Optimization, Spring 2010: Homework 2 Solutions
March 23, 2010
1
Vertex Cover [Sivaraman, 20 points]
The goal of this problem is to illustrate the use of LPs for approximating NP-hard optimization problems.
We will obtain an approximation to

Convex function
f(x): !n ! ! is convex iff
"
f is strictly convex iff
"
1
Epigraph
Epigraph = points on or above a function
" epi f =
If f is (strictly) convex, then epi f is:
2
Level sets
The k-level set:
The k-sublevel set:
A convex fn has:
Converse?
3

Spring
10-725 Optimization, Spring 2010: Homework 1
Due: Wednesday, February 3, beginning of class
Instructions There are 7 questions on this assignment. The last question involves coding. Do not attach
your code to the writeup. Instead, copy your impleme

http:/select.cs.cmu.edu/class/10725-S10/
Whats all the fuss
about?
Linear Programming
Optimization - 10725
Carlos Guestrin
Carnegie Mellon University
2008-2010 Carlos Guestrin
January 11th, 2010
1
So, how did I get a job at CMU ?
2008-2010 Carlos Guestrin

Review
LPs and linear feasibility problems
!
!
!
!
!
n real variables, possibly bounded
m linear (in)equality constraints
linear objective (in LP only)
matrix notation
feasible, infeasible, unbounded
Inequality form, standard form
! slack variables
Ske

Linear programs
Geoff Gordon
1
Linear programs
n variables:
! ranges:
Objective:
m constraints:
! linear equality:
! linear inequality:
Example:
2
Sketching an LP
max 2x+3y s.t.
" x + y # 4"
" 2x + 5y # 12
" x + 2y # 5"
" x, y $ 0
3
Did the prof get i

LP vs. linear feasibility
min cTx s.t. Ax ! b
! dual: max bTy s.t. ATy = c, y ! 0
nd x s.t. Ax ! b
Monday, March 15, 2010
1
Reminder: separation
oracle
Monday, March 15, 2010
2
Simplied preview: ellipsoid
nd x s.t. Ax ! b
Monday, March 15, 2010
3
Difcult

Spring
10-725 Optimization, Spring 2010: Homework 5
Due: Wednesday, April 14, beginning of class
Instructions There are 6 questions on this assignment. The last question involves coding. Do not attach
your code to the writeup. Instead, copy your implement

Spring
10-725 Optimization, Spring 2010: Homework 5 Solutions
April 30, 2010
1
Conjugate functions [Sivaraman, 15 points]
Many thanks to Yuandong Tian whose solution we have used for this part.
Derive the conjugates of the following functions (3 points ea

Spring
10-725 Optimization, Spring 2010: Homework 4
Due: Wednesday, March 31, beginning of class
Instructions There are 3 questions on this assignment. The last question involves coding. Do not attach
your code to the writeup. Instead, copy your implement

+
Quadratic Programming
and Duality
Sivaraman Balakrishnan
+
Outline
Quadratic Programs
General Lagrangian Duality
Lagrangian Duality in QPs
+
Norm approximation
Problem
min |Ax b|
x
Interpretation
Geometric try to find projection of b into ran(A)
Statist

+
Ellipsoid Algorithm
Sivaraman Balakrishnan
+
Outline
Motivation
Consequences
Brief review of algorithm from class
Some more intuition of what the algorithm is trying to do
Very quick/rough example
+
Motivation
Typical subgradient has two drawbacks
Findi

+
Examples of convex
programs
Sivaraman Balakrishnan
+
Outline
Introduction to convex programming
Example 1 Minimum volume ellipsoid
Example 2 Distance metric learning
Example 3 Graphical Lasso
Efficient algorithms for the Graphical Lasso
+
Convex program

Convex Programs
Programs,
Duality, and
y,
Optimality Conditions
Yi Zhang
Outline
Duals of norms, cones, and functions
Convex programs and duality
Strong duality and Slaters conditions
KKT optimality conditions
Outline
Duals of norms, cones, and functions

Interior Point Methods
Yi Zhang
Outline
Interior point method: basic idea
Log barrier function
Central path
Barrier (interior point) method
A hierarchy of convex optimization
Outline
Interior point method: basic idea
Log barrier function
Central path
Barr

+
Second order methods
Sivaraman Balakrishnan
+
Outline
Big picture (and a little preview of next week)
Example
Smooth unconstrained minimization
KKT review
Equality constrained minimization
+
Big picture
Lets say we were trying to solve an LP
T
min
subje

Spring
10-725 Optimization, Spring 2010: Homework 4 Solutions
April 28, 2010
1
Max-Cut via SDP [Sivaraman, 35 points]
The goal of this problem is to illustrate the use of semidenite programming for approximating NP-hard
optimization problems. We will obta

Spring
10-725 Optimization, Spring 2010: Homework 3
Due: Wednesday, March 17, beginning of class
Instructions There are 4 questions on this assignment. The last question involves coding. Do not attach
your code to the writeup. Instead, copy your implement

What if were lazy
A hard LP:
! min x + y s.t.
" x+y#2
" x, y # 0
Wednesday, February 3, 2010
1
OK, we got lucky
What if it were:
! min x + 3y s.t.
" x+y#2
" x, y # 0
Wednesday, February 3, 2010
2
How general is this?
What if it were:
! min px + qy s.t.
"