Convex Optimization 10-725/36-725
Homework 2, due Oct 3
Instructions:
You must complete Problems 13 and either Problem 4 or Problem 5 (your choice
between the two).
When you submit the homework, upload a single PDF (e.g., produced by LaTeX, or
scanned h

Convex Optimization 10-725/36-725
Homework 5, due Nov 26
Instructions:
You must complete Problems 13 and either Problem 4 or Problem 5 (your choice
between the two).
When you submit the homework, upload a single PDF (e.g., produced by LaTeX, or
scanned

Convex Optimization 10-725/36-725
Homework 5, due Nov 26
Instructions:
You must complete Problems 13 and either Problem 4 or Problem 5 (your choice
between the two).
When you submit the homework, upload a single PDF (e.g., produced by LaTeX, or
scanned

Convex Optimization 10-725/36-725
Homework 4, due Oct 31
Instructions:
You must complete Problems 13 and either Problem 4 or Problem 5 (your choice
between the two).
When you submit the homework, upload a single PDF (e.g., produced by LaTeX, or
scanned

Solving Very Large or
Infinite Problems:
Constraint Generation
Optimization - 10725
Carlos Guestrin
Carnegie Mellon University
2008 Carlos Guestrin
February 6th, 2008
1
Weighted Least-Squares
Least-squares regression problem:
Basis functions:
Find coeffic

Delayed Column
Generation
(aka variable generation)
Optimization - 10725
Carlos Guestrin
Carnegie Mellon University
2008 Carlos Guestrin
February 13th, 2008
1
Why constraint generation converges
LP with many many constraints:
Solve with subset of constrai

Linear Programming:
the geometry of LPs
Optimization - 10725
Carlos Guestrin
Carnegie Mellon University
2008 Carlos Guestrin
January 23rd, 2008
1
Understanding the Geometry of LPs
Todays lecture: Understanding geometry of LPs
Focus on inequality constrain

http:/www.cs.cmu.edu/~guestrin/Class/10725/
Whats all the fuss
about?
Linear Programming
Optimization - 10725
Carlos Guestrin
Carnegie Mellon University
2008 Carlos Guestrin
January 14th, 2008
1
So, how did I get a job at CMU ?
2008 Carlos Guestrin
2
1
Ho

Linear Programming:
problem statement
the geometry of LPs
Optimization - 10725
Carlos Guestrin
Carnegie Mellon University
2008 Carlos Guestrin
January 16th, 2008
1
Maximizing revenue
n products, how much do we produce of each?
Amounts:
Profit for each pro

Solving Very Large or
Infinite Problems:
Constraint Generation
Optimization - 10725
Carlos Guestrin
Carnegie Mellon University
2008-2010 Carlos Guestrin
February 17th, 2010
1
Weighted Least-Squares
Least-squares regression problem:
Basis functions:
Find c

Review
Geometric
interpretation of dual
linear combo of d
independent constrs in
d dimensions: rotate
around common
intersection
rotate until orthogonal
to objective
Sunday, February 14, 2010
1
Review
Properties of dual
how , , = match w/ +ve, -ve, f

Convex Optimization
CMU-10725
3. Linear Programs
Barnabs Pczos & Ryan Tibshirani
Administrivia
Please ask questions!
Slides: http:/www.stat.cmu.edu/~ryantibs/convexopt/
Anonym feedback survey will be on black board today.
Please use it! Constructive fe

Administrivia
1
Simplex Algorithm in 1 Slide
Canonical form:
If we do pivot in Ar,s >0, where cs<0
New cost value:
New b vector:
2
The full Simplex Algorithm
So far we have assumed that a basic feasible solution in canonical form is
available to start the

Gradient descent
Barnabas Poczos & Ryan Tibshirani
Convex Optimization 10-725/36-725
1
Gradient descent
First consider unconstrained minimization of f : Rn R, convex
and dierentiable. We want to solve
min f (x),
xRn
i.e., nd x such that f (x ) = minx f (x

Convex Optimization 10-725/36-725
Homework 3 Solution
Instructions:
You must complete Problems 13 and either Problem 4 or Problem 5 (your choice
between the two).
When you submit the homework, upload a single PDF (e.g., produced by LaTeX, or
scanned han

Convex Optimization 10-725/36-725
Homework 1 Solution, Due Sep 19
Instructions:
You must complete Problems 13 and either Problem 4 or Problem 5 (your choice
between the two).
When you submit the homework, upload a single PDF (e.g., produced by LaTeX, or

Convex Optimization 10-725/36-725
Homework 3, due Oct 17
Instructions:
You must complete Problems 13 and either Problem 4 or Problem 5 (your choice
between the two).
When you submit the homework, upload a single PDF (e.g., produced by LaTeX, or
scanned

Convex Optimization 10-725/36-725
Homework 1, due September 19
Instructions:
You must complete Problems 13 and either Problem 4 or Problem 5 (your choice
between the two).
When you submit the homework, upload a single PDF (e.g., produced by LaTeX, or
sc

Convex Optimization 10-725/36-725
Homework 4 Solutions
Instructions:
You must complete Problems 13 and either Problem 4 or Problem 5 (your choice
between the two).
When you submit the homework, upload a single PDF (e.g., produced by LaTeX, or
scanned ha

Convex Optimization 10-725/36-725
Homework 2, due Oct 3
Instructions:
You must complete Problems 13 and either Problem 4 or Problem 5 (your choice
between the two).
When you submit the homework, upload a single PDF (e.g., produced by LaTeX, or
scanned h

Convex Functions
Optimization - 10725
Carlos Guestrin
Carnegie Mellon University
2008-2010 Carlos Guestrin
March 17th, 2010
1
Convex Functions
Function f:Rn
R is convex if
Domain is convex
Generalization: Jensens inequality:
Strictly convex function:
2008

Linear programs
Geoff Gordon
1
Linear programs
n variables:
ranges:
Objective:
m constraints:
linear equality:
linear inequality:
Example:
2
Sketching an LP
max 2x+3y s.t.
x + y 4
2x + 5y 12
x + 2y 5
x, y 0
3
Did the prof get it right?
4
Matrix

10-725 Optimization, Spring 2010: Homework 4 Solutions
April 28, 2010
1
Max-Cut via SDP [Sivaraman, 35 points]
The goal of this problem is to illustrate the use of semidenite programming for approximating NP-hard
optimization problems. We will obtain a 0.

10-725 Optimization, Spring 2010: Homework 1
Due: Wednesday, February 3, beginning of class
Instructions There are 7 questions on this assignment. The last question involves coding. Do not attach
your code to the writeup. Instead, copy your implementation

10-725 Optimization, Spring 2010: Homework 2
Due: Wednesday, February 17, beginning of class
Instructions There are 4 questions on this assignment. The last question involves coding. Do not attach
your code to the writeup. Instead, copy your implementatio

10-725 Optimization, Spring 2010: Homework 2 Solutions
March 23, 2010
1
Vertex Cover [Sivaraman, 20 points]
The goal of this problem is to illustrate the use of LPs for approximating NP-hard optimization problems.
We will obtain an approximation to the ve

+
Review of Linear Algebra
10-725 - Optimization
1/14/10 Recitation
Sivaraman Balakrishnan
+
Outline
Matrix
subspaces
Linear
independence and bases
Gaussian
Eigen
elimination
values and Eigen vectors
Definiteness
Matlab
essentials
Geoffs LP sketcher
linpr

10-725 Optimization, Spring 2010: Homework 3 Solutions
April 30, 2010
1
Convexity [Sivaraman, 15 points]
1.1
Linear Maps between convex sets
Assume C1 Rn and C2 Rm are both convex for n, m Z+ . Dene S to be the set of all matrices which
correspond to a li

Matrix Calculus and
Algebra
Yi Zhang
Outline
Matrix calculus and algebra
Dimensions of derivatives
Basic calculations of matrix derivatives
Rules for product chain trace determinant and
product, chain, trace,
norms
Matrix Derivatives:
Dimensions
The basic