UC Berkeley Department of Electrical Engineering and Computer Science EECS 227A Nonlinear and Convex Optimization Solutions 1 Fall 2009 The rst part of this problem set provides some practice on the mathematical pre-requisites for this course (vector calc
UC Berkeley
Department of Electrical Engineering and Computer Science
EECS 227A
Nonlinear and Convex Optimization
Problem Set 2
Fall 2009
Issued: Tuesday, September 8
Due: Tuesday, September 22, 2009
Reading: Sections 1.21.3 of Nonlinear programming by Be
EE 127 Discussion 1
February 4, 2015
A Markov chain is a nite state machine (a collection of states and transitions between states)
where transitions between states are selected randomly at each time step. For example, in the
Markov chain shown, each mont
Notes
Optimization Models
EE 127 / EE 227AT
Laurent El Ghaoui
EECS department
UC Berkeley
Spring 2015
Sp15
1 / 23
Notes
LECTURE 14
Robust Optimization Models
Each problem that I solved became
a rule which served afterwards to
solve other problems.
Ren Des
Notes
Optimization Models
EE 127 / EE 227AT
Laurent El Ghaoui
EECS department
UC Berkeley
Spring 2015
Sp15
1 / 46
Notes
LECTURE 2
Vectors and Functions
Mathematicians are like
Frenchmen: whatever you say to
them, they translate into their own
language, an
Notes
Optimization Models
EE 127 / EE 227AT
Laurent El Ghaoui
EECS department
UC Berkeley
Spring 2015
Sp15
1 / 37
Notes
LECTURE 12
Linear and Quadratic Programs
I want to emphasize again that the
greater part of the problems of
which I shall speak, relati
Notes
Optimization Models
EE 127 / EE 227AT
Laurent El Ghaoui
EECS department
UC Berkeley
Spring 2015
Sp15
1 / 27
Notes
LECTURE 5
Singular Value Decomposition
The license plate of Gene Golub
(19322007).
Sp15
2 / 27
Outline
1
The singular value decompositi
Notes
Optimization Models
EE 127 / EE 227AT
Laurent El Ghaoui
EECS department
UC Berkeley
Spring 2015
Sp15
1 / 19
Notes
LECTURE 13
Second-Order Cone Models
Each problem that I solved became
a rule which served afterwards to
solve other problems.
Ren Desca
Optimization Models
EE 127 / EE 227AT
Laurent El Ghaoui
EECS department
UC Berkeley
Spring 2015
LECTURE 13
Applications of Duality
XXX
XXX
Outline
Duality
What well do
Review some practical applications of duality
Decomposition methods
Closed-form solutio
Optimization Models
EE 127 / EE 227AT
Laurent El Ghaoui
EECS department
UC Berkeley
Spring 2015
Sp15
1 / 32
LECTURE 11
Convex Optimization Problems
All truth passes through three
stages: First, it is ridiculed;
Second, it is violently opposed;
Third, it i
Optimization Models
EE 127 / EE 227AT
Laurent El Ghaoui
EECS department
UC Berkeley
Spring 2015
LECTURE 13
Duality
All truth passes through three
stages: First, it is ridiculed;
Second, it is violently opposed;
Third, it is accepted as
self-evident.
Arthu
Notes
Optimization Models
EE 127 / EE 227AT
Laurent El Ghaoui
EECS department
UC Berkeley
Spring 2015
Sp15
1 / 20
Notes
LECTURE 18
Subgradients
The gradient does not exist,
implying that the function may
have kinks or corner points, and
thus cannot be app
Notes
Optimization Models
EE 127 / EE 227AT
Laurent El Ghaoui
EECS department
UC Berkeley
Spring 2015
Sp15
1 / 18
Notes
LECTURE 6
Linear Equations
One pint of good wine costs 50
gold pieces, while one pint of poor
wine costs 10. Two pints of wine
are boug
EE 127 Discussion 3
February 11, 2015
Today we will be exploring a number of closed form expressions for solving least squares.
1
Exercise
Let A be a full rank matrix (either row- or column-). Find the solution to the regularized least
squares problem:
mi
EE 127 Discussion 1
January 27, 2015
1
Exercise
Given
A=
1
1
3
1
,
(1)
nd an orthogonal basis for the columns of A using the Gram-Schmidt procedure.
2
Exercise
A QR-decomposition of a matrix A Rmn is one which writes A = QR, where Q Rmm is
an orthonormal
A Review of Linear Algebra
Introduction
ThisisalistofsectionsinDavidLays
LinearAlgebraandItsApplications
textbook,usedfor
Math54atUCBerkeley.Thesectionsselectedarethosewithrelevantbackgroundmaterial
forEE127.Asarecommendationforreadingthis,skipsectionstha
EE127A
L. El Ghaoui
YOUR NAME HERE: SOLUTIONS
YOUR SID HERE: 42
3/27/09
Midterm 1 Solutions
The exam is open notes, but access to the Internet is not allowed. The maximum grade is 20.
When asked to prove something, do not merely take an example; provide a
EE127A
L. El Ghaoui
3/19/10
Midterm Solutions
1. (4 points) Consider the set in R3 , dened by the equation
P := x R3 : x1 + 2x2 + 3x3 = 1 .
(a) Show that the set P is an ane subspace of dimension 2. To this end, express it
as x0 + span(x1 , x2 ), where x0
EE127A
L. El Ghaoui
3/19/11
Midterm Solutions
1. (6 points) Find the projection z of the vector x = (2, 1) on the line that passes through
x0 = (1, 2) and with direction given by the vector u = (1, 1).
Solution: The line is the set
L = cfw_x0 + tu : t R =
EE127A
L. El Ghaoui
9/28/11
Quiz 1: Solutions
1. Consider the matrix A = uv T , with u Rn , v Rm .
(a) Find the nullspace and range of A.
(b) Explain how to compute an SVD of A.
Solutions: We assume u = 0, v = 0 to avoid trivialities.
(a) For the nullspac
EE127A
L. El Ghaoui
5/9/11
EE 127A Final: Solutions
NAME:
SID:
The exam lasts 3 hours. The maximum number of points is 50. Notes are not allowed except
for a two-sided cheat sheet of regular format.
This booklet is 17 pages total, with extra blank spaces
Notes
Optimization Models
EE 127 / EE 227AT
Laurent El Ghaoui
EECS department
UC Berkeley
Spring 2015
Sp15
1 / 15
Notes
LECTURE 17
Geometric Programs
There is geometry in the humming
of the strings, there is music in the
spacing of the spheres.
Pythagoras
Notes
Optimization Models
EE 127 / EE 227AT
Laurent El Ghaoui
EECS department
UC Berkeley
Spring 2015
Sp15
1 / 31
Notes
LECTURE 16
Semidenite Programming Models
Theory is when you know
something, but it doesnt work.
Practice is when something works,
but y
Optimization Models
EE 127 / EE 227AT
Laurent El Ghaoui
EECS department
UC Berkeley
Spring 2015
LECTURE 12
Subgradients
The gradient does not exist,
implying that the function may
have kinks or corner points, and
thus cannot be approximated
locally by a t
Notes
Optimization Models
EE 127 / EE 227AT
Laurent El Ghaoui
EECS department
UC Berkeley
Spring 2015
Sp15
1 / 23
Notes
LECTURE 7
Least Squares and Variants
If others would but reect on
mathematical truths as deeply and
continuously as I have, they would
Notes
Optimization Models
EE 127 / EE 227AT
Laurent El Ghaoui
EECS department
UC Berkeley
Spring 2015
Sp15
1 / 40
Notes
LECTURE 3
Matrices and Linear Maps
The Matrix is everywhere. It is all
around us.
Morpheus
Sp15
2 / 40
Outline
1
Introduction
Basics
2
UC Berkeley Department of Electrical Engineering and Computer Science EECS 227A Nonlinear and Convex Optimization Problem Set 4 Fall 2009 Issued: Tuesday, October 6 Due: Thursday, October 15, 2009
Reading: Boyd and Vandenberghe: Chapter 2, 4.3, 4.4 Proble
UC Berkeley
Department of Electrical Engineering and Computer Science
EECS 227A
Nonlinear and Convex Optimization
Problem Set 4
Fall 2009
Issued: Tuesday, October 6
Due: Thursday, October 15, 2009
Reading: Boyd and Vandenberghe: Chapter 2, 4.3, 4.4
Proble
UC Berkeley Department of Electrical Engineering and Computer Science EECS 227A Nonlinear and Convex Optimization Solutions 3 Fall 2009
Solution 3.1 (a) We have f (x) = x
2 x
and 2 f (x) = ( 2) x
4 xxT x 2 x
4 xxT
+ x
2 I ,
so
2 f (x)x = ( 2) x = ( 2)
UC Berkeley Department of Electrical Engineering and Computer Science EECS 227A Nonlinear and Convex Optimization Problem Set 3 Fall 2009 Issued: Tuesday, September 22 Problem 3.1 Consider applying Newtons method to the cost function x Due: Tuesday, Octob