Math 408
Homework Set 2
This homework set will focus on the linear least squares problem
1
LLS
min Ax b 2 ,
2
n 2
xR
where A Rmn and b Rm .
(1) Listed below are two functions. In each case write the problem minx f (x) as a linear least squares
problem by
Math 408
Homework Set 2
This homework set will focus on the linear least squares problem
LLS
min
n
xR
1
Ax b
2
2
2
,
where A Rmn and b Rm .
(1) Listed below are two functions. In each case write the problem minx f (x) as a linear least squares
problem by
Math 408
Homework Set 3
This homework set will focus on the optimization problem
1
Q
min xT Hx + g T x ,
xRn 2
nn
n
where H R
is symmetric and g R .
(1) Each of the following functions can be written in the form f (x) = 1 xT Hx + g T x with H symmetric.
2
MATH 408 QUIZ
NAME (Please print):
There are 2 problems. Stop now and make sure you have both problems. If you do not have them
both, then request a new quiz. The rst problem is worth 30 points and the second is worth 40 points
for a total of 70 points. S
1. Review of Multi-variable Calculus
Throughout this course we will be working with the vector space Rn . For this reason we
begin with a brief review of its metric space properties
Denition 1.1 (Vector Norm). A function : Rn R is a vector norm on Rn if t
CHAPTER 2
Review of Matrices and Block Structures
Numerical linear algebra lies at the heart of modern scientic computing and computational science. Today
it is not uncommon to perform numerical computations with matrices having millions of components. Th
Nonlinear Optimization
James V. Burke
University of Washington
Contents
Chapter 1.
Introduction
5
Chapter 2. Review of Matrices and Block Structures
1. Rows and Columns
2. Matrix Multiplication
3. Block Matrix Multiplication
4. Gauss-Jordan Elimination Ma
Math 408
This homework set will focus on the following two optimization problems:
1
Q
min xT Hx + g T x
xRn 2
and
1
LLS
min Ax b 2 ,
2
n 2
xR
where H Rnn is symmetric, g Rn , A Rnk and b Rn . For easy of reference dene
1
1
f (x) := xT Hx + g T x
and
h(x)
Math 408
Homework Set 5
(1) Show that the functions
f (x1 , x2 ) = x2 + x3 ,
1
2
(2)
(3)
(4)
(5)
(6)
(7)
(8)
and g(x1 , x2 ) = x2 + x4
1
2
both have a critical point at (x1 , x2 ) = (0, 0) (i.e. f (x1 , x2 ) = g(x1 , x2 ) = 0) and that their
associated he
Math 408
Homework Set 3 Solutions
This homework set will focus on the optimization problem
1
Q
minn xT Hx + g T x ,
xR 2
nn
n
where H R
is symmetric and g R .
(1) Each of the following functions can be written in the form f (x) = 12 xT Hx + g T x with H s
Math 408
Solutions
This homework set will focus on the following two optimization problems:
1
min xT Hx + g T x
n 2
xR
Q
and
LLS
min
n
xR
1
Ax b 2 ,
2
2
where H Rnn is symmetric, g Rn , A Rnk and b Rn . For easy of reference dene
1
f (x) := xT Hx + g T x
Math 408
Homework Set 6
(1) Use the delta method (as we did in class for the Linear Least Squares function) to compute
the gradient and the Hessian of the following functions.
(a) f (x) := 1 Ax b 2 , where A Rmn and b Rm .
2
2
Solution
f (x) =
1
Ax b 2 ,
Math 408
Homework Set 5
Solutions
(1) Show that the functions
f (x1 , x2 ) = x2 + x3 ,
1
2
and g(x1 , x2 ) = x2 + x4
1
2
both have a critical point at (x1 , x2 ) = (0, 0) and that their associated Hessians are positive semidenite. Then show that (0, 0) is
MATH 408
FINAL EXAM
SOLUTIONS TO SAMPLE QUESTIONS
Sample Questions
Question 1: Theory Question
1. State the rst- and second-order conditions for optimality for the following two problems:
(a) Linear least squares: minxRn
1
2
Ax b 2 , where A Rmn and b Rm
Math 408
Homework Set 1
Linear Algebra Review Problems
(1) Consider the system
4x1
x3 = 200
9x1 + x2 x3 = 200
7x1 x2 + 2x3 = 200 .
(a) Write the augmented matrix corresponding to this system.
(b) Reduce the augmented system in part (a) to echelon form.
(
Math 408
Homework Set 3 Solutions
This homework set will focus on the optimization problem
1
Q
min xT Hx + g T x ,
n 2
xR
nn
n
where H R
is symmetric and g R .
(1) Each of the following functions can be written in the form f (x) = 1 xT Hx + g T x with H s
Math 408
Homework Set 4 Solutions
This homework set will focus on the following two optimization problems:
1
min xT Hx + g T x
n 2
xR
Q
and
LLS
min
n
xR
1
Ax b 2 ,
2
2
where H Rnn is symmetric, g Rn , A Rnk and b Rn . For easy of reference dene
1
f (x) :=
Math 408
Homework Set 4
This homework set will focus on the following two optimization problems:
1
Q
min xT Hx + g T x
xRn 2
and
1
LLS
min Ax b 2 ,
2
n 2
xR
where H Rnn is symmetric, g Rn , A Rnk and b Rn . For easy of reference dene
1
1
f (x) := xT Hx +
Math 408
Homework Set 5
(1) Consider the linearly constrained quadratic optimization problem
1 T
x Hx + g T x
Q(H, g, A, b)
minimize
2
subject to Ax = b ,
where H Rnn is symmetric an positive denite and A Rmn has rank (A) = m.
(a) Write necessary and suci
Math 408
Homework Set 6
(1) Compute the directional derivative for any norm an the origin in any direction and show
that no norm can be dierentiable at the origin.
Solution
Let f (x) := x be a norm on Rn . Then, for every d Rn ,
f (0 + td) f (0)
td
= lim
Math 408
Homework Set 1
Linear Algebra Review Problems
(1) Consider the system
4x1
x3 = 200
9x1 + x2 x3 = 200
7x1 x2 + 2x3 = 200 .
(a) Write the augmented matrix corresponding to this system.
(b) Reduce the augmented system in part (a) to echelon form.
(
Math 408
Homework Set 5
(1) Consider the linearly constrained quadratic optimization problem
Q(H, g, A, b)
1 T
x Hx + g T x
2
subject to Ax = b ,
minimize
where H Rnn is symmetric an positive denite and A Rmn has rank (A) = m.
(a) Write necessary and suci
Math 408
Homework Set 2
This homework set will focus on the linear least squares problem
LLS
min
n
xR
1
Ax b
2
2
2
,
where A Rmn and b Rm .
(1) Listed below are two functions. In each case write the problem minx f (x) as a linear least squares
problem by
MATH 408
MIDTERM GUIDE SOLUTIONS
OUTLINE
Sample Questions
(I) Linear Least Squares
Question 1:
Let A Rmn and b Rm , and consider the linear least squares problem
LLS
min
1
Ax b
2
2
2
.
a. Show that the matrix AT A is always positive semi-denite and provid