Math 408
Homework Set 2
This homework set will focus on the linear least squares problem
LLS
min
n
xR
1
Ax b
2
2
2
,
where A Rmn and b Rm .
(1) Listed below are two functions. In each case write the problem minx f (x) as a linear least squares
problem by
Math 408
Homework Set 2
This homework set will focus on the linear least squares problem
1
LLS
min Ax b 2 ,
2
n 2
xR
where A Rmn and b Rm .
(1) Listed below are two functions. In each case write the problem minx f (x) as a linear least squares
problem by
MATH 408
MIDTERM GUIDE SOLUTIONS
OUTLINE
Sample Questions
(I) Linear Least Squares
Question 1:
Let A Rmn and b Rm , and consider the linear least squares problem
LLS
min
1
Ax b
2
2
2
.
a. Show that the matrix AT A is always positive semi-denite and provid
Math 408
Homework Set 1
Linear Algebra Review Problems
(1) Consider the system
4x1
x3 = 200
9x1 + x2 x3 = 200
7x1 x2 + 2x3 = 200 .
(a) Write the augmented matrix corresponding to this system.
(b) Reduce the augmented system in part (a) to echelon form.
(
Math 408
Homework Set 2
This homework set will focus on the linear least squares problem
LLS
min
n
xR
1
Ax b
2
2
2
,
where A Rmn and b Rm .
(1) Listed below are two functions. In each case write the problem minx f (x) as a linear least squares
problem by
1. Review of Multi-variable Calculus
Throughout this course we will be working with the vector space Rn . For this reason we
begin with a brief review of its metric space properties
Definition 1.1 (Vector Norm). A function : Rn R is a vector norm on Rn if
MATH 408
MIDTERM GUIDE SOLUTIONS
OUTLINE
Sample Questions
(I) Linear Least Squares
Question 1:
Let A Rmn and b Rm , and consider the linear least squares problem
LLS
1
min kAx bk22 .
2
a. Show that the matrix AT A is always positive semi-definite and prov
MATH 408
MIDTERM EXAM
OUTLINE
The midterm exam will consist of two parts: (I) Linear Least Squares and (II) Quadratic Optimization.
In each part, the first question concerns definitions, theorems, and proofs, and the remaining questions are
computational.
MATH 408
FINAL EXAM GUIDE
GUIDE
This exam will consist of three parts: (I) Linear Least Squares, (II) Quadratic Optimization, and (III)
Optimality Conditions and Line Search Methods. The topics covered on the first two parts (I) Linear Least
Squares and (
Nonlinear Optimization
MATH 408 Winter 2017
Homework 1
Due January 18, 2017
Reading: Read Chapter 1 (except for the Matrix norms section).
Do the following exercises showing all work.
1. Consider the system
4x1
x3 = 200
9x1 + x2 x3 = 200
7x1 x2 + 2x3 = 2
i
i
Downloaded 01/06/17 to 69.91.157.97. Redistribution subject to SIAM license or copyright; see http:/www.siam.org/journals/ojsa.php
i
NLO
2014/9/17
page 275
i
Bibliographic Notes
Chapter 1 For a comprehensive treatment of multidimensional calculus and
MATH 408 QUIZ
NAME (Please print):
There are 2 problems. Stop now and make sure you have both problems. If you do not have them
both, then request a new quiz. The rst problem is worth 30 points and the second is worth 40 points
for a total of 70 points. S
1. Review of Multi-variable Calculus
Throughout this course we will be working with the vector space Rn . For this reason we
begin with a brief review of its metric space properties
Denition 1.1 (Vector Norm). A function : Rn R is a vector norm on Rn if t
MATH 408
FINAL EXAM
SOLUTIONS TO SAMPLE QUESTIONS
Sample Questions
Question 1: Theory Question
1. State the rst- and second-order conditions for optimality for the following two problems:
(a) Linear least squares: minxRn
1
2
Ax b 2 , where A Rmn and b Rm
Math 408
Homework Set 5
Solutions
(1) Show that the functions
f (x1 , x2 ) = x2 + x3 ,
1
2
and g(x1 , x2 ) = x2 + x4
1
2
both have a critical point at (x1 , x2 ) = (0, 0) and that their associated Hessians are positive semidenite. Then show that (0, 0) is
Math 408
Homework Set 6
(1) Use the delta method (as we did in class for the Linear Least Squares function) to compute
the gradient and the Hessian of the following functions.
(a) f (x) := 1 Ax b 2 , where A Rmn and b Rm .
2
2
Solution
f (x) =
1
Ax b 2 ,
Math 408
Solutions
This homework set will focus on the following two optimization problems:
1
min xT Hx + g T x
n 2
xR
Q
and
LLS
min
n
xR
1
Ax b 2 ,
2
2
where H Rnn is symmetric, g Rn , A Rnk and b Rn . For easy of reference dene
1
f (x) := xT Hx + g T x
Math 408
Homework Set 3 Solutions
This homework set will focus on the optimization problem
1
Q
minn xT Hx + g T x ,
xR 2
nn
n
where H R
is symmetric and g R .
(1) Each of the following functions can be written in the form f (x) = 12 xT Hx + g T x with H s
Math 408
Homework Set 5
(1) Show that the functions
f (x1 , x2 ) = x2 + x3 ,
1
2
(2)
(3)
(4)
(5)
(6)
(7)
(8)
and g(x1 , x2 ) = x2 + x4
1
2
both have a critical point at (x1 , x2 ) = (0, 0) (i.e. f (x1 , x2 ) = g(x1 , x2 ) = 0) and that their
associated he
Math 408
This homework set will focus on the following two optimization problems:
1
Q
min xT Hx + g T x
xRn 2
and
1
LLS
min Ax b 2 ,
2
n 2
xR
where H Rnn is symmetric, g Rn , A Rnk and b Rn . For easy of reference dene
1
1
f (x) := xT Hx + g T x
and
h(x)
Nonlinear Optimization
James V. Burke
University of Washington
Contents
Chapter 1.
Introduction
5
Chapter 2. Review of Matrices and Block Structures
1. Rows and Columns
2. Matrix Multiplication
3. Block Matrix Multiplication
4. Gauss-Jordan Elimination Ma
CHAPTER 2
Review of Matrices and Block Structures
Numerical linear algebra lies at the heart of modern scientic computing and computational science. Today
it is not uncommon to perform numerical computations with matrices having millions of components. Th
HOMEWORK SOLUTIONS FOR MATH 524
Assignment: page 31, #1,8,9,10ab.
Problem #1. Find the local and global minimizers and maximizers
(when they exist) of the functions:
a)
b)
c)
d)
f (x) = x2 + 2x.
2
f (x) = x2 ex .
f (x) = x4 + 4x3 + 6x2 + 4x.
f (x) = x + s
i
i
Downloaded 01/06/17 to 69.91.157.97. Redistribution subject to SIAM license or copyright; see http:/www.siam.org/journals/ojsa.php
i
NLO
2014/9/17
page 147
i
Chapter 8
Convex Optimization
8.1 Definition
A convex optimization problem (or just a convex
Math 408
Homework Set 5
(1) Consider the linearly constrained quadratic optimization problem
Q(H, g, A, b)
1 T
x Hx + g T x
2
subject to Ax = b ,
minimize
where H Rnn is symmetric an positive denite and A Rmn has rank (A) = m.
(a) Write necessary and suci