This preview shows pages 1–3. Sign up to view the full content.
UC Berkeley
Department of Electrical Engineering and Computer Science
EECS 227A
Nonlinear and Convex Optimization
Solutions 2
Fall 2009
Reading:
Sections 1.2—1.3 of Nonlinear programming by Bertsekas. (
Note:
Be sure to
use the scanned PDFs on webpage to obtain the correct numbering of the problems in this
assignment.)
Solution 2.1
The problem is to perform an unconstrained minimization of the function
f
(
x,y
) = 3
x
2
+
y
4
using various methods. Using the starting point
x
0
= (1
,

2), we have
f
(
x
0
) = 19
,
∇
f
(
x
0
) =
±
6
x
4
y
3
²
=
±
6

32
²
(a) Steepest descent with
s
= 1,
σ
= 0
.
1 and
β
= 0
.
5 takes the form:
x
1
=
x
0
+
β
m
sd
0
where
d
0
=
∇
f
(
x
0
). Here are the results of determining the appropriate
m
according to the
Armijo rule:
m
β
m
s
f
(
x
0
)

f
(
x
0
+
β
m
sd
0
)

σβ
m
s
∇
f
(
x
0
)
T
d
0
0
1
810056
106
1
.5
38409
53
2
.25
1277.75
26.5
3
.125
2.8125
13.25
4
.0625
17.828125
6.625
So one iteration of steepest descent requires 5 internal iterations to determine the value
of
m
; it yields the new point
x
1
= (0
.
625
,
0) where
f
(
x
1
) = 1
.
17.
(b) Steepest descent with
s
= 1
,σ
= 0
.
1 and
β
= 0
.
1.
m
β
m
s
f
(
x
0
)

f
(
x
0
+
β
m
sd
0
)

σβ
m
s
∇
f
(
x
0
)
T
d
0
0
1
810056
106
1
.1
16.4464
10.6
Now only 2 internal iterations are required, and yield the new point
x
1
= (0
.
4
,
1
.
2) where
f
(
x
1
) = 2
.
55. The smaller value of
β
reduced the amount of time to ﬁnd the stepsize, but
yielded a new point
x
1
with higher cost.
(c) Newton’s method with
s
= 1
,σ
= 0
.
1 and
β
= 0
.
5. We now use the descent direction
d
0
=

(
∇
2
f
(
x
0
))

1
∇
f
(
x
0
). Calculations yield
d
0
=

³
6
0
0 48
´

1
±
6 32
²
=
±

1 2
/
3
²
.
1
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Documentm
β
m
s
f
(
x
0
)

f
(
x
0
+
β
m
sd
0
)

σβ
m
s
∇
f
(
x
0
)
T
d
0
0
1
15.8395
2.733
So one iteration of Newton’s method requires only 1 internal iteration to determine
m
, and
yields
x
1
= (0
,

4
/
3)
0
with
f
(
x
1
) = 3
.
1605. Although fewer internal iterations were required,
the cost of the resulting
x
1
is higher. In this case, determining the descent direction (involving
the inverse of the Hessian) was straightforward, but this calculation is potentially expensive.
Solution 2.2
This is the end of the preview. Sign up
to
access the rest of the document.
 Spring '11
 Staff
 Electrical Engineering

Click to edit the document details