Goldstein Steplength Criterion
0
F ( x) + 2g T p
-0.5
F ( x + p )
T
F ( x) + g p
Tangent line
-1
F ( x) + 1g T p
F
-1.5
0
0.5
1
1.5
2
Acceptable region
2.5
Gamma Steplength Criterion
0.5
F ( x + p )
0
-0.5
-1
g T p
g T p
F ( x) + g T p
-1.5
-2
-2.5
F
-3

2. LINEAR CONSTRAINTS
In this chapter, we consider optimization problems with constraints defined by linear functions of the variables. The general form of a linear function is v(x) = aTx , for some row
vector aT and scalar . By linearity, the column vect

1
Supplementary Notes on Linear Programming
1.
Introduction
This note is concerned with barrier-function methods for the solution of linear programs in the standard form
minimize
cTx
subject to
Ax = b,
x
x 0,
(1.1)
where A is an m n matrix with m n.
1.1.

1.1. METHODS FOR UNIVARIATE FUNCTIONS
1.1.1. Finding the Zero of a Univariate Function
It will be shown that a necessary condition for x to be an unconstrained minimizer of a
twice-continuously differentiable univariate function f (x) is that f 0 (x ) = 0

APPENDIX
RESULTS FROM CALCULUS AND LINEAR ALGEBRA
Several useful results from real analysis and basic calculus are summarized here.
Limit point of a sequence in <n . Let cfw_xk be a sequence in <n . A point x is
called a limit point of cfw_xk if there e

The problem
An inequality-constrained nonlinear programming problem may be posed in
the form
minimize
f (x)
xIRn
(1)
subject to c(x) 0,
where f (x) is a nonlinear function and c(x) is an m-vector of nonlinear
functions with ith component ci (x), i = 1,. .

Figure 4a. The peinte an, 5:; end :3 denote the ret three membere ef the eequenee
generated by the bieeeen algal-item applied he e en the intehrel [e, e]. Figure 4b. The paint n+1 denetee the Newton approximation to the zero of re].
The new point in dened

1.2. OPTIMALITY CONDITIONS FOR UNCONSTRAINED PROBLEMS
We shall be concerned with the unconstrained problem:
minimize
x2D
F (x),
where F (x) is a real-valued function defined on an open convex set D <n .
Definition 1.2.1. A point x 2 <n is a local minimize

Chapter 3
NONLINEAR CONSTRAINTS
Even one nonlinear constraint considerably increases the difficulty of solving an
optimization problem. It often pays to try and eliminate nonlinear constraints if at
all possible. A measure of the increase in difficulty ma