Lecture Supplemental 2 Optimization in One Dimension
Differentiability in One Dimension
f is differentiable at x D iff
f ( x + x) f ( x) L = lim x 0 x
exists. If f is differentiable at x, L is the derivative of f at x, denoted f (x) or df(x)/dx. Differentiability implies continuity but not vice versa.
II - 2
Mean-Value Theorem
Let f : [a, b] R be a differentiable function. Then there must exist c (a, b) such that
df (c) f (b) f (a) = . dx ba
increasing. f (x) 0 for all x iff f is weakly decreasing.
ac
b
> increasing. If f (x) 0 for all x, f must be strictly < decreasing.
II - 3
Global Minima and Maxima
Let x* D. x* is a global maximum of f iff, for all x D, f(x) f(x*). Likewise, x* is a global minimum of f iff, for all x D, f(x) f(x*). When optimizing the function f on D, our aim is to find a global maximum (or minimum) of f.
II - 4
Local Minima and Maxima
Let x* D. x* is a local maximum of f iff there exists a > 0 such that for all x D (x* , x* + ), f(x) f(x*). Likewise, x* is a local minimum of f iff there exists a > 0 such that for all x D (x* , x* + ), f(x) f(x*).
II - 5
Open Sets
A set S R is open if any point in S is a member of an open interval that is also contained in S. Formally, for any x S, there exists > 0 such that (x , x + ) S.
II - 6
First-Derivative Test
If f : D R is differentiable, where D is open, and x* D is a local maximum of f then f (x*) = 0. The first step to find a maximum of a differentiable function f is solve for roots of f .
This test cannot distinguish between local maxima and local minima. A root of f need not be a minimum or maximum. Some authors refer to a root of f as an extremum.
II - 7
Cubic Function
For example, let f(x) = x3. The derivative is f (x) = 3x2, which has a root at x = 0. But x = 0 is neither a minimum nor a maximum of f.
1 0.75 0.5 0.25 -1 -0.5 -0.25 -0.5 -0.75 -1 0.5 1
II - 8
Second Derivative Test
Let f be twice continuously differentiable. If x* D satisfies f (x*) = 0, then
if f (x*) < 0, x* is a local maximum. if f (x*) > 0, x* is a local minimum.
If f (x*) = 0, the second derivative test does not tell us what kind of extremum x* is, and you must look at higher-order derivatives.
II - 9
Global Optimization
The first and second derivative tests provide ways of finding local minima or maxima. There is no simple way to find a global optimum. Generally, you must go through all local optima and see which has the best value of f. However, if f has certain properties, there will be a unique local maximum (or minimum).
II - 1 0
Existence of Optima
In the following, we will assume that f : [a, b] R. If f is continuous, then there will exist xmin and xmax [a, b] such that for all x [a, b]
f ( xmin ) f ( x) f ( xmax ).
II - 1 1
Convexity
f is convex iff for all x, y [a, b] and for all [0, 1],
f (x + (1 ) y ) f ( x) + (1 ) f ( y ).
If, in addition, for all (0, 1)
x y
f (x + (1 ) y ) < f ( x) + (1 ) f ( y ),
we say that f is strictly convex.
II - 1 2
Concavity
Likewise, f is concave iff for all x, y [a, b] and for all [0, 1],
f (x + (1 ) y ) f ( x) + (1 ) f ( y ).
If, in addition, for all (0, 1)
x y
f (x + (1 ) y ) > f ( x) + (1 ) f ( y ),
we say that f is strictly concave.
II - 1 3
Preservation Under Addition
Let f and g be concave functions on [a, b]. Let x, y [a, b] and ( 0 , 1 ).
f (x + (1 ) y ) f ( x) + (1 ) f ( y ) g (x + (1 ) y ) g ( x) + (1 ) g ( y )
If we define (f + g)(z) = f(z) + g(z) for [a, z b],
( f + g )(x + (1 ) y ) ( f + g )( x) + (1 )( f + g )( y ).
Convexity, strict convexity, and strict concavity are also preserved.
II - 1 4
Convexity of Maximal Set for a Concave Function
Let f : [a, b] R be concave and continuous. Let S = {x [a, b] : (y [a, b])[f(x) f(y)]}.
S is denoted arg max f. We know S .
Then S is a convex set, meaning that if x < y S then, for all z (x, y), z S. Note that f(x) must be the same for all x S.
II - 1 5
Uniqueness of Maximum for a Strictly Concave Function
If f is strictly concave, S must be a singleton. Suppose x, y S and x < y. Then
x + y f ( x) + f ( y ) f = f ( x). > 2 2
But this contradicts x and y both being in S.
II - 1 6
First-Derivative Test and Concavity
Let f : [a, b] R be concave and differentiable. If x* [a, b] satisfies f (x*) = 0, then x* is a global maximum. Likewise if f is convex and f (x*) = 0, then x* is a global minimum.
II - 1 7
Taylors Theorem
A function is called Cn if its nth-order derivative exists and is continuous. Let f be Cn+1 for x (x0 , x0 + ) for some x0 R and > 0. For all x (x0 , x0 + ), there exists a z between x and x0 such that
n
1 (i ) 1 i f ( x) = f ( x0 )( x x0 ) + f ( n +1) ( z )( x x0 ) n +1. (n + 1)! i = 0 i!
II - 1 8
Convexity, Concavity, and The Second Derivative
Let f : [a, b] R be twice differentiable. f is everywhere nonnegative iff f is convex. If f is everywhere positive, f is strictly convex. f is everywhere nonpositive iff f is concave. If f is everywhere negative, f is strictly concave.
II - 1 9
Strict Concavity And the Second Derivative
You might expect that strict concavity would imply a strictly negative second derivative. This is not true. f(x) = 4 is strictly x concave. But f (x) = x2, so 12 f (0) = 0.
-1 -0.5 -0.2 -0.4 -0.6 -0.8 -1 0.5 1
II - 2 0
To What Extent Is the Strict Converse True?
With more advanced methods, you can show that if f is strictly concave, then
{x [a, b] : f ' ' ( x) = 0}
is a set of measure zero. That is, f (x) < 0 almost everywhere. Likewise if f is strictly convex.
II - 2 1
Optimization on Non-Open Sets
If f is differentiable on an open set D, we have seen that, at a local optimum x*, f (x*) = 0. But most optimization is not done on open sets.
For example, we usually require consumption of goods to be nonnegative, not strictly positive. We have a theorem that says global optima must exist on closed and bounded sets, but not open sets.
If D is not open, we can have f (x*) 0 at a local or global optimum x* D.
II - 2 2
A Corner Solution
Suppose you are maximizing f(x) = x on [0, 1]. We have f (x) = 1 for all x [0, 1]. There is no point in [0, 1] that satisfies the first-derivative test.
1 0.8
Yet clearly x* = 1 is a global maximum. This is a corner solution.
0.6 0.4 0.2
0.2
0.4
0.6
0.8
1
II - 2 3
Constrained Optimization
Suppose you are maximizing f : [a, b] R, where f is differentiable. An interior solution x* (a, b) must satisfy f (x*) = 0. There are two possible corner solutions:
x* = a must satisfy f (a) 0. x* = b must satisfy f (b) 0.
Constrained optimization is more difficult in multiple dimensions.
a
b
II - 2 4
Summary
1. If f is a differentiable function,
An interior maximum must satisfy f = 0. A boundary maximum can satisfy weaker conditions.
2. Second-order conditions are needed to verify that an interior extremum is an optimum. 3. The maxima of a concave function will form a convex set, of a strictly concave function a singleton set.
II - 2 5