CONVERGENCE PROPERTIES OF THE
NELDER–MEAD SIMPLEX METHOD IN LOW DIMENSIONS
*
JEFFREY C. LAGARIAS
†
, JAMES A. REEDS
‡
, MARGARET H. WRIGHT
§
,
AND
PAUL E. WRIGHT
¶
SIAM J. O
PTIM
.
c
1998 Society for Industrial and Applied Mathematics
Vol. 9, No. 1, pp. 112–147
Abstract.
The Nelder–Mead simplex algorithm, first published in 1965, is an enormously pop
ular direct search method for multidimensional unconstrained minimization. Despite its widespread
use, essentially no theoretical results have been proved explicitly for the Nelder–Mead algorithm.
This paper presents convergence properties of the Nelder–Mead algorithm applied to strictly convex
functions in dimensions 1 and 2. We prove convergence to a minimizer for dimension 1, and various
limited convergence results for dimension 2. A counterexample of McKinnon gives a family of strictly
convex functions in two dimensions and a set of initial conditions for which the Nelder–Mead algo
rithm converges to a nonminimizer. It is not yet known whether the Nelder–Mead method can be
proved to converge to a minimizer for a more specialized class of convex functions in two dimensions.
Key words.
direct search methods, Nelder–Mead simplex methods, nonderivative optimization
AMS subject classifications.
49D30, 65K05
PII.
S1052623496303470
1. Introduction.
Since its publication in 1965, the Nelder–Mead “simplex” al
gorithm [6] has become one of the most widely used methods for nonlinear uncon
strained optimization. The Nelder–Mead algorithm should not be confused with the
(probably) more famous simplex algorithm of Dantzig for linear programming; both
algorithms employ a sequence of simplices but are otherwise completely different and
unrelated—in particular, the Nelder–Mead method is intended for unconstrained op
timization.
The Nelder–Mead algorithm is especially popular in the fields of chemistry, chem
ical engineering, and medicine. The recent book [16], which contains a bibliography
with thousands of references, is devoted entirely to the Nelder–Mead method and vari
ations. Two measures of the ubiquity of the Nelder–Mead method are that it appears
in the bestselling handbook
Numerical Recipes
[7], where it is called the “amoeba
algorithm,” and in
Matlab
[4].
The Nelder–Mead method attempts to minimize a scalarvalued nonlinear func
tion of
n
real variables using only function values, without any derivative information
(explicit or implicit). The Nelder–Mead method thus falls in the general class of
di
rect search methods
; for a discussion of these methods, see, for example, [13, 18]. A
large subclass of direct search methods, including the Nelder–Mead method, maintain
at each step a nondegenerate
simplex
, a geometric figure in
n
dimensions of nonzero
volume that is the convex hull of
n
+ 1 vertices.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '08
 Staff
 Applied Mathematics, Derivative, Convex set, Convex function, NelderMead algorithm

Click to edit the document details