CONVERGENCE PROPERTIES OF THE
NELDER–MEAD SIMPLEX METHOD IN LOW DIMENSIONS
*
JEFFREY C. LAGARIAS
†
, JAMES A. REEDS
‡
, MARGARET H. WRIGHT
§
,
AND
PAUL E. WRIGHT
¶
SIAM J. O
PTIM
.
c
±
1998 Society for Industrial and Applied Mathematics
Vol. 9, No. 1, pp. 112–147
Abstract.
The Nelder–Mead simplex algorithm, ﬁrst published in 1965, is an enormously pop-
ular direct search method for multidimensional unconstrained minimization. Despite its widespread
use, essentially no theoretical results have been proved explicitly for the Nelder–Mead algorithm.
This paper presents convergence properties of the Nelder–Mead algorithm applied to strictly convex
functions in dimensions 1 and 2. We prove convergence to a minimizer for dimension 1, and various
limited convergence results for dimension 2. A counterexample of McKinnon gives a family of strictly
convex functions in two dimensions and a set of initial conditions for which the Nelder–Mead algo-
rithm converges to a nonminimizer. It is not yet known whether the Nelder–Mead method can be
proved to converge to a minimizer for a more specialized class of convex functions in two dimensions.
Key words.
direct search methods, Nelder–Mead simplex methods, nonderivative optimization
AMS subject classiﬁcations.
49D30, 65K05
PII.
S1052623496303470
1. Introduction.
Since its publication in 1965, the Nelder–Mead “simplex” al-
gorithm [6] has become one of the most widely used methods for nonlinear uncon-
strained optimization. The Nelder–Mead algorithm should not be confused with the
(probably) more famous simplex algorithm of Dantzig for linear programming; both
algorithms employ a sequence of simplices but are otherwise completely diﬀerent and
unrelated—in particular, the Nelder–Mead method is intended for unconstrained op-
timization.
The Nelder–Mead algorithm is especially popular in the ﬁelds of chemistry, chem-
ical engineering, and medicine. The recent book [16], which contains a bibliography
with thousands of references, is devoted entirely to the Nelder–Mead method and vari-
ations. Two measures of the ubiquity of the Nelder–Mead method are that it appears
in the best-selling handbook
Numerical Recipes
[7], where it is called the “amoeba
algorithm,” and in
Matlab
[4].
The Nelder–Mead method attempts to minimize a scalar-valued nonlinear func-
tion of
n
real variables using only function values, without any derivative information
(explicit or implicit). The Nelder–Mead method thus falls in the general class of
di-
rect search methods
; for a discussion of these methods, see, for example, [13, 18]. A
large subclass of direct search methods, including the Nelder–Mead method, maintain
at each step a nondegenerate
simplex
, a geometric ﬁgure in
n
dimensions of nonzero
volume that is the convex hull of
n
+ 1 vertices.