Industrial Engineering and Operations Research
Columbia University, New York, NY, 10027
David R. Karger
Laboratory for Computer Science
MIT, Cambridge, MA, 02139
Martin J. Wainwright
Electrical Engineering and Computer Science
UC Berkeley, CA, 94720
Linear programming (LP) relaxation is a common technique used to find good solutions to
complex optimization problems. We present the method of “LP decoding”: applying LP relaxation to
the problem of maximum-likelihood (ML) decoding. An arbitrary binary-input memoryless channel is
considered. This treatment of the LP decoding method places our previous work on turbo codes  and
low-density parity-check (LDPC) codes  into a generic framework. We define the notion of a
relaxation, and show that any LP decoder that uses a proper relaxation exhibits many useful properties.
We describe the notion of
under LP decoding, unifying many known characterizations
for specific codes and channels. The
of an LP decoder is defined, and it is shown
that LP decoders correct a number of errors equal to half the fractional distance. We also discuss the
application of LP decoding to binary linear codes. We define the notion of a relaxation being
for a binary linear code. We show that if a relaxation is symmetric, one may assume that the all-zeros
codeword is transmitted.
The problem of maximum-likelihood (ML) decoding is to find the codeword most likely to
have been transmitted, given a corrupted codeword from a noisy channel. Linear programming
is the problem of finding an optimal solution to a system of linear inequalities under a linear
objective function . In this paper, we consider linear programming (LP) formulations of the
ML decoding problem on binary codes. We use LP variables to represent code bits, and the LP
objective function is defined by the channel likelihood ratios.
Previous work on LP decoding [6, 4, 7, 8, 5] has focused on two specific cases:
low-density parity-check codes
. These two families of codes have received
a lot of attention recently due to their excellent performance.
Performance bounds for LP
decoding in these cases are for specific LP formulations, code constructions, and/or channel
In this paper we consider LP decoders for arbitrary binary codes, under an arbitrary binary-
input memoryless channel. We provide a framework for designing LP decoders, and general
techniques for analyzing them.
Central to every LP decoder is its associated
set of points that satisfy the constraints of the LP. A decoding polytope should contain ev-
ery codeword, and should also exclude every binary word that is not a codeword. We define
such polytopes as
. We show that LP decoders that use proper polytopes have the
property: whenever they output a codeword, it is guaranteed to be the ML codeword.
In general, for any sub-optimal decoder, the