LP Decoding
Jon Feldman
*
Industrial Engineering and Operations Research
Columbia University, New York, NY, 10027
jonfeld@ieor.columbia.edu
David R. Karger
†
Laboratory for Computer Science
MIT, Cambridge, MA, 02139
karger@theory.lcs.mit.edu
Martin J. Wainwright
Electrical Engineering and Computer Science
UC Berkeley, CA, 94720
wainwrig@eecs.berkeley.edu
Abstract.
Linear programming (LP) relaxation is a common technique used to find good solutions to
complex optimization problems. We present the method of “LP decoding”: applying LP relaxation to
the problem of maximum-likelihood (ML) decoding. An arbitrary binary-input memoryless channel is
considered. This treatment of the LP decoding method places our previous work on turbo codes [6] and
low-density parity-check (LDPC) codes [8] into a generic framework. We define the notion of a
proper
relaxation, and show that any LP decoder that uses a proper relaxation exhibits many useful properties.
We describe the notion of
pseudocodewords
under LP decoding, unifying many known characterizations
for specific codes and channels. The
fractional distance
of an LP decoder is defined, and it is shown
that LP decoders correct a number of errors equal to half the fractional distance. We also discuss the
application of LP decoding to binary linear codes. We define the notion of a relaxation being
symmetric
for a binary linear code. We show that if a relaxation is symmetric, one may assume that the all-zeros
codeword is transmitted.
1
Introduction
The problem of maximum-likelihood (ML) decoding is to find the codeword most likely to
have been transmitted, given a corrupted codeword from a noisy channel. Linear programming
is the problem of finding an optimal solution to a system of linear inequalities under a linear
objective function [2]. In this paper, we consider linear programming (LP) formulations of the
ML decoding problem on binary codes. We use LP variables to represent code bits, and the LP
objective function is defined by the channel likelihood ratios.
Previous work on LP decoding [6, 4, 7, 8, 5] has focused on two specific cases:
turbo
codes
[1] and
low-density parity-check codes
[11]. These two families of codes have received
a lot of attention recently due to their excellent performance.
Performance bounds for LP
decoding in these cases are for specific LP formulations, code constructions, and/or channel
models.
In this paper we consider LP decoders for arbitrary binary codes, under an arbitrary binary-
input memoryless channel. We provide a framework for designing LP decoders, and general
techniques for analyzing them.
Central to every LP decoder is its associated
polytope
: the
set of points that satisfy the constraints of the LP. A decoding polytope should contain ev-
ery codeword, and should also exclude every binary word that is not a codeword. We define
such polytopes as
proper
. We show that LP decoders that use proper polytopes have the
ML
certificate
property: whenever they output a codeword, it is guaranteed to be the ML codeword.
In general, for any sub-optimal decoder, the