498
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 47, NO. 2, FEBRUARY 2001
Factor Graphs and the SumProduct Algorithm
Frank R. Kschischang
, Senior Member, IEEE
, Brendan J. Frey
, Member, IEEE
, and
HansAndrea Loeliger
, Member, IEEE
Abstract—
Algorithms that must deal with complicated global
functions of many variables often exploit the manner in which the
given functions factor as a product of “local” functions, each of
which depends on a subset of the variables. Such a factorization
can be visualized with a bipartite graph that we call a
factor graph
.
In this tutorial paper, we present a generic messagepassing algo
rithm, the sumproduct algorithm, that operates in a factor graph.
Following a single, simple computational rule, the sumproduct
algorithm
computes—either
exactly
or
approximately—var
ious marginal functions derived from the global function. A
wide variety of algorithms developed in artificial intelligence,
signal processing, and digital communications can be derived as
specific instances of the sumproduct algorithm, including the
forward/backward algorithm, the Viterbi algorithm, the iterative
“turbo” decoding algorithm, Pearl’s belief propagation algorithm
for Bayesian networks, the Kalman filter, and certain fast Fourier
transform (FFT) algorithms.
Index Terms—
Belief propagation, factor graphs, fast Fourier
transform, forward/backward algorithm, graphical models, iter
ative decoding, Kalman filtering, marginalization, sumproduct
algorithm, Tanner graphs, Viterbi algorithm.
I. I
NTRODUCTION
T
HIS paper provides a tutorial introduction to factor graphs
and the sumproduct algorithm, a simple way to under
stand a large number of seemingly different algorithms that have
been developed in computer science and engineering. We con
sider algorithms that deal with complicated “global” functions
of many variables and that derive their computational efficiency
by exploiting the way in which the global function factors into
a product of simpler “local” functions, each of which depends
on a subset of the variables. Such a factorization can be visual
ized using a
factor graph
, a bipartite graph that expresses which
variables are arguments of which local functions.
Manuscript received August 3, 1998; revised October 17, 2000. The work
of F. R. Kschischang was supported in part, while on leave at the Massachu
setts Institute of Technology, by the Office of Naval Research under Grant
N000149610930, and by the Army Research Laboratory under Cooperative
Agreement DAAL019620002. The work of B. J. Frey was supported,
while a Beckman Fellow at the Beckman Institute of Advanced Science and
Technology, University of Illinois at UrbanaChampaign, by a grant from
the Arnold and Mabel Beckman Foundation. The material in this paper was
presented in part at the 35th Annual Allerton Conference on Communication,
Control, and Computing, Monticello, IL, September 1997.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Fall '04
 MartinWainwright
 Algorithms, Optimization, Probability theory, probability density function, factor graph, KSCHISCHANG

Click to edit the document details