This preview shows pages 1–12. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: 1698 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 42, NO. 6, NOVEMBER I996 A Distance Spectrum Interpretation of Turbo Codes Lance C. Perez, Member, IEEE, Jan Seghers, and Daniel J. Costello, In, Fellow, ,IEEE * Abstract— The performance of Turbo codes is addressed by V _ examining the code’s distance spectrum. The “error ﬂoor” that occurs at moderate signaltonoise ratios is shown to be a conse
quence of the relatively low free distance of the code. It is also
shown that the “error ﬂoor” can be lowered by increasing the size
of the interleaver without changing the free distance of the code.
Alternatively, the free distance of the code may be increased by
using primitive feedback polynomials. The excellent performance
of Turbo codes at low signaltonoise ratios is explained in terms
‘ of the distance spectrum. The interleaver in the Turbo encoder
is shown to reduce the number of lowweight codewords through
a process called “spectral thinning.” This thinned distance spec
trum results in the free distance asymptote being the dominant
performance parameter for low and moderate signaltonoise
ratios. — Index Terms—Turbo codes,'convoluti0nal codes, distance spec
‘ trum. I. INTRODUCTION HE DISCOVERY of Turbo codes and the near—capacity performance reported in [1] has stimulated a ﬂurry of
research efforts to fully understand this new coding scheme
[2]—[43]. Initially greeted with some skepticism, the original
results were independently reproduced by several researchers
[6], [7], [10]—[13], and [30]. Subsequently, recent research on
Turbo codes hasfocused on understanding the reasons for their
outstanding performance [8], [9], [16] [22]—[24], [28]. At this point, there are two fundamental questions regard—
ing Turbo codes. First, does the iterative decoding scheme
presented in [1] always converge tothe optimum solution?
Second, assuming optimum ornearoptimum decoding, why
do the Turbo codes perform so well? In this paper, we'address
the second issue in a semitutorial manner by examining the
distance spectrum of Turbo codes. In doing so, we will draw
on the work of several research groups involved with Turbo
codes [6]—[9], [12]—[14], [18]—[24]. Due to the intense interest
in this subject, many results involving Turbo codes have
been developed independently by others and the reader is
encouraged to peruse the references for an alternative point
of View. In particular, the recent papers by Benedetto and Montorsi [8], [9] were the ﬁrst to offer a comprehensive picture
of Turbo codes. ‘ . Manuscript received January 18, 1996; revised April 30, 1996. This work
was supported by NASA under Grant NAG 5—557, by NSF under Grant
NCR9522939, and by the Lockheed—Martin Corporation. L. C. Perez was with the Division of Engineering, The University of Texas
at San Antonio, San Antonio, TX 78229 USA. He is now with the Department
of Electrical Engineering, The University of Nebraska»Lincoln, Lincoln, NE
68588—05 11 USA. ' J. Seghers is with the Institute for Signal and Information Processing, Swiss
Federal Institute of Technology, Zurich, Switzerland. D. .l. Costello, Jr. is with the Department of Electrical Engineering,
University of Notre Dame, Notre'Dame, IN 46616 USA. Publisher Item Identiﬁer S 0018*9448(96)07521—9. 10‘ .. ' ' ‘ ‘ ‘ "3 M (37,21,65536) Turbo _
H (2,1,14) MFD Convolutional vn—mr, rrnnrq Bit Error Rate
H
c:
. 10 E  10“ T E 107  A i . r . I . . i . . , . . I . . r . . . i . i
0.0 1.0 2.0  3.0 Eth, (dB) Fig. 1. Simulation results for a(37,k21, 65536) Turbo code and a (2, 17 14)
MFD convolutional code. ' The simulated performance of a rate 1 / 2 Turbo code with
the same parameters as in [1] is shown in Fig. 1 along with
simulation results for a rate R = 1/2, memory V : 14,
convolutional code. The comparison of these‘simulation results
raises two issues regarding the performance 0f Turbo codes.
First, what is it that allows Turbo codes to achieve a bit
error rate (BER) of 10“5 at a signalto—noise ratio (SNR) of
Eb/No = 0.7 dB, which is only 0.7 dB from the Shannon
limit? Second, what causes the “error ﬂOor” [10], [13], that
is, the ﬂattening of the performance curve, for moderate to
high SNR’s? Here, we endeavOr to explain the performance
of Turbo codes, and thus address these two issues, in terms of
the code’s distance spectrum. We do not attempt to address the
many interesting questions concerning the iterative decoding
method (see, e.g., [15], [26], [42], and [43]), i.e., we assume
that an optimum or near—optimum decoder is available. In order to explain their performance in terms of the free distance and the distance spectrum, we willexamine the
codeword structure of Turbo codes in detail. Here, the free distance is deﬁned to be the minimum Hamming weight of
all possible codewords and the error coefﬁcient is the total
number, or multiplicity, of free—distance codewords. The goal
is to use speciﬁc examples to elucidate the key structural
properties that result in the near—capacity performance of Turbo
codes at BER’s around 10—5. As will be seen, this effort leads
to an explanation that applies to Turbo. codes and also lends
insight into designing codes in general. Throughout the paper,
Turbo codes are compared to a maximum free—distance, rate 0018—9448/96$05.CO © 1996 IEEE PEREZ et al.: A DISTANCE SPECTRUM INTERPRETATION OF TURBO CODES Jti Interleaver Fig. 2. Block diagram of a Turbo encoder with two identical constituent
encoders (h0(D) = 1 + D2,h1(D) = D) and without puncturing. ' R = 1/2, memory 11 = 14, i.e., a (2, 1, 14), convolutional code
to emphasize the differences in performance and structure.
Techniques for analyzing the performance of Turbo codes
using transfer functions and related methods may be found
in [6], [9], and [22]. The paper begins with a detailed examination of the struc
ture of codewords in a Turbo code in Section II. This leads to
the calculation of the free distance of a particular Turbo code
and an explanation for the “error ﬂoor” in its performance
curve. In Section III, the distance spectrum of “average” Turbo
codes is considered and a theory called spectral thinning is
introduced and used to explain the performance of Turbo codes at low SNR’s. The idea of spectral thinning is then formalized  in Section IV through the use of random interleaving. Finally,
some conclusions are drawn concerning the distance spectrum
of Turbo codes and the consequences of this on the design of
codes in general. _ II. THE FREE DISTANCE OF TURBO CODES In order to ﬁnd the free distance of a Turbo code, it is
necessary to understand the basic structure of the encoder and
the resulting codewords. A typical Turbo encoder consists of
the parallel concatenation of two or more, usually identical,
rate 1/2 encoders, realized in systematic feedback form, and
an interleaver. This encoder structure is called a parallel
concatenation because the two encoders operate on the same
set of input bits, rather than one encoding the output of
the other. A block diagram of a Turbo encoder with two
constituent convolutional encoders is shown in Fig. 2. For the
remainder of the paper, only Turbo encoders with two identical
constituent convolutional encoders are considered, though the
conclusions are easily extended to the general case. The interleaver is used to permute the input bits such that
the two encoders are operating on the same set of input bits,
but different input sequences. Thus the ﬁrst encoder receives
the input bit .73, and produces the output pair while
the second encoder receives the input bit and produces the
output pair The input bits are grouped into ﬁnite
length sequences whose length N equals the size of the
interleaver. Since both the encoders are systematic and operate
on the same set of input bits, it is only necessary to transmit the
input bits once and the overall code has rate 1 / 3. In order to
increase the overall rate of the code to 1 / 2, the two parity
sequences {yl} and {342} can be punctured by alternately
deleting y1 and y2. We will refer to a Turbo code whose l 699 constituent encoders have parity—check polynomials ha and
hl and whose interleaver is of length N as an (h0,h1,N)
Turbo code. For example, consider the Turbo encoder shown in Fig. 2,
where each constituent encoder is a (2,1,2) encoder with
parity—check polynomials ho(D) = 1 + D2 and h1(D) = D,
where D is the delay operator [44]. For purposes of illustration,
assume a pseudorandom interleaver of size N = 16 bits which
generates a (1 + D2,D,16) Turbo code. The interleaver is
realized as a 4 X 4 matrix. which is ﬁlled sequentially, row
by row, with the input bits 10,. Once the interleaver has been
ﬁlled, the input sequence to the second encoder, 272, is obtained
by reading the interleaver in a pseudorandom manner until
each bit has been read once and only once. The pseudorandom
nature of the interleaver in this example is represented by a
permutation H16 2 {6, 14, 4, 7, 11, 8, 3, 5, 9, 13, 0, 2, 12, 1, 10, 15} which implies x6 = 0515,1"1 = $10, and so on.
If the input sequence is ‘x = {x15~:1:0} = {0,1,0,1,0,0,0,0,0,0,0,1,0,0,0,1} and the interleaver is represented by the permutation 1116, then
the input sequence to the second encoder is 117' = H16(:1:) = {0,1,1,0,0,0,0,0,0,0,1,0,1,0,0,0}. The trellis diagrams for both constituent encoders with these
inputs are shown in Fig. 3. The corresponding unpunctured
parity sequences are ' y1 = {0,0,1,0,0,0,0,0,0,0,0,0,1,0,1,0} I and g2 = {1,1,0,0,0.0,0,0,0,0,0,1,0,0,0,0}. ‘
The resulting codeword has Hamming weight ,
d=w(x)+w(y1)+w(y2) =4+3+3= 10 without puncturing, where w(at) is the Hamming weight of
the sequence x. If the code is punctured beginning with yé,
then the resulting codeword has weight d : 4 + 3 4— 2 = 9.
If, on the other hand, the puncturing begins with ya, then the
punctured codeword has Hamming weight 4 + 1 = 5. Finding the free distance of a Turbo code is complicated
by the fact that Turbo encoders are timevarying due to the
interleaver. That is, if i’ :2 Dav, then 331 = Dyl, but 1’ 7A Dx’
(with high probability) and g2 96 Dyz. (Here, we only consider
delays of a ﬁnite length sequence ac for which no ones are lost.)
Continuing the example, if 5: : Dag, then a' = 1116(55) = {0,0, 0,0,0,0,0,1,0, 1,0,0,0, 1,0, 1}
and
g2 = {0,0,0,0,0,0,0,0,1,0,0,0,0,0,1,0}. Thus timeshifting the input bits results in codewords that
differ in both the bit positions of the ones and overall Ham
ming weight. The variation in the weights of codewords 1700 State
1 1 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 42, NO. 6, NOVEMBER 1996 Encoder 1 10
01 00
xyl 10 01 00 _01 State
11 1000000000000000 10 01 10 00 Encoder 2 10 01 00 x'yloo .00 00 10 01 10 00 00 00 00 00 00 oo 10 ii 01 Fig. 3. Trellis diagrams for a codeword in the (1 + 1'_)2,D7 16) Turbo code. corresponding to timeshifted input sequences is magniﬁed by
puncturing. This simple example illustrates several important points
concerning the structure of the codewords. First, because the
pseudorandom interleaver permutes the input bits, the two
input sequences 30 and x’ are almost always different, though
of the same weight, and the two encoders will (with high
probability) produce parity sequences of different weights.
Second, it is easily seen that a codeword may consist of a
number of distinct error events in each encoder, where an
error event is a path in the trellis that diverges from the all
zero state and then remerges with the all—zero state within
a ﬁnite number of branches. Note that since the constituent
encoders are realized in systematic feedback form a nonzero
tail of 1/ bits is required to return the encoder to the allzero
state. Thus since at least one nonzero bit is required to start
an error event, all error events are associated with information
sequences of weight 2 or greater [8]. ‘ Finally, with a pseudorandom interleaver it is highly un
likely that both encoders will be returned to the all—zero state
at the end of the codeword, even when the last 1! bits of the
input sequence as are used to force the ﬁrst encoder back to
the all—zero state. This is due to the fact the two encoders will
with high probability end up in different states and thus require
two different V bit tails. It is highly unlikely that the 11—bit tail
for the ﬁrst encoder will be interleaved by a pseudorandom interleaver to the correct tail for the second encoder.
If neither encoder is forced back to the all—zerostat‘e, i.e., no tail is used, then the sequenCe consisting of N a 1 zeroes
followed by a one is a valid input sequence a: to the ﬁrst
encoder. For some interleavers, this :2: will be permuted to
itself and as’ will be the same sequence. In this case, with
puncturing, the weight of this codeword and the free distance
of the code will be at most two. Note that this codeword is
caused by an information sequence of weight one. Thus forcing
the ﬁrst encoder to return to the all—zero state insures that every
information sequence has at least weight two and eliminates
the possibility of a weighttwo codeword. For this reason, it is common to assume that the ﬁrst encoder is forced to return
to the all—zero state by appending a V—bit tail. ‘ The ambiguity of the ﬁnal state of the second encoder has
been shown by simulation to result in negligible performance
degradation for large interleavers [[12], [30]. For these reasons,
it will be assumed in the remainder of the paper that the ﬁrst
encoder is forced to return to the all—zero state and that the ﬁnal
state of the second encoder is unknown. Special interleaver
structures that result in both encoders returning to the all—zero
state are discussed in [31], [32], [34]—[36]. , A. Performance Bounds In order to make clear the distinction between Thrbo codes ‘
and convolutional codes, it is useful to consider these codes as
block codes. To this end, the input sequences are restricted to
length N, where N corresponds to the size of theiinterleaver
in the Turbo encoder. With ﬁnite—length input sequences of
length N , a (2, 1, V) convolutional code may be viewed as a ,
block code‘with 2N codewords of length 2(1/ + N ' The bit error rate (BER) performance of a convolutional
code with maximumlikelihood (ML) decoding on an additive
white Gaussian noise (AWGN) channel can be upper—bounded
using a union bound technique by [12] 2N I ‘
, ’ 2RE
PngEUNQC/d, N 5) (1)
i=1 0  , _ where w, and d, are the information, weight and total Ham
ming weight, respectively, of the ith codeword. Collecting
codewords of the same total Hamming weight and deﬁning
the average information weight per codeword as “in
d—Nd where Wd is the total information weight of all codewords
of weight d and Nd is the total number, or multiplicity, of PEREZ et (11.: A DISTANCE SPECTRUM INTERPRETATION OF TURBO CODES codewords of weight d, yields 2(V+N) ” 2R
B. s Nd“ Q d E" (2)
dzdf N No where deee is the free distance of the code. (In this develop
ment, the multiplicity Nd includes codewords due to multiple
error events for d 2 Zdﬁee). For large enough N, if a convolutional code has N3
codewords of weight d caused by information sequences at
whose ﬁrst one occurs at time 0, then it also has N 3 codewords
of weight d caused by the information sequences D27, N3
codewords of weight d caused by the information sequences
D233, and so on. Thus as the length of the information
sequences increases, we. have . Nd_ 0
Haw—Nd
and
Wd_W§é lirn 12),, = lim of] N —)oo N—>oo Nd — where W3 is the total information weight of all codewords
with weight d which are caused by information sequences
whose ﬁrst one occurs at time 0. Thus the bound on the BER of a convolutional code with ML decoding becomes
2(V+N)  2RE
Hg 2 Ngwgo< d Nob) dzdfree 2(V+N) 2RE
= 2 W362( d N0”) (3) dzdftee which is the standard union bound for ML decoding. For this
reason, efforts to ﬁnd good convolutional codes for use with
ML decoders have focused on ﬁnding codes that maximize the
free distance dﬁee and minimize the number of free—distance
paths Ngee for a given rate and constraint length. The performance of a Turbo code with maximumlikelihood
decoding can also be bounded using the union bound of (2).
However, in the Turbo encoder the pseudorandom interleaver
maps the input sequence at to w’ and the input sequence But to
a sequence at” that is different from Dz’ with high probability.
Thus unlike convolutional codes,’ the input sequences :17 and
D30 produce different codewords with different Hamming
weights. For Turbo codes with pseudorandom interleavers,
Ndwd is much less than N for lowweight codewords. This
is due to the pseudorandom interleaver which, with high
probability, maps lowweight parity sequences in the ﬁrst
constituent encoder to highweight parity sequences in the
second constituent encoder. Thus for low—weight codewords Ndﬂld 1
N < Where ‘
Nd
— 4
N ( )
is called the effective multiplicity of codewords of weight d.
The effect of the interleaver size on the multiplicity is also studied in [7]—[9]. I701 B. Asymptotic Performance For moderate and high signaltonoise ratios, it is well
known that the free—distance term in the union bound on the
bit error rate performance dominates the bound [44]. Thus for
Turbo codes the asymptotic performance approaches N11361:} tee P1. z LIV—f— Q< dire. ——‘i> (5) where Nfree is the multiplicity of free—distance codewords
and we, is the average weight of the information sequences
causing freedistance codewords. The expression on the right—
hand side of (5) and its associated graph is called the free—
distance asymptote, Pfree, of a Turbo code. An algorithm for ﬁnding the free distance of Turbo codes is
described in [28]. This algorithm was applied to a Turbo code
with the same constituent encoders, puncturing pattern, and
interleaver size N as in [1] and a particular pseudorandom
interleaving pattern. The parity check polynomials for this ‘
code are ho = D4+D3+D2+D+1 and h1 = D4+1, or ho :
37 and h1 = 21 using octal notation. This (37, 21, 65536) code
was found to have Nﬁee = 3 paths with weight dfree = 6. Each
of these paths was caused by an input sequence of weight 2
and thus of... = 2. Though this result was for a particular
pseudorandom interleaver, it is true for most pseudorandom
interleavers with N = 65536. This is consistent with the
conclusions in [6] and [8] in which the performance of Turbo
codes is averaged over all possible pseudorandom interleavers. For this particular Turbo code, the free distance asymptote
is given by 3X2 Eb
Pfree“6533_‘6Q( where the rate loss due to the addition of a 4bit tail is ignored
and N free 3 7T “ 65536 is the effective multiplicity. The free—distance asymptote is
shown plotted in Fig. 4 along with simulation results for this
code using the iterative decoding algorithm of [l] with 18
iterations. From Fig. 4, it can clearly be seen that the simula~
tion results do in fact approach the free—distance asymptote for
moderate and high SNR’s. Since the slope of the asymptote is
essentially determined by the free distance of the code, it can
be concluded that the “error ﬂoor” observed with Turbo codes
is due to the fact that they have a relatively small free distance
and consequently a relatively ﬂat freedistance asymptote.
Further examination of (5) reveals that the manifestation
of the “error ﬂoor” can be manipulated in two ways. First,
increasing the length of the interleaver while preserving the
free distance and the multiplicity will lower the asymptote
without changing its slope by reducing the effective multiplic
ity. In this case, the performance curve of Turbo codes does not
ﬂatten out until higher SNR’s and lower BER’s are reached.
Conversely, decreasing the interleaver size while maintaining
the free distance and multiplicity results in the error ﬂoor being
raised and the performance curve ﬂattens at lower SNR’s and 1702 — o—e (37,21,65536) Turbo 7 G~~~~~O (2,1,14) MFD Convolutional Code a H Turbo Free Distance Asymptote (2,1,14) Free Distance Asyynptole Bit Error Rate rib/N, (dB) Fig. 4. Simulation results for a (37, 21, 65536) Turbo code and a (2, 1, 14)
MFD convolutional code along wiht their free~distance asymptotes. 1 10' 3 'v‘ < r r l I
E 3
 G—O N=65536, 18 iterations .
102 __ EL—El N=16384, 18 iterations _:
I E HN=1024, 18 iterations =
3
10 um“, Bit Error Rate
‘ H
on
I U‘n IWL IIWIMI HI 10 r
10" 10" . i . . l . . . 1 . . . i I . . . y i . . a . 0.0 0.5 1.0 ’ 1.5 2.0 2.5
Eb/NB (dB)
Fig. 5. Simulation results for a (37 , 21, N) Turbo code with varying inter— leaver size N. I higher BER’s. This can be seen in the simulation results shown
in Fig. 5 for the (37, 21,N) Turbo code with varying N. If
the ﬁrst constituent encoder is not forced to return to the all— zero state and the weight 2 codewords mentioned earlier are . ' allowed, then the error floor is raised to the extent that the
code performs poorly even for large interleavers. Thus one
cannot completely disregard free distance when constructing
Turbo codes. If the size of the interleaver is ﬁxed, then the “error
' ﬂoor” can be modiﬁed by increasing the free distance of the
7 code while preserving the multiplicity. This has the effect
of changing the slope of the free—distance asymptote. That
is, increasing the free distance increases the slope of the
asymptote'and decreasing the free distance decreases the slope IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 42, NO. 6, NOVEMBER 1996 of the asymptote. It has been shown in [8], [24], [28], and
[29] that for a ﬁxed interleaver size, choosing the feedback
polynomial to be a primitive polynomial results in an increased
free distance and thus a steeper asymptote. An argument to
support the use of primitive feedback polynomials in Turbo
codes is presented in Section IV. ‘ C. Comparison to the (2,1,14) Code The role that the free distance and effective multiplicity play
in determining the asymptotic performance of a Turbo code
is further Clariﬁed by examining the asymptotic performance
of a convolutional code. The free—distance, asymptote of a
convolutional code is given by the ﬁrst term in therunion bound
of (3). The maximum free distance (MFD) (2,1,14) code
whose performance is shown in Fig. 4 has dfree = 18, Ngee =
18 and W0 = 137 [45]. Thus the free—distance asymptote free for this code is
, E
PM = 137Q( 18 which is also shown in Fig.4. As eXpected, the free—distance asymptote of the (2,1,14)
code is much steeper than the free—distance asymptote of the
Turbo code due'to the increased free distance. However, be
cause the effective multiplicity of the free—distance codewords,
given by (4), of the Turbo code is much smaller than the
multiplicity of the (2,1,14) code, the two asymptotes do
not cross until Eb/NO = 3.5 dB. At this Eb/NO, the BER
of both codes is less than 10‘7, which is lower than the
targeted BER of many practical systems. Thus even though
the (2, 1,14) convolutional code is asymptotically better than
the (37 ,21,65536) Turbo code, the Turbo code is better for
the error rates at which many systems operate. D. A Turbo Code with a Rectangular Interleaver To emphasize the importance of using a pseudorandom
interleaver with Turbo codes, we now consider a Turbo code
with a rectangular interleaver. Turbo codes with rectangular
interleavers have also been considered in [2], where the
effect of the interleaver on the free distance of the code
is discussed. The same constituent encoders and puncturing
pattern as in [1] are used in conjunction'with a 120 X 120
rectangular interleaver. This rectangular interleaver is realized
as a 120 x 120 matrix into which the information sequence a: is
written row by row. The input sequence to the second encoder
117‘ is then Obtained by reading the matrix column by column.
A 120 X 120 rectangular interleaver implies an interleaver size
of N = 14400 and thus this is a (37, 21, 14400) Turbo code. Using the algorithm described in [28], this code was found
to have a free distance of dfree : 12 with a multiplicity of
Nfree = 28900. For this code, each of the free—distance paths is
caused by an information sequence of weight 4, so wfree : 4.,
The free—distance asymptote for this code is thus given by ‘ 28900 x 4 Eb
Pfree — ‘ 12 w) .. PEREZ et (11.: A DISTANCE SPECTRUM INTERPRETATION OF TURBO CODES rV—‘T‘ﬁl—lr—Vﬁ 9—9 Simulations
0""0 Free Distance Asymptote Bit Error Rate
H
e 10
10'5
10“ ._4._Al_u..kt_..4c.tt....L._4t__x_lg.JL
0.0 0.5 1.0 1.5 2.0 2.5
El/N, (dB)
Fig. 6. Simulation results and the freedistance asymptote for the (37, 21, 14400) Turbo code with a 120x 120 rectangular interleaver. The free—distance asymptote is plotted in Fig. 6 along with
simulation results using the iterative decoding algorithm of
[l] with 18 iterations. This ﬁgure clearly shows that the free
distance asymptote accurately estimates the performance of
the code for moderate and high Eb/No’s. This code achieves a bit error rate of 10“5 at an Eb/No of
2.7 dB and thus performs 2 dB worse than the (37, 21, 65536)
Turbo code with a pseudorandom interleaver even though it
has a much larger free distance. The relatively poor perfor
mance of the (37, 21, 14400) Turbo code with a rectangular
interleaver is due to the large multiplicity of dfree paths. This
results in an effective multiplicity of Nfree 28900 N N ‘ 14400 “‘2 which is much larger than the effective multiplicity of the
(37, 21, 65536) Turbo code. We now show that the large mul
tiplicity is a direct consequence of the use of the rectangular
interleaver and that, furthermore, increasing the size of the
interleaver does not result in a signiﬁcant reduction in the
effective multiplicity of the freedistance codewords. The freedistance paths in the Turbo code with the rectan—
gular interleaver are due to four basic information sequences
of weight 4. These information sequences are depicted in
Fig. 7 as they would appear in the rectangular interleaver. The
“square” sequence in Fig. 7a) depicts the sequence a; = 110703070:1705947170707070717000 Where 0594 denotes a sequence of 594 consecutive zeroes
and 0,,O represents a sequence of zeroes that continues to the
end of the information sequence. In this case, the rectangular
interleaver maps the sequence a: to itself and therefore m’ = ac.
The sequence 2: results in a parity sequence y1 from the
ﬁrst constituent encoder which, after puncturing, has weight
4. Similarly, the input sequence ac’ = 1; results in a parity
sequence y2 from the second constituent encoder which, after 1703 puncturing, also has weight 4. The weight of the codeword is
then dfree = 4+4+4 = 12. By counting the number of distinct
positions in which these “square” sequences can appear in the
rectangular interleaver, we can ﬁnd the multiplicity of the free
distance codewords. Since the “square” sequence in Fig. 7a)
can appear in —5) x —5) = 13225 distinct positions
in the rectangular interleaver, and in each case 37’ = :17 and a
codeword of weight dfree = 12 results, this results in 13225
free—distance codewords. Note that for every Occurrence of the
“square” sequence to result in a codeword of weight 12 the
weight of both parity sequences must be invariant to which
is punctured ﬁrst. The “rectangular” sequences in Fig. 7b) and c) also result
in weight 12 codewords. For these two sequences, the weight
of one of the parity sequences is affected by whether or not it
is punctured ﬁrst and only every other position in which the
“rectangular” sequences appear in the interleaver results in a
codeword of weight dﬁee = 12. Thus the sequences in Fig. 7b)
and c) each result in 0.5(W— 10) x — 5) : 6325 free
distance codewords. For the “square” sequence in Fig. 7d),
the weight of both parity sequences is affected by which is
punctured ﬁrst and only one out of four positions in which
this “square” sequence appears in the interleaver results in a
codeword of weight dfree = 12. Consequently, this “square”
sequence results in 0.25(\/1V— 10) x (x/TV— — 10) = 3025 free—
distance codewords. Summing the contributions of each type
of sequence results in a total of thee 2 28900 codewords of
weight dfree = 12. It is tempting to try to improve the performance of a Turbo
code with a rectangular interleaver by increasing the size of the
interleaver. However, all of the information sequences shown
in Fig. 7 would still occur in a larger rectangular interleaver,
so the free distance cannot be increased by increasing N.
Also, since the number of freedistance codewords is on the
order of N, increasing the size of the interleaver results
in a corresponding increase in Nfree such that the effective
multiplicity Nfree/N does not change signiﬁcantly. Without
the beneﬁt of a reduced effective multiplicity, the freedistance
asymptote, and thus the “error ﬂoor,” of Turbo codes with
rectangular interleavers is not lowered enough for them to
manifest the excellent performance of Turbo codes with pseu
dorandom interleavers for moderate BER’ s. Attempts to design
interleavers for Turbo codes generally introduce structure to
the interleaver and thus destroy the very randomness that
results in such excellent performance at low SNR’s. III. THE DISTANCE SPECTRUM OF TURBO CODES In the previous section, it was shown that the “error ﬂoor”
observed in the performance of Turbo codes is due to their
relatively low free distance. It is now shown that the out—
standing performance of Turbo codes at low SNR’s is a
manifestation of the sparse distance spectrum that results when
a pseudorandom interleaver is used in a parallel concatenation
scheme. To illustrate this the distance spectrum of an “average”
(37, 21, 65536) Turbo code is found and its relationship to the
performance of the code is discussed. The distance spectrum
of the “average” Turbo code is then compared to the distance 1704 33
ME
55
0
mm
99
1
53 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 42, NO. 6, NOVEMBER 1996 Ea ! 33353553
535333535 3 3 5 5 5 3 5 3 5 5 5 3 5 3 3 533335335 35333555555
333533335335
533355535335 Fig. 7. Portions of the interleaver and information sequences resulting in free—distance codewords for a (37,21,14400) Turbo code with a 120 X 120 rectangular interleaver. spectrum of the (2,1,14) code. An “average” Turbo code is
one whose properties have been averaged over all possible
pseudorandom interleavers as in [8]. By doing this, the analysis
of Turbo cedes given in [8] and the algorithm for ﬁnding the
distance spectrum given in [28] are simpliﬁed. Using the algorithm described in [28], an “average”
(37, 21, 65536) Turbo code was found to have the following
distance spectrum d Nd ’Wd
6 4.5 9 8 11 22
10 20.5 41
12 75 150 where Nd is the total number of codewords of weight d and
Wd : Ndrfid is the total information weight of all codewords
of weight d. The distance spectrum information for a particular
distance d is referred to as a spectral line. This data can be
used in conjunction with the bound of (2) to estimate the
performance of the code. In addition, by plotting each term
of (2) the contribution of each spectral line to the overall
performance of the code can be estimated. The performance of a particular (37, 21, 65536) Turbo code
is shown in Fig. 8 along with curves showing the contribution
of each spectral line for an “average” Turbo code with the same interleaver length. This clearly shows that the contribution to
the code’s BER by the higher distance spectral lines is less
than the contribution of the free—distance term for E1, /N0’s
greater than 0.5 dB. (The higher distance spectral lines become
important, though, as capacity is approached.) Thus the free
distance asymptote dominates the performance of the code I
not only for moderate and high Eb/Nos but also for low
Eb/No’s. We characterize distance spectra for which this is
true as sparse or spectrally thin; ' I A. Comparison to the (2,1,14) Code The ramiﬁcations of a sparse—distance spectrum are made
evident by examining the distance spectrum of convolutional
codes. The (2, 1, 14) convolutional code introduced in Section
II has the following distance‘spectrum: ' ‘ d rtg .773 18 33 187 20 136 1034
22 835 7857
24 4787 53994 I
26 27941 361762
28 '162513 2374453
30 945570' 15452996
32 %%M4%%%% PEREZ et al.: A DISTANCE SPECTRUM INTERPRETATION OF TURBO CODES 10 .—r—Tr mm?
' G—OSimulation —
10" .— Edm=6 _
E <><>d=s 3
L AAd=10 :
4 — ~<ld=12
10  a.
.u] Bit Error Rate
H
a
 H
G.
a Eb/No (dB) Fig. 8: Performance of a (377 21. 65536) Turbo code decomposed by spec—
tral line. .vrvrn.. HSimuhtions
*m'iédmsm
0"”0 d=20 ,,
ElmEl d=22
0"<>d=24
AN"A d=26
<l I> I'I—l—r—l'l—V1l—I'III—Tl ~<1d=2s
~+> d=30 1.1.1.141 .M : [KJLkJL—ILJ‘IJJJJ. Bit Error Rate
5 0.0 1.0 2.0 3.0
Eth,] (dB) Fig. 9. Performance of the (2.1,14) MFD convolutional code decomposed
by spectral line. as reported in [45]. When comparing the distance spectrum
of a convolutional code and a Turbo code, it is important to
remember that for a convolutional code Nd z N X N2 for
the low~weight codewords. Fig. 9 shows the performance of
this code and the contribution of each spectral line from the
bound of (3). In this case, the contribution of the higher distance spectral
lines to the overall BER is greater. than the contribution
of the, freedistance term for E1, /N0’s less than 2.7 dB,
which corresponds to BER’s of less than 10*6! The large
SNR required for the freedistance asymptote to dominate the
performance of the (2, 1, 14) code is due to the rapid increase
in the path multiplicity for increasing d. We characterize
distance spectra for which this is true as spectrally dense. The 1705 dense distance spectrum of convolutional codes also accounts
for the discrepancy between the real coding gain at a particular
SNR and the asymptotic coding gain calculated using just the
free distance [44]. Thus it can be concluded that the outstanding performance
of Turbo codes at low signalto—noise ratios is a result of the
dominance of the freedistance asymptote, which in turn is a
consequence of the sparsedistance spectrum of Turbo codes,
as opposed to spectrally dense convolutional codes. Finally,
the sparsedistance spectrum of Turbo codes is due to the
structure of the codewords in a parallel concatenation and the
use of pseudorandom interleaving. IV. SPECTRAL THINNING In this section, the observations made concerning the dis
tance spectrum and spectral thinning of Turbo codes is formal
ized from the point of View of random interleaving. Random
interleaving was introduced in [6]—[9] to develop transfer func
tion bounds on the average performance of Turbo codes and
to explore issues of code design. Here, random interleaving
is used to explore the effect of the interleaver on the distance
spectrum of the code. In order to simplify the notation and
discussion, only nonpunctured Turbo codes are considered
explicitly. The extension to punctured codes is straightforward
and may be found in [28]. ' The fundamental idea of random interleaving is to consider
the performance of a Turbo code averaged over all possible
pseudorandom interleavers of a given length. For a given
N, there are N! possible pseudorandom interleavers and,
assuming a uniform distribution, each occurs with probability
1 / N !. Let a particular interleaver map an information sequence
a: of weight w to an information sequence x’, also of weight
11). Then there are a total of w!(N — w)! interleavers in the
ensemble of N! interleavers that perform this same mapping.
Thus the probability that such a mapping occurs and hence
that the codeword that results from the input sequences a: and
35’ occurs, is w!(N —w)! 1 ‘65) Following [8], deﬁne the input redundancy weight enumer
ating function (IRWEF) of a systematic code as A(W. 2) = Z Z AWWWZZ <6) w I where Aw1z is the number of codewords of weight d = w + 2 generated by input sequences of weight w and parity sequences
of weight 2. The goal is now to develop a relationship between
the codewords in the constituent encoders and AM for the
Turbo code and to see how that relationship changes with the size of the interleaver. Recall from Section II that a Turbo codeword is essentially
the combination of a codeword from the ﬁrst constituent
encoder plus a codeword from the second constituent encoder.
A codeword of weight d1 = w + 21 from the ﬁrst constituent
encoder caused by an information sequence ac of weight w 1706 is composed of m error events of total length ll. To avoid
difﬁculties in counting codewords when the second constituent
encoder is left unterrninated, we consider different orderings
of the same error events as distinct cases. The ordered set of
n1 error events in the ﬁrst encoder is denoted by 51. The
information sequence a: that results in the set 31 is mapped
by a particular interleaver to the information sequence m’, also of weight in, which is then encoded by the second constituent
encoder. This results in a codeword of weight d2 = w + 22, with the ordered set 5'2 consisting of n2 error events of total
length l2. For example, Fig. 3 depicts a codeword of weight d1 =
4 + 3 = 7 in the ﬁrst constituent encoder caused by an
information sequence of weight w = 4 and composed of
m = 2 error events of total length ll 2 5 + 3 = 8. For
the interleaver described in Section II, a: is mapped to an :13’
that results in a codeword of weight d2 = 4 + 3 : 7 in the
second constituent encoder consisting of n2 2 2 error events
of total length l2 : 3 + 3 = 6. Thus this 51 and 5'2 result in a
codeword of weight 10 and a single contribution to A45 of the
Turbo code for this particular interleaver. Averaged over the
ensemble of interleavers of length N, a set 51 with information
weight w and parity weight 21 and a set 52 with information
weight w and parity weight 22 will contribute a fraction N
w
to the enumerating function coefﬁcients AW,“2 of an “av—
erage” Turbo code.
Because the sequence of zeros connecting any two distinct error events has no effect on the weight of the information
sequence'or the parity sequence, there are <N—l1+n1> (8) 711 ways that the ordered set 51 of n1 error events can be arranged
such that their contribution to AwyzlﬂZ is not changed. This
is simply the number ofways in which n1 distinct error events
can be arranged in a sequence of length N while maintaining
the order in which they appear. Similarly, if the codeword in
the second constituent encoder ends in the allzero state, then
the ordered set Sg will make (N—l2+n2) (9) 712 contributions to Awning. However, because the second en—
coder is not guaranteed to return to the allzero state, it is
possible that the last of the ﬁg error events is not actually an
error event, but instead ends in a nonzero state. In this case,
the last error event cannot be moved and the set 82 makes (N—l2+(n2—1)> (n2 — 1) “0) contributions to Aw721+22. IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 42, NO: 6,'NOVEMBER 1996 The contribution to the distance spectrum of a Turbo code
due to any pair of ordered sets 81 and $2, averaged over the
ensemble of pseudorandom interleavers, can now be computed
using (7)—(lO). If the codeword in the second constituent en—
coder happens to end in the allzero state, then the contribution
of an 31 and 5’2 to Awﬁzlirz2 is given by N—ll—l—nl N4l2+7l2
( m >< n2 )
I N . .. . .
(w)
If the codeword in the second constituent encoder does not
end in the all—zero state, then the contribution to Aw7zl+Z2 is given by
N—l1+n1><N—12+(n21))
“1 (712—1) _ (if) The expressions in (11) and (12) can now be used to explore
the effect of changing the interleaver size on the distance ,
spectrum of an “average” Turbo code. Since we are primarily concerned r with low—weight
codewords in the distance spectrum, we assume that N >
n1,n2,l1, and £2. If this is not true, then 51 and 52 either
contain a, large number of short error events or a few long
error events. In both cases, it is very unlikely that the result
is a codeword of low weight. With this assumption, (11) can
be approximated by ' (11) (12) w! Nn1+ng~w nilngl ‘
where, without loss of generality, it is assumed that m 2 722.
Since each error event is caused by an information sequence
of weight at least two, to Z 2711. The behavior of (13) for
increasing N can be broken down into three casesr’
1) 711 > 71;: r
The exponent of N is strictly negative’and the contribu—
tion to Aw,le2 decreases as N increases.
2) n1 =n2 and w>2n1: The exponent of N is strictly negative and the contribu—
tion to vale2 decreases as N increases.
3) n1 : ng and w : 2721:
The exponent of N is exactly zero and the contribution
to Aw,zl+22 converges to a ﬁnite value as N increases. For N >> n1, n2, [1, and Z2, (12) can be approximated by w! N711 +112 —w‘—1
n1!(n2 — 1)! (14)
where w 2 2m. However, since the tail in the second encoder
may be caused by an information sequence of weight 1, we
also have in Z 2712 — 1. The behavior of (14).for increasing
N can be broken down into three cases:
1) 711 > m: _
Since 112 2 2711, the exponent of N is strictly nega— tive and the contribution to Awyzl+22 decreases as N
increases. PEREZ et al.: A DISTANCE SPECTRUM INTERPRETATION OF TURBO CODES 2) n1 2 n2:
Again, since u; 2 2m, the exponent of N is strictly
negative and the contribution to Aw7zl+Z2 decreases as
N increases. 3) m < m:
Since 11) 2 2m — 1, the exponent of N is strictly
negative and the contribution to Aw,le2 decreases as
N increases. The following lemma can now be stated. A similar result
was proven in [9]. 1 Lemma 1: Given a Turbo code based on two systematic
feedback encoders and a pseudorandom interleaver of length
N in which the ﬁrst encoder is assumed to be forced back
to the allzero state, the contribution of two ordered sets of
error events 81 and $2 with the same information weight
to the distance spectrum of the Turbo code, averaged over
all pseudorandom interleavers of length N, converges to a
nonzero constant as N —> 00, if and only if 1) 82 leaves the second encoder in the allzero state; 2) 81 and 8'2 contain the same number of error events; 3) each error event in $1 and $2 is caused by a weight two
information sequence. In all other cases, the contribution goes to zero as N —> 00. For each term Awyz in the input redundancy weight enu
merating function of a Turbo code there is a ﬁnite number
of pairs of sets 5'1 and 52 that contribute to it. If either set
contains a long error event, then it is possible that that pair
will be excluded for small interleavers. As the interleaver size
increases, eventually all pairs of sets will be allowed and
any further increase in N will not result in additional pairs
of sets contributing to AM. Also, as N —> 00, AM will
be determined by pairs of 31 and 82 that satisfy the three
conditions of Lemma 1, and thus each Awﬁz will converge
to a ﬁnite value. Since each spectral line is a ﬁnite sum of
A,” terms, each spectral line converges to a ﬁnite value as
the interleaver size increases. The convergence of each spectral line to a ﬁnite value as
the size of the interleaver increases results in spectral thinning.
That is, for small interleavers there may be pairs of sets 51
and 52 that do not satisfy the conditions of Lemma 1, but
which contribute to the multiplicity of a lowweight spectral
line. As the size of the interleaver increases the number of
these aberrational sets decreases until the spectral line reaches
its ﬁnal value as determined by Lemma 1. This process of
spectral thinning is represented graphically in Fig. 10, which depicts the thinning of lowweight codewords as the size of the interleaver increases for hypothetical distance spectra. It is this
thinning of the distance spectrum that enables the free—distance
asymptote of a Turbo code to dominate the performance for
low SNR and thus to achieve near—capacity performance. A. Primitive Polynomials and Free Distance We now consider the ramiﬁcations of Lemma 1 with respect
to the freedistance codewords of a Turbo code. That is, what
does Lemma 1 imply about the information sequences that
generate the freedistance codewords in an “average” Turbo
code? ‘ 1707 N Inﬁnite Spectral Thinning Multiplicity, Nd Hamming Distance, d Fig. 10. Graphical representation of spectral thinning for increasing inter
leaver size. For an “average” Turbo code, Lemma 1 states that as
the size of the interleaver increases each spectral line is the
result of contributions only from pairs of ordered sets of
error events in which each error event is caused by a weight—
two information sequence. It is reasonable to expect that the
freedistance spectral line will be among the ﬁrst spectral
lines to converge to its ﬁnal value. Thus for reasonably large
interleavers, the free distance will be determined by the sets. 5'1
and 52 satisfying the conditions of Lemma 1. Let 31 and 52 be
any pair of error events caused by a weight—two information
sequence that results in a minimum—weight parity sequence in
the ﬁrst and second constituent encoders, respectively. Note
that there may be more than one such pair of minimumweight
error events for the constituent encoders. A free—distance codeword in an “average” Turbo code
must be the result of sets 31 and 82 that consist of only
those minimum»weight error events 31 and 32, respectively.
Furthermore, since each additional error event in either 5'1 and
82 adds weight to the codeword, 51 and 82 must each contain
only one minimumweight error event. (If each error event
does not add weight, then a weight—two information sequence
exists that generates a zero weight parity sequence and the
free distance of the code would be 2.) Thus the freedistance
codewords of an “average” Turbo code are caused by weight—
two information sequences, provided the interleaver is large
enough. Therefore we have the following Lemma. Lemma 2: For an “average” Turbo code, as the size of the
interleaver N approaches 00: ' i 1) the free distance codewords are caused by information
sequences of weight 2; 2) the free distance of an “average” Turbo code is maxi—
mized by choosing constituent encoders that have the
largest output weight for weighttwo information se
quences. Based on this lemma, we now present an intuitive argument
showing that choosing the feedback polynomial h0(D) in a
(2, 1, V) systematic feedback encoder to be a primitive polyno
mial maximizes the output weight for weighttwo information
sequences. It follows that the free distance of an “average”
Turbo code is maximized by using a primitive polynomial 1708 as the feedback polynomial in the constituent encoders. This
result was independently derived in [9] using the transfer
function‘of an “average” Turbo code. The generator matrix of a (2, 1,1/) systematic feedback
encoder is given by ' 2:831 an), 2 [1 where hl (D) and h0(D) are referred to as the feedforward and
feedback polynomials, respectively, and h0(D) is of degree
1/. Since only information sequences of weight 2 are being
considered, the systematic output contributes weight 2 to the
overall codeword weight for all the encoders being considered. Therefore, only the weight contributed by the parity sequence,
that is, h1(D) MD) = ma?) MD), needs to be maximized. Furthermore, since we are concerned
only with the choice of h0(D), MO?) is assumed to be a
polynomial such that h0(D) and hl (D) are relatively prime.
(There is empirical evidence that the choice of both polyno—
mials can affect the performance of the code [28], [30], but
we will not address that issue in this paper.) Let 1 + BK, for some ﬁnite K , be the shortest input
sequence of weight 2 that generates a ﬁnite~length codeword.
The resultant parity sequence is hiiD) _ hl(D)
l10(1)) T hog?) K . K 711(1))
y(D) (1+D) .D MUD).
Since y(D) is of ﬁnite length, h1(D)/h0(D) must be periodic
with period K. Increasing the period K increases the length
of the shortest weight 2input sequence that generates a ﬁnite
length codeword and therefore increases the length of that
codeword. Intuitively, one would expect that increasing its
length would result in the codeword gaining weight. That is,
on average, half of the added bits would be ones. A strictly proper rational function of two polynomials, like
h1(D)/h0(D), is periodic with period K g 2" — 1. The
period is maximized, that is, K = 2" — 1, when h0(D) is a
primitive polynomial. Since the free distance of an “average”
Turbo code is determined by information» sequences of weight
2, for sufﬁciently large interleavers the free distance will be
maximized by maximizing K. Therefore, choosing h0(D)_ to
be a primitive polynomial will result in a larger free, distance
for an “average” Turbo code. To test this, we compare a (37 , 21,400) Turbo code which
has K = 5 and free distance dfree' : 6 to a (23,35,400)
Turbo code. Both codes are punctured as in [l]. The feedback
polynomial ho : 23 in the second Turbo code is a primitive
polynomial of degree V =‘ 4 and thus 1/h0 has a period
of K = 2" — 1 = 15. The free distance of this Turbo
code was found to be dfree = 10‘ [12], [28]. Fig. 11 shoWs
simulation results for these two codes using the iterative
decoding algorithm of [l] with 18 iterations. As expected, IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 42,] NO. 6, NOVEMBER 1996 10' E  . w r 'u  »  x   <—‘
i e—e_(37,21,400) ‘ i
 ~“ (23,35,400) Primitive
101 ' ‘E
a 10‘3 — '~
N I
a:
E
In!
5
i5 104 g
105 3‘ _l
E
10“ .1“  . 4* I r r r    . ._x 1 . . r r . . . .
0.0 10 « 2.0 3.0
rah/man).
Fig. 11. Simulation results for a (37,21,400) Turbo code and 21 (23,35,400) primitive Turbo code. the second Turbo code performs better at moderate and high
SNR’s because its free distance'asymptote is steeper due to
the increased free distance. V. CONCLUSIONS . The excellent performance of Turbo codes may be explained
in terms of the distance Spectrum of the code. The “error ﬂoor”
observed in Simulations of Turbo codes is a manifestation of
the free—distance asymptote. Since Turbo codes haverelatively
low free distances, the free—distance asymptote has a shallow
slope, and thus the performance curves ﬂatten out at moderate
to high SNR’s. The “error ﬂoor” may be lowered by increasing
the size of the interleaver for a ﬁxed free distance, that is, by
reducing the effective multiplicity of the code. Altemately, for
ﬁxed interleaver lengths, the performance may be‘improved
for moderate and high SNR’s by increasing the free distance.
Choosing primitive polynomials as the feedback polynomials
in the constituent encoders usually results in an increased free
distance. _ , The exceptional performance of Turbo codes at low SNR’s
is due to the sparse distance spectrum and the resultant
ability of the code to follow the freedistance asymptote at ‘
moderate to low SNR’S. The use of systematic feedback
encoders and pseudorandom interleavers results in spectral
thinning, in which information sequences which generate low— ~
weight parity sequences from the ﬁrst constituent encoder are
interleaved with high probability to information sequences
that generate high—weight parity sequences in the second
constituent encoder. Spectral thinning is enhanced by increas
ing interleaver lengths. For very large interleavers, spectral
thinning results in a sparse distance spectrum in which the ﬁrst
several spectral lines are determined Solely by input sequences
of weight two. Thus spectral thinning results in few low—weight
codewords and a large number of codewords of “average”.
weight. This is very similar to the type of distance spectrum
achieved by “randomlike” codes [4]. PEREZ et al.: A DISTANCE SPECTRUM INTERPRETATION OF TURBO CODES In a more philosophical light, Turbo codes remind us
that informationtheoretical arguments imply that long block
lengths, but not necessarily large free distances, are required to
achieve capacity at moderate ‘BER’s. Thus like convolutional
codes, Turbo codes are a class of codes that achieve long block
lengths, but without the corresponding increased density of
the distance spectrum common to convolutional codes, and
for which a practical, albeit nontrivial, decoding algorithm
exists. In addition, Turbo codes are timevarying due to the
pseudorandom interleaver, and the time—varying structure is
essential in achieving the thin distance spectrum that results in
near—capacity performance at moderate BER’s. This suggests
that some effort should be made to ﬁnd other classes of
timevarying codes, and decoding algorithms, that have thin
distance spectra, rather than just large free distances. Finally,
since, in fact, long block lengths are required to achieve near
capacity performance at moderate BER’s, only modest coding
gains will be achievable in systems that use relatively short
block lengths. REFERENCES [l] C. Berrou, A. Glavieux, and P. Thitimajshima, “Near Shannon limit
errorcorrecting coding and decoding: Turbo—codes,” in Proc. 1993
IEEE Int. Conf. on Communication (Geneva, Switzerland, 1993), pp. 1064—1070.
[2] C. Berrou and A. Glavieux, “Turbocodes: General principles and appli— cations,” in Proc 6th Tirrenia Int. Workshop on Digital Communications
(Tirrenia, Italy, Sept. 1993), pp. 215—226. [3] G. Battail, C. Berrou, and A. Glavieux, “Pseudo—random recursive
convolutional coding for near—capacity performance,” in Proc. Commu—
nication Theory MiniConf, Globecom ’93 (Houston, TX, Dec. 1993),
pp, 2327. [4] G. Battail, “On random—like codes,” preprint. [5] , “Pseudo—random Turbo—codes,” in Proc. Int. Symp. on Signals,
Systems and Electronics (San Francisco, CA, Oct. 25—27, 1995), pp. 419—422. I
[6] 'S. Benedetto and G. Montorsi, “Average performance of parallel con— catenated block codes,” Electron. Lett., vol. 31, no. 3, pp. 156—158, Feb. 2, 1995.
, “Performance evaluation of TURBOcodes,” Electron. Lett., vol. 31, no. 3, pp. 163—165, Feb. 2, 1995. , “Unveiling turbo codes: some results on parallel concatenated coding schemes,” IEEE Trans. Inform. Theory, vol. 42, no. 2, pp. 409—428, Mar. 1996. [9] , “Design of parallel concatenated convolutional codes,” IEEE
Trans. Commun., vol. 44, no. 5, pp. 591—600, May 1996. [10] J. D. Andersen, “The TURBO coding scheme,” Rep. IT—146, Tech. Univ. of Denmark, Lyngby, June 1994. [11] J. D. Andersen, “TURBO coding for deep space applications,” in Proc
1995 IEEE Int. Symp. on Information Theory (Whistler, BC, Canada,
Sept. 1995), p. 36. [12] P. Robertson, “Illuminating the structure of parallel concatenated re
cursive systematic (TURBO) codes,” in Proc. GLOBECOM ’94 (San
Francisco, CA, Nov. 1994), vol. 3, pp. 1298—1303. [13] J. Hagenauer and L. Papke, “Decoding ‘Turbo’—codes with the soft
Output Viterbi algorithm,” in Proc. 1994 IEEE Int.. Symp. on Information * Theory (Trondheim, Norway, June 1994), p. 164. [14] J. Hagenauer, P. Robertson, and L. Papke, “Iterative (TURBO) decoding
of systematic convolutional codes with MAP and SOVA algorithms,” in
Proc. ITG Conf on “Source and Channel Coding” (Frankfurt, Germany, Oct. 1994), pp. 1—9.
[15] J. Hagenauer, E. Offer, and L. Papke, “Iterative decoding of binary block and convolutional codes,” IEEE Trans. Inform. Theory, vol. 42, no. 2, pp. 429—445, Mar. 1996.
[16] Y. V. Svirid, “Weight distributions and bounds for turbo codes,” European Trans. Telecommun, vol. 6, no. 5, pp. 543—556, Sept—Oct. 1995.
[17] Y. V. Svirid and S. Riedel, “Threshold decoding of turbo codes,” in Proc. 1995 IEEE Int. Symp. on Information Theory (Whistler, BC,
Canada, Sept. 1995), p. 39. [7]
[8] 1709 [18] D. Divsalar and F. Pollara, “Lowrate turbo codes for deep—space
communications,” in Proc. 1995 IEEE Int. Symp. on Information Theory
(Whistler, BC, Canada, Sept. 1995), p. 35. [19] D. Divsalar and F. Pollara, “Turbo codes for PCS applications,” in Proc. 1995 IEEE Int. Conf. on Communications (Seattle, WA, June 1995).
, “Turbo codes for deep~space communications,” TDA Progr. Rep. 42—120, Jet Propulsion Lab., Pasadena, CA, pp. 2939, Feb. 15,
1995. [20] [21] , “Multiple turbo codes for deep—space communications,” TDA Progr. Rep. 42—121, Jet Propulsion Lab., Pasadena, CA, pp. 66~77,
May 15, 1995. [22] D. Divsalar, S. Dolinar, F. Pollara, and R. J. McEliece, “Transfer
function bounds on the performance of turbo codes,” TDA Progr.
Rep. 42—122, Jet Propulsion Lab., Pasadena, CA, pp. 44—55, Aug. 15,
1995. [23] S. Dolinar and D. Divsalar, “Weight distributions for turbo codes using »
random and nonrandom permutations,” TDA Progr. Rep. 42—122, Jet Propulsion Lab., Pasadena, CA, pp. 56—65, Aug. 15, 1995.
[24] D. Divsalar and F. Pollara, “On the design of turbo codes,” TDA Progr. Rep. 42—123, Jet Propulsion Lab., Pasadena, CA, pp. 99—121, Nov. 15,
1995. [25] J .—F. Cheng and R. J. McEliece, “Unit—memory Hamming turbo codes,”
in Proc. 1995 IEEE Int. Symp. on Information Theory (Whistler, BC,
Canada, Sept. 1995), p. 33. . . [26] R. J. McEliece, E. R. Rodemich, and J.F. Cheng, “The turbo decision
algorithm,” in Proc. 33rd Annu. Allerton Conf. on Communication,
Control and Computing (Monticello, IL, Oct. 1995), p. 366. V [27] D. Divsalar and R. J. McEliece, “The effective free distance of turbo
codes,” Electron. Lett., submitted for publication. [28] J. Seghers, “On the free distance of TURBO codes and related product
codes,” Final Rep., Diploma Project SS 1995, no. 6613, Swiss Federal Institute of Technology, Zurich, Switzerland, Aug. 1995.
[29] J. Seghers, L. C. Perez, and D. J. Costello, Jr., “0n selecting code generators for turbo codes,” in Proc. 33rd Annual Allerton Conf. on
Communication, Control and Computing (Monticello, IL,'OCt. 1995),
pp. 357—361. [30] D. Arnold and G. Meyerhans, “The realization of the the turbocoding
system,” Semester Project Rep, Swiss Federal Institute of Technology,
Zurich, Switzerland, July 1995. [31] A. S. Barbulescu and S. S. Pietrobon, “Interleaver design for turbo
codes,” Electron. Lett., ,vol. 30, no. 25,,p. 2107, Dec. 8, 1994. , “Terminating the trellis of turbocodes in the same state,” Electron. Lett., vol. 31, no. 1, pp. 22—23, Jan. 5, 1995. , “Rate compatible turbo codes,” Electron. Lett., vol. 31, no. 7,
pp. 535—536, Mar. 30, 1995. [34] _, “Interleaver design for three dimensional turbo codes,” in Proc.
1995 IEEE Int. Symp. on Information Theory (Whistler, BC, Canada,
Sept. 1995), pg. 37. [35] M. C. Reed and S. S. Pietrobon, “Turbo—code termination for short
frames,” Electron. Lett., submitted for publication. [36] O. Joerssen and H. Meyr, “Terminating the trellis of turbocodes,”
Electron. Lett., vol. 30, no. 16, pp. 1285—1286, Aug. 4, 1994. [37] P. Jung and M. Nthan, “Performance evaluation of turbo codes for
short frame transmission systems,” Electron. Lett., vol. 30, no. 2, pp.
111—113, Jan. 20, 1994. , “Dependence of the error performance of turbo—codes on the
interleaver structure in short frame transmission systems,” Electron.
Lett., vol. 30, no. 4, pp. 285—288, Feb. 17, 1994. [39] P. Jung, “Novel low complexity decoder for turbo—codes,” Electron.
Lett., vol. 31, no. 2, pp. 86—87, Jan. 19, 1995. ' [40] R. Podemski, W. Holubowicz, C. Berrou, and A. Glavieux, “Distance
spectrum of the turbocodes,” in Proc. 1995 IEEE Int. Symp. on Infor
mation Theory (Whistler, BC, Canada, Sept. 1995), p. 34. [41] R. Podemski, W. Holubowicz, C. Berrou, and G. Battail, “Hamming
distance spectra of turbo—codes,” Ann. Te’le’commun., vol. 50, no. 910,
pp. 790—797, Sept—Oct. 1995. [42] G. Caire, G. Taricco, and E. Biglieri, “On the convergence of the iterated
decoding algorithm,” in Proc. 1995 IEEE Int. Symp. on Information
Theory (Whistler, BC, Canada, Sept. 1995), p. 472. [43] N. Wiberg, H.—A. Loeliger, and R. Kotter, “Codes and iterative decoding
on general graphs,” European Trans. Telecommun, vol. 6, no. 5, pp. 513—525, Sept—Oct. 1995.
[44] S. Lin and D. J. Costello, Jr., Error Control Coding: Fundamentals and Applications. Englewood Cliffs, NJ ; Prentice—Hall, 1983.
[45] M. Cedervall and R. Johannesson, “A fast algorithm for computing
distance spectrum of convolutional codes,” IEEE Trans. Inform. Theory, vol. 35, no. 6, pp. 1146—1159, Nov. 1989. [321 [33] [38] ...
View
Full
Document
 Spring '12
 Shea

Click to edit the document details