TurboDistanceSpectrum_Costello

TurboDistanceSpectrum_Costello - 1698 IEEE TRANSACTIONS ON...

Info iconThis preview shows pages 1–12. Sign up to view the full content.

View Full Document Right Arrow Icon
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 2
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 4
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 6
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 8
Background image of page 9

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 10
Background image of page 11

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 12
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 1698 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 42, NO. 6, NOVEMBER I996 A Distance Spectrum Interpretation of Turbo Codes Lance C. Perez, Member, IEEE, Jan Seghers, and Daniel J. Costello, In, Fellow, ,IEEE * Abstract— The performance of Turbo codes is addressed by V _ examining the code’s distance spectrum. The “error floor” that occurs at moderate signal-to-noise ratios is shown to be a conse- quence of the relatively low free distance of the code. It is also shown that the “error floor” can be lowered by increasing the size of the interleaver without changing the free distance of the code. Alternatively, the free distance of the code may be increased by using primitive feedback polynomials. The excellent performance of Turbo codes at low signal-to-noise ratios is explained in terms ‘ of the distance spectrum. The interleaver in the Turbo encoder is shown to reduce the number of low-weight codewords through a process called “spectral thinning.” This thinned distance spec- trum results in the free distance asymptote being the dominant performance parameter for low and moderate signal-to-noise ratios. — Index Terms—Turbo codes,'convoluti0nal codes, distance spec- ‘ trum. I. INTRODUCTION HE DISCOVERY of Turbo codes and the near—capacity performance reported in [1] has stimulated a flurry of research efforts to fully understand this new coding scheme [2]—[43]. Initially greeted with some skepticism, the original results were independently reproduced by several researchers [6], [7], [10]—[13], and [30]. Subsequently, recent research on Turbo codes hasfocused on understanding the reasons for their outstanding performance [8], [9], [16] [22]—[24], [28]. At this point, there are two fundamental questions regard— ing Turbo codes. First, does the iterative decoding scheme presented in [1] always converge tothe optimum solution? Second, assuming optimum or-near-optimum decoding, why do the Turbo codes perform so well? In this paper, we'address the second issue in a semitutorial manner by examining the distance spectrum of Turbo codes. In doing so, we will draw on the work of several research groups involved with Turbo codes [6]—[9], [12]—[14], [18]—[24]. Due to the intense interest in this subject, many results involving Turbo codes have been developed independently by others and the reader is encouraged to peruse the references for an alternative point of View. In particular, the recent papers by Benedetto and Montorsi [8], [9] were the first to offer a comprehensive picture of Turbo codes. ‘ . Manuscript received January 18, 1996; revised April 30, 1996. This work was supported by NASA under Grant NAG 5—557, by NSF under Grant NCR95-22939, and by the Lockheed—Martin Corporation. L. C. Perez was with the Division of Engineering, The University of Texas at San Antonio, San Antonio, TX 78229 USA. He is now with the Department of Electrical Engineering, The University of Nebraska»Lincoln, Lincoln, NE 68588—05 11 USA. ' J. Seghers is with the Institute for Signal and Information Processing, Swiss Federal Institute of Technology, Zurich, Switzerland. D. .l. Costello, Jr. is with the Department of Electrical Engineering, University of Notre Dame, Notre'Dame, IN 46616 USA. Publisher Item Identifier S 0018*9448(96)07521—9. 10‘ .. ' ' ‘ ‘ ‘ "3 M (37,21,65536) Turbo _ H (2,1,14) MFD Convolutional vn—mr, r-r-n-n-rq Bit Error Rate H c: . 10 E ----------------------------------------------------------------------------------------------------------- 10“ T E 10-7 - A i . r . I . . i . . , . . I . . r . . . i . i 0.0 1.0 2.0 - 3.0 Eth, (dB) Fig. 1. Simulation results for a(37,k21, 65536) Turbo code and a (2, 17 14) MFD convolutional code. ' The simulated performance of a rate 1 / 2 Turbo code with the same parameters as in [1] is shown in Fig. 1 along with simulation results for a rate R = 1/2, memory V : 14, convolutional code. The comparison of these‘simulation results raises two issues regarding the performance 0f Turbo codes. First, what is it that allows Turbo codes to achieve a bit error rate (BER) of 10“5 at a signal-to—noise ratio (SNR) of Eb/No = 0.7 dB, which is only 0.7 dB from the Shannon limit? Second, what causes the “error flOor” [10], [13], that is, the flattening of the performance curve, for moderate to high SNR’s? Here, we endeavOr to explain the performance of Turbo codes, and thus address these two issues, in terms of the code’s distance spectrum. We do not attempt to address the many interesting questions concerning the iterative decoding method (see, e.g., [15], [26], [42], and [43]), i.e., we assume that an optimum or near—optimum decoder is available. In order to explain their performance in terms of the free distance and the distance spectrum, we willexamine the codeword structure of Turbo codes in detail. Here, the free distance is defined to be the minimum Hamming weight of all possible codewords and the error coefficient is the total number, or multiplicity, of free—distance codewords. The goal is to use specific examples to elucidate the key structural properties that result in the near—capacity performance of Turbo codes at BER’s around 10—5. As will be seen, this effort leads to an explanation that applies to Turbo. codes and also lends insight into designing codes in general. Throughout the paper, Turbo codes are compared to a maximum free—distance, rate 0018—9448/96$05.CO © 1996 IEEE PEREZ et al.: A DISTANCE SPECTRUM INTERPRETATION OF TURBO CODES Jti Interleaver Fig. 2. Block diagram of a Turbo encoder with two identical constituent encoders (h0(D) = 1 + D2,h1(D) = D) and without puncturing. ' R = 1/2, memory 11 = 14, i.e., a (2, 1, 14), convolutional code to emphasize the differences in performance and structure. Techniques for analyzing the performance of Turbo codes using transfer functions and related methods may be found in [6], [9], and [22]. The paper begins with a detailed examination of the struc- ture of codewords in a Turbo code in Section II. This leads to the calculation of the free distance of a particular Turbo code and an explanation for the “error floor” in its performance curve. In Section III, the distance spectrum of “average” Turbo codes is considered and a theory called spectral thinning is introduced and used to explain the performance of Turbo codes at low SNR’s. The idea of spectral thinning is then formalized - in Section IV through the use of random interleaving. Finally, some conclusions are drawn concerning the distance spectrum of Turbo codes and the consequences of this on the design of codes in general. _ II. THE FREE DISTANCE OF TURBO CODES In order to find the free distance of a Turbo code, it is necessary to understand the basic structure of the encoder and the resulting codewords. A typical Turbo encoder consists of the parallel concatenation of two or more, usually identical, rate 1/2 encoders, realized in systematic feedback form, and an interleaver. This encoder structure is called a parallel concatenation because the two encoders operate on the same set of input bits, rather than one encoding the output of the other. A block diagram of a Turbo encoder with two constituent convolutional encoders is shown in Fig. 2. For the remainder of the paper, only Turbo encoders with two identical constituent convolutional encoders are considered, though the conclusions are easily extended to the general case. The interleaver is used to permute the input bits such that the two encoders are operating on the same set of input bits, but different input sequences. Thus the first encoder receives the input bit .73,- and produces the output pair while the second encoder receives the input bit and produces the output pair The input bits are grouped into finite- length sequences whose length N equals the size of the interleaver. Since both the encoders are systematic and operate on the same set of input bits, it is only necessary to transmit the input bits once and the overall code has rate 1 / 3. In order to increase the overall rate of the code to 1 / 2, the two parity sequences {yl} and {342} can be punctured by alternately deleting y1 and y2. We will refer to a Turbo code whose l 699 constituent encoders have parity—check polynomials ha and hl and whose interleaver is of length N as an (h0,h1,N) Turbo code. For example, consider the Turbo encoder shown in Fig. 2, where each constituent encoder is a (2,1,2) encoder with parity—check polynomials ho(D) = 1 + D2 and h1(D) = D, where D is the delay operator [44]. For purposes of illustration, assume a pseudorandom interleaver of size N = 16 bits which generates a (1 + D2,D,16) Turbo code. The interleaver is realized as a 4 X 4 matrix. which is filled sequentially, row by row, with the input bits 10,-. Once the interleaver has been filled, the input sequence to the second encoder, 272, is obtained by reading the interleaver in a pseudorandom manner until each bit has been read once and only once. The pseudorandom nature of the interleaver in this example is represented by a permutation H16 2 {6, 14, 4, 7, 11, 8, 3, 5, 9, 13, 0, 2, 12, 1, 10, 15} which implies x6 = 0515,1"1 = $10, and so on. If the input sequence is ‘x = {x15-~:1:0} = {0,1,0,1,0,0,0,0,0,0,0,1,0,0,0,1} and the interleaver is represented by the permutation 1116, then the input sequence to the second encoder is 117' = H16(:1:) = {0,1,1,0,0,0,0,0,0,0,1,0,1,0,0,0}. The trellis diagrams for both constituent encoders with these inputs are shown in Fig. 3. The corresponding unpunctured parity sequences are ' y1 = {0,0,1,0,0,0,0,0,0,0,0,0,1,0,1,0} I and g2 = {1,1,0,0,0.0,0,0,0,0,0,1,0,0,0,0}. ‘ The resulting codeword has Hamming weight , d=w(x)+w(y1)+w(y2) =4+3+3= 10 without puncturing, where w(at) is the Hamming weight of the sequence x. If the code is punctured beginning with yé, then the resulting codeword has weight d : 4 + 3 4— 2 = 9. If, on the other hand, the puncturing begins with ya, then the punctured codeword has Hamming weight 4 + 1 = 5. Finding the free distance of a Turbo code is complicated by the fact that Turbo encoders are time-varying due to the interleaver. That is, if i’ :2 Dav, then 331 = Dyl, but 1’ 7A Dx’ (with high probability) and g2 96 Dyz. (Here, we only consider delays of a finite length sequence ac for which no ones are lost.) Continuing the example, if 5: : Dag, then a' = 1116(55) = {0,0, 0,0,0,0,0,1,0, 1,0,0,0, 1,0, 1} and g2 = {0,0,0,0,0,0,0,0,1,0,0,0,0,0,1,0}. Thus time-shifting the input bits results in codewords that differ in both the bit positions of the ones and overall Ham- ming weight. The variation in the weights of codewords 1700 State 1 1 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 42, NO. 6, NOVEMBER 1996 Encoder 1 10 01 00 xyl 10 01 00 _01 State 11 1000000000000000 10 01 10 00 Encoder 2 10 01 00 x'yloo .00 00 10 01 10 00 00 00 00 00 00 oo 10 ii 01 Fig. 3. Trellis diagrams for a codeword in the (1 + 1'_)2,D7 16) Turbo code. corresponding to time-shifted input sequences is magnified by puncturing. This simple example illustrates several important points concerning the structure of the codewords. First, because the pseudorandom interleaver permutes the input bits, the two input sequences 30 and x’ are almost always different, though of the same weight, and the two encoders will (with high probability) produce parity sequences of different weights. Second, it is easily seen that a codeword may consist of a number of distinct error events in each encoder, where an error event is a path in the trellis that diverges from the all- zero state and then remerges with the all—zero state within a finite number of branches. Note that since the constituent encoders are realized in systematic feedback form a nonzero tail of 1/ bits is required to return the encoder to the all-zero state. Thus since at least one nonzero bit is required to start an error event, all error events are associated with information sequences of weight 2 or greater [8]. ‘ Finally, with a pseudorandom interleaver it is highly un- likely that both encoders will be returned to the all—zero state at the end of the codeword, even when the last 1! bits of the input sequence as are used to force the first encoder back to the all—zero state. This is due to the fact the two encoders will with high probability end up in different states and thus require two different V bit tails. It is highly unlikely that the 11—bit tail for the first encoder will be interleaved by a pseudorandom interleaver to the correct tail for the second encoder. If neither encoder is forced back to the all—zerostat‘e, i.e., no tail is used, then the sequenCe consisting of N a 1 zeroes followed by a one is a valid input sequence a: to the first encoder. For some interleavers, this :2: will be permuted to itself and as’ will be the same sequence. In this case, with puncturing, the weight of this codeword and the free distance of the code will be at most two. Note that this codeword is caused by an information sequence of weight one. Thus forcing the first encoder to return to the all—zero state insures that every information sequence has at least weight two and eliminates the possibility of a weight-two codeword. For this reason, it is common to assume that the first encoder is forced to return to the all—zero state by appending a V—bit tail. ‘ The ambiguity of the final state of the second encoder has been shown by simulation to result in negligible performance degradation for large interleavers [[12], [30]. For these reasons, it will be assumed in the remainder of the paper that the first encoder is forced to return to the all—zero state and that the final state of the second encoder is unknown. Special interleaver structures that result in both encoders returning to the all—zero state are discussed in [31], [32], [34]—[36]. , A. Performance Bounds In order to make clear the distinction between Thrbo codes ‘ and convolutional codes, it is useful to consider these codes as block codes. To this end, the input sequences are restricted to length N, where N corresponds to the size of theiinterleaver in the Turbo encoder. With finite—length input sequences of length N , a (2, 1, V) convolutional code may be viewed as a , block code‘with 2N codewords of length 2(1/ + N ' The bit error rate (BER) performance of a convolutional code with maximum-likelihood (ML) decoding on an additive white Gaussian noise (AWGN) channel can be upper—bounded using a union bound technique by [12] 2N I ‘ ,- ’ 2RE PngEUNQC/d, N 5) (1) i=1 0 - , _ where w,- and d,- are the information, weight and total Ham- ming weight, respectively, of the ith codeword. Collecting codewords of the same total Hamming weight and defining the average information weight per codeword as “in d—Nd where Wd is the total information weight of all codewords of weight d and Nd is the total number, or multiplicity, of PEREZ et (11.: A DISTANCE SPECTRUM INTERPRETATION OF TURBO CODES codewords of weight d, yields 2(V+N) ” 2R B. s Nd“ Q d E" (2) dzdf N No where de-ee is the free distance of the code. (In this develop- ment, the multiplicity Nd includes codewords due to multiple error events for d 2 Zdfiee). For large enough N, if a convolutional code has N3 codewords of weight d caused by information sequences at whose first one occurs at time 0, then it also has N 3 codewords of weight d caused by the information sequences D27, N3 codewords of weight d caused by the information sequences D233, and so on. Thus as the length of the information sequences increases, we. have . Nd_ 0 Haw—Nd and Wd_W§é lirn 12),, = lim of] N —)oo N—>oo Nd — where W3 is the total information weight of all codewords with weight d which are caused by information sequences whose first one occurs at time 0. Thus the bound on the BER of a convolutional code with ML decoding becomes 2(V+N) - 2RE Hg 2 Ngwgo< d Nob) dzdfree 2(V+N) 2RE = 2 W362( d N0”) (3) dzdftee which is the standard union bound for ML decoding. For this reason, efforts to find good convolutional codes for use with ML decoders have focused on finding codes that maximize the free distance dfiee and minimize the number of free—distance paths Ngee for a given rate and constraint length. The performance of a Turbo code with maximum-likelihood decoding can also be bounded using the union bound of (2). However, in the Turbo encoder the pseudorandom interleaver maps the input sequence at to w’ and the input sequence But to a sequence at” that is different from Dz’ with high probability. Thus unlike convolutional codes,’ the input sequences :17 and D30 produce different codewords with different Hamming weights. For Turbo codes with pseudorandom interleavers, Ndwd is much less than N for low-weight codewords. This is due to the pseudorandom interleaver which, with high probability, maps low-weight parity sequences in the first constituent encoder to high-weight parity sequences in the second constituent encoder. Thus for low—weight codewords Ndflld 1 N < Where ‘ Nd —- 4 N ( ) is called the effective multiplicity of codewords of weight d. The effect of the interleaver size on the multiplicity is also studied in [7]—[9]. I701 B. Asymptotic Performance For moderate and high signal-to-noise ratios, it is well known that the free—distance term in the union bound on the bit error rate performance dominates the bound [44]. Thus for Turbo codes the asymptotic performance approaches N11361:} tee P1. z LIV—f— Q< dire. ——‘i> (5) where Nfree is the multiplicity of free—distance codewords and we, is the average weight of the information sequences causing free-distance codewords. The expression on the right— hand side of (5) and its associated graph is called the free— distance asymptote, Pfree, of a Turbo code. An algorithm for finding the free distance of Turbo codes is described in [28]. This algorithm was applied to a Turbo code with the same constituent encoders, puncturing pattern, and interleaver size N as in [1] and a particular pseudorandom interleaving pattern. The parity check polynomials for this ‘ code are ho = D4+D3+D2+D+1 and h1 = D4+1, or ho : 37 and h1 = 21 using octal notation. This (37, 21, 65536) code was found to have Nfiee = 3 paths with weight dfree = 6. Each of these paths was caused by an input sequence of weight 2 and thus of... = 2. Though this result was for a particular pseudorandom interleaver, it is true for most pseudorandom interleavers with N = 65536. This is consistent with the conclusions in [6] and [8] in which the performance of Turbo codes is averaged over all possible pseudorandom interleavers. For this particular Turbo code, the free distance asymptote is given by 3X2 Eb Pfree“6533_‘6Q( where the rate loss due to the addition of a 4-bit tail is ignored and N free 3 7T “ 65536 is the effective multiplicity. The free—distance asymptote is shown plotted in Fig. 4 along with simulation results for this code using the iterative decoding algorithm of [l] with 18 iterations. From Fig. 4, it can clearly be seen that the simula~ tion results do in fact approach the free—distance asymptote for moderate and high SNR’s. Since the slope of the asymptote is essentially determined by the free distance of the code, it can be concluded that the “error floor” observed with Turbo codes is due to the fact that they have a relatively small free distance and consequently a relatively flat free-distance asymptote. Further examination of (5) reveals that the manifestation of the “error floor” can be manipulated in two ways. First, increasing the length of the interleaver while preserving the free distance and the multiplicity will lower the asymptote without changing its slope by reducing the effective multiplic- ity. In this case, the performance curve of Turbo codes does not flatten out until higher SNR’s and lower BER’s are reached. Conversely, decreasing the interleaver size while maintaining the free distance and multiplicity results in the error floor being raised and the performance curve flattens at lower SNR’s and 1702 — o—e (37,21,65536) Turbo 7 G~~~~~O (2,1,14) MFD Convolutional Code a H Turbo Free Distance Asymptote (2,1,14) Free Distance Asyynptole Bit Error Rate rib/N, (dB) Fig. 4. Simulation results for a (37, 21, 65536) Turbo code and a (2, 1, 14) MFD convolutional code along wiht their free~distance asymptotes. 1 10' 3 'v‘ < r r l I E 3 - G—-O N=65536, 18 iterations . 10-2 __ EL—El N=16384, 18 iterations _: I E HN=1024, 18 iterations = -3 10 um“, Bit Error Rate ‘ H on I U‘n IWL IIW-IMI HI- 10 r 10" 10" . i . . l . . . 1 . . . i I . . . y i . . a . 0.0 0.5 1.0 ’ 1.5 2.0 2.5 Eb/NB (dB) Fig. 5. Simulation results for a (37 , 21, N) Turbo code with varying inter— leaver size N. I higher BER’s. This can be seen in the simulation results shown in Fig. 5 for the (37, 21,N) Turbo code with varying N. If the first constituent encoder is not forced to return to the all— zero state and the weight 2 codewords mentioned earlier are . ' allowed, then the error floor is raised to the extent that the code performs poorly even for large interleavers. Thus one cannot completely disregard free distance when constructing Turbo codes. If the size of the interleaver is fixed, then the “error ' floor” can be modified by increasing the free distance of the 7 code while preserving the multiplicity. This has the effect of changing the slope of the free—distance asymptote. That is, increasing the free distance increases the slope of the asymptote'and decreasing the free distance decreases the slope IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 42, NO. 6, NOVEMBER 1996 of the asymptote. It has been shown in [8], [24], [28], and [29] that for a fixed interleaver size, choosing the feedback polynomial to be a primitive polynomial results in an increased free distance and thus a steeper asymptote. An argument to support the use of primitive feedback polynomials in Turbo codes is presented in Section IV. ‘ C. Comparison to the (2,1,14) Code The role that the free distance and effective multiplicity play in determining the asymptotic performance of a Turbo code is further Clarified by examining the asymptotic performance of a convolutional code. The free—distance, asymptote of a convolutional code is given by the first term in therunion bound of (3). The maximum free distance (MFD) (2,1,14) code whose performance is shown in Fig. 4 has dfree = 18, Ngee = 18 and W0 = 137 [45]. Thus the free—distance asymptote free for this code is , E PM = 137Q( 18 which is also shown in Fig.4. As eXpected, the free—distance asymptote of the (2,1,14) code is much steeper than the free—distance asymptote of the Turbo code due'to the increased free distance. However, be cause the effective multiplicity of the free—distance codewords, given by (4), of the Turbo code is much smaller than the multiplicity of the (2,1,14) code, the two asymptotes do not cross until Eb/NO = 3.5 dB. At this Eb/NO, the BER of both codes is less than 10‘7, which is lower than the targeted BER of many practical systems. Thus even though the (2, 1,14) convolutional code is asymptotically better than the (37 ,21,65536) Turbo code, the Turbo code is better for the error rates at which many systems operate. D. A Turbo Code with a Rectangular Interleaver To emphasize the importance of using a pseudorandom interleaver with Turbo codes, we now consider a Turbo code with a rectangular interleaver. Turbo codes with rectangular interleavers have also been considered in [2], where the effect of the interleaver on the free distance of the code is discussed. The same constituent encoders and puncturing pattern as in [1] are used in conjunction'with a 120 X 120 rectangular interleaver. This rectangular interleaver is realized as a 120 x 120 matrix into which the information sequence a: is written row by row. The input sequence to the second encoder 117‘ is then Obtained by reading the matrix column by column. A 120 X 120 rectangular interleaver implies an interleaver size of N = 14400 and thus this is a (37, 21, 14400) Turbo code. Using the algorithm described in [28], this code was found to have a free distance of dfree : 12 with a multiplicity of Nfree = 28900. For this code, each of the free—distance paths is caused by an information sequence of weight 4, so wfree : 4., The free—distance asymptote for this code is thus given by ‘ 28900 x 4 Eb Pfree — ‘ 12 w) .. PEREZ et (11.: A DISTANCE SPECTRUM INTERPRETATION OF TURBO CODES rV—‘T‘fil—lr—Vfi 9—9 Simulations 0""0 Free Distance Asymptote Bit Error Rate H e 10 10'5 10“ ._4._Al_u..k-t_..4c.tt....L._4t__x_lg.JL 0.0 0.5 1.0 1.5 2.0 2.5 El/N, (dB) Fig. 6. Simulation results and the free-distance asymptote for the (37, 21, 14400) Turbo code with a 120x 120 rectangular interleaver. The free—distance asymptote is plotted in Fig. 6 along with simulation results using the iterative decoding algorithm of [l] with 18 iterations. This figure clearly shows that the free- distance asymptote accurately estimates the performance of the code for moderate and high Eb/No’s. This code achieves a bit error rate of 10“5 at an Eb/No of 2.7 dB and thus performs 2 dB worse than the (37, 21, 65536) Turbo code with a pseudorandom interleaver even though it has a much larger free distance. The relatively poor perfor- mance of the (37, 21, 14400) Turbo code with a rectangular interleaver is due to the large multiplicity of dfree paths. This results in an effective multiplicity of Nfree 28900 N N ‘ 14400 “‘2 which is much larger than the effective multiplicity of the (37, 21, 65536) Turbo code. We now show that the large mul- tiplicity is a direct consequence of the use of the rectangular interleaver and that, furthermore, increasing the size of the interleaver does not result in a significant reduction in the effective multiplicity of the free-distance codewords. The free-distance paths in the Turbo code with the rectan— gular interleaver are due to four basic information sequences of weight 4. These information sequences are depicted in Fig. 7 as they would appear in the rectangular interleaver. The “square” sequence in Fig. 7a) depicts the sequence a; = 110703070:1705947170707070717000 Where 0594 denotes a sequence of 594 consecutive zeroes and 0,,O represents a sequence of zeroes that continues to the end of the information sequence. In this case, the rectangular interleaver maps the sequence a: to itself and therefore m’ = ac. The sequence 2: results in a parity sequence y1 from the first constituent encoder which, after puncturing, has weight 4. Similarly, the input sequence ac’ = 1; results in a parity sequence y2 from the second constituent encoder which, after 1703 puncturing, also has weight 4. The weight of the codeword is then dfree = 4+4+4 = 12. By counting the number of distinct positions in which these “square” sequences can appear in the rectangular interleaver, we can find the multiplicity of the free- distance codewords. Since the “square” sequence in Fig. 7a) can appear in —5) x —5) = 13225 distinct positions in the rectangular interleaver, and in each case 37’ = :17 and a codeword of weight dfree = 12 results, this results in 13225 free—distance codewords. Note that for every Occurrence of the “square” sequence to result in a codeword of weight 12 the weight of both parity sequences must be invariant to which is punctured first. The “rectangular” sequences in Fig. 7b) and c) also result in weight 12 codewords. For these two sequences, the weight of one of the parity sequences is affected by whether or not it is punctured first and only every other position in which the “rectangular” sequences appear in the interleaver results in a codeword of weight dfiee = 12. Thus the sequences in Fig. 7b) and c) each result in 0.5(W— 10) x — 5) : 6325 free- distance codewords. For the “square” sequence in Fig. 7d), the weight of both parity sequences is affected by which is punctured first and only one out of four positions in which this “square” sequence appears in the interleaver results in a codeword of weight dfree = 12. Consequently, this “square” sequence results in 0.25(\/1V— 10) x (x/TV— — 10) = 3025 free— distance codewords. Summing the contributions of each type of sequence results in a total of thee 2 28900 codewords of weight dfree = 12. It is tempting to try to improve the performance of a Turbo code with a rectangular interleaver by increasing the size of the interleaver. However, all of the information sequences shown in Fig. 7 would still occur in a larger rectangular interleaver, so the free distance cannot be increased by increasing N. Also, since the number of free-distance codewords is on the order of N, increasing the size of the interleaver results in a corresponding increase in Nfree such that the effective multiplicity Nfree/N does not change significantly. Without the benefit of a reduced effective multiplicity, the free-distance asymptote, and thus the “error floor,” of Turbo codes with rectangular interleavers is not lowered enough for them to manifest the excellent performance of Turbo codes with pseu- dorandom interleavers for moderate BER’ s. Attempts to design interleavers for Turbo codes generally introduce structure to the interleaver and thus destroy the very randomness that results in such excellent performance at low SNR’s. III. THE DISTANCE SPECTRUM OF TURBO CODES In the previous section, it was shown that the “error floor” observed in the performance of Turbo codes is due to their relatively low free distance. It is now shown that the out— standing performance of Turbo codes at low SNR’s is a manifestation of the sparse distance spectrum that results when a pseudorandom interleaver is used in a parallel concatenation scheme. To illustrate this the distance spectrum of an “average” (37, 21, 65536) Turbo code is found and its relationship to the performance of the code is discussed. The distance spectrum of the “average” Turbo code is then compared to the distance 1704 33 ME 55 0 mm 99 1 53 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 42, NO. 6, NOVEMBER 1996 Ea ! 33353553 535333535 3 3 5 5 5 3 5 3 5 5 5 3 5 3 3 533335335 35333555555 333533335335 533355535335 Fig. 7. Portions of the interleaver and information sequences resulting in free—distance codewords for a (37,21,14400) Turbo code with a 120 X 120 rectangular interleaver. spectrum of the (2,1,14) code. An “average” Turbo code is one whose properties have been averaged over all possible pseudorandom interleavers as in [8]. By doing this, the analysis of Turbo cedes given in [8] and the algorithm for finding the distance spectrum given in [28] are simplified. Using the algorithm described in [28], an “average” (37, 21, 65536) Turbo code was found to have the following distance spectrum d Nd ’Wd 6 4.5 9 8 11 22 10 20.5 41 12 75 150 where Nd is the total number of codewords of weight d and Wd : Ndrfid is the total information weight of all codewords of weight d. The distance spectrum information for a particular distance d is referred to as a spectral line. This data can be used in conjunction with the bound of (2) to estimate the performance of the code. In addition, by plotting each term of (2) the contribution of each spectral line to the overall performance of the code can be estimated. The performance of a particular (37, 21, 65536) Turbo code is shown in Fig. 8 along with curves showing the contribution of each spectral line for an “average” Turbo code with the same interleaver length. This clearly shows that the contribution to the code’s BER by the higher distance spectral lines is less than the contribution of the free—distance term for E1, /N0’s greater than 0.5 dB. (The higher distance spectral lines become important, though, as capacity is approached.) Thus the free- distance asymptote dominates the performance of the code I not only for moderate and high Eb/Nos but also for low Eb/No’s. We characterize distance spectra for which this is true as sparse or spectrally thin; ' I A. Comparison to the (2,1,14) Code The ramifications of a sparse—distance spectrum are made evident by examining the distance spectrum of convolutional codes. The (2, 1, 14) convolutional code introduced in Section II has the following distance‘spectrum: ' ‘ d rtg .773 18 33 187 20 136 1034 22 835 7857 24 4787 53994 I 26 27941 361762 28 '162513 2374453 30 945570' 15452996 32 %%M4%%%% PEREZ et al.: A DISTANCE SPECTRUM INTERPRETATION OF TURBO CODES 10 .—r—Tr mm? ' G—OSimulation — 10" .— -----Edm=6 _ E <>----<>d=s 3 L A-----Ad=10 : 4 — ---~<ld=12 10 - a. .u] Bit Error Rate H a | H G. a Eb/No (dB) Fig. 8: Performance of a (377 21. 65536) Turbo code decomposed by spec— tral line. .-v--rvr-n.. HSimuhtions *m'iédmsm 0"”0 d=20 ,, Elm-El d=22 0--"<>d=24 AN-"A d=26 <l I> I'I—l—r—l'l—V1l—I'III—Tl --~<1d=2s --~+> d=30 1.1.1.141 .M : [KJLkJL—ILJ‘IJJJJ. Bit Error Rate 5 0.0 1.0 2.0 3.0 Eth,] (dB) Fig. 9. Performance of the (2.1,14) MFD convolutional code decomposed by spectral line. as reported in [45]. When comparing the distance spectrum of a convolutional code and a Turbo code, it is important to remember that for a convolutional code Nd z N X N2 for the low~weight codewords. Fig. 9 shows the performance of this code and the contribution of each spectral line from the bound of (3). In this case, the contribution of the higher distance spectral lines to the overall BER is greater. than the contribution of the, free-distance term for E1, /N0’s less than 2.7 dB, which corresponds to BER’s of less than 10*6! The large SNR required for the free-distance asymptote to dominate the performance of the (2, 1, 14) code is due to the rapid increase in the path multiplicity for increasing d. We characterize distance spectra for which this is true as spectrally dense. The 1705 dense distance spectrum of convolutional codes also accounts for the discrepancy between the real coding gain at a particular SNR and the asymptotic coding gain calculated using just the free distance [44]. Thus it can be concluded that the outstanding performance of Turbo codes at low signal-to—noise ratios is a result of the dominance of the free-distance asymptote, which in turn is a consequence of the sparse-distance spectrum of Turbo codes, as opposed to spectrally dense convolutional codes. Finally, the sparse-distance spectrum of Turbo codes is due to the structure of the codewords in a parallel concatenation and the use of pseudorandom interleaving. IV. SPECTRAL THINNING In this section, the observations made concerning the dis- tance spectrum and spectral thinning of Turbo codes is formal- ized from the point of View of random interleaving. Random interleaving was introduced in [6]—[9] to develop transfer func- tion bounds on the average performance of Turbo codes and to explore issues of code design. Here, random interleaving is used to explore the effect of the interleaver on the distance spectrum of the code. In order to simplify the notation and discussion, only nonpunctured Turbo codes are considered explicitly. The extension to punctured codes is straightforward and may be found in [28]. ' The fundamental idea of random interleaving is to consider the performance of a Turbo code averaged over all possible pseudorandom interleavers of a given length. For a given N, there are N! possible pseudorandom interleavers and, assuming a uniform distribution, each occurs with probability 1 / N !. Let a particular interleaver map an information sequence a: of weight w to an information sequence x’, also of weight 11). Then there are a total of w!(N — w)! interleavers in the ensemble of N! interleavers that perform this same mapping. Thus the probability that such a mapping occurs and hence that the codeword that results from the input sequences a: and 35’ occurs, is w!(N —w)! 1 ‘65) Following [8], define the input redundancy weight enumer- ating function (IRWEF) of a systematic code as A(W. 2) = Z Z AWWWZZ <6) w I where Aw1z is the number of codewords of weight d = w + 2 generated by input sequences of weight w and parity sequences of weight 2. The goal is now to develop a relationship between the codewords in the constituent encoders and AM for the Turbo code and to see how that relationship changes with the size of the interleaver. Recall from Section II that a Turbo codeword is essentially the combination of a codeword from the first constituent encoder plus a codeword from the second constituent encoder. A codeword of weight d1 = w + 21 from the first constituent encoder caused by an information sequence ac of weight w 1706 is composed of m error events of total length ll. To avoid difficulties in counting codewords when the second constituent encoder is left unterrninated, we consider different orderings of the same error events as distinct cases. The ordered set of n1 error events in the first encoder is denoted by 51. The information sequence a: that results in the set 31 is mapped by a particular interleaver to the information sequence m’, also of weight in, which is then encoded by the second constituent encoder. This results in a codeword of weight d2 = w + 22, with the ordered set 5'2 consisting of n2 error events of total length l2. For example, Fig. 3 depicts a codeword of weight d1 = 4 + 3 = 7 in the first constituent encoder caused by an information sequence of weight w = 4 and composed of m = 2 error events of total length ll 2 5 + 3 = 8. For the interleaver described in Section II, a: is mapped to an :13’ that results in a codeword of weight d2 = 4 + 3 : 7 in the second constituent encoder consisting of n2 2 2 error events of total length l2 : 3 + 3 = 6. Thus this 51 and 5'2 result in a codeword of weight 10 and a single contribution to A45 of the Turbo code for this particular interleaver. Averaged over the ensemble of interleavers of length N, a set 51 with information weight w and parity weight 21 and a set 52 with information weight w and parity weight 22 will contribute a fraction N w to the enumerating function coefficients AW,“2 of an “av— erage” Turbo code. Because the sequence of zeros connecting any two distinct error events has no effect on the weight of the information sequence'or the parity sequence, there are <N—l1+n1> (8) 711 ways that the ordered set 51 of n1 error events can be arranged such that their contribution to AwyzlflZ is not changed. This is simply the number ofways in which n1 distinct error events can be arranged in a sequence of length N while maintaining the order in which they appear. Similarly, if the codeword in the second constituent encoder ends in the all-zero state, then the ordered set Sg will make (N—l2+n2) (9) 712 contributions to Awning. However, because the second en— coder is not guaranteed to return to the all-zero state, it is possible that the last of the fig error events is not actually an error event, but instead ends in a nonzero state. In this case, the last error event cannot be moved and the set 82 makes (N—l2+(n2—1)> (n2 — 1) “0) contributions to Aw721+22. IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 42, NO: 6,'NOVEMBER 1996 The contribution to the distance spectrum of a Turbo code due to any pair of ordered sets 81 and $2, averaged over the ensemble of pseudorandom interleavers, can now be computed using (7)—(lO). If the codeword in the second constituent en— coder happens to end in the all-zero state, then the contribution of an 31 and 5’2 to Awfizlirz2 is given by N—ll—l—nl N4l2+7l2 ( m >< n2 ) I N . .. . . (w) If the codeword in the second constituent encoder does not end in the all—zero state, then the contribution to Aw7zl+Z2 is given by N—l1+n1><N—12+(n2-1)) “1 (712—1) _ (if) The expressions in (11) and (12) can now be used to explore the effect of changing the interleaver size on the distance , spectrum of an “average” Turbo code. Since we are primarily concerned r with low—weight codewords in the distance spectrum, we assume that N > n1,n2,l1, and £2. If this is not true, then 51 and 52 either contain a, large number of short error events or a few long error events. In both cases, it is very unlikely that the result is a codeword of low weight. With this assumption, (11) can be approximated by ' (11) (12) w! Nn1+ng~w nilngl ‘ where, without loss of generality, it is assumed that m 2 722. Since each error event is caused by an information sequence of weight at least two, to Z 2711. The behavior of (13) for increasing N can be broken down into three casesr’ 1) 711 > 71;: r The exponent of N is strictly negative’and the contribu— tion to Aw,le2 decreases as N increases. 2) n1 =n2 and w>2n1: The exponent of N is strictly negative and the contribu— tion to vale2 decreases as N increases. 3) n1 : ng and w : 2721: The exponent of N is exactly zero and the contribution to Aw,zl+22 converges to a finite value as N increases. For N >> n1, n2, [1, and Z2, (12) can be approximated by w! N711 +112 —w‘—1 n1!(n2 — 1)! (14) where w 2 2m. However, since the tail in the second encoder may be caused by an information sequence of weight 1, we also have in Z 2712 — 1. The behavior of (14).for increasing N can be broken down into three cases: 1) 711 > m: _ Since 112 2 2711, the exponent of N is strictly nega— tive and the contribution to Awyzl+22 decreases as N increases. PEREZ et al.: A DISTANCE SPECTRUM INTERPRETATION OF TURBO CODES 2) n1 2 n2: Again, since u; 2 2m, the exponent of N is strictly negative and the contribution to Aw7zl+Z2 decreases as N increases. 3) m < m: Since 11) 2 2m — 1, the exponent of N is strictly negative and the contribution to Aw,le2 decreases as N increases. The following lemma can now be stated. A similar result was proven in [9]. 1 Lemma 1: Given a Turbo code based on two systematic feedback encoders and a pseudorandom interleaver of length N in which the first encoder is assumed to be forced back to the all-zero state, the contribution of two ordered sets of error events 81 and $2 with the same information weight to the distance spectrum of the Turbo code, averaged over all pseudorandom interleavers of length N, converges to a nonzero constant as N —> 00, if and only if 1) 82 leaves the second encoder in the all-zero state; 2) 81 and 8'2 contain the same number of error events; 3) each error event in $1 and $2 is caused by a weight two information sequence. In all other cases, the contribution goes to zero as N —-> 00. For each term Awyz in the input redundancy weight enu- merating function of a Turbo code there is a finite number of pairs of sets 5'1 and 52 that contribute to it. If either set contains a long error event, then it is possible that that pair will be excluded for small interleavers. As the interleaver size increases, eventually all pairs of sets will be allowed and any further increase in N will not result in additional pairs of sets contributing to AM. Also, as N —> 00, AM will be determined by pairs of 31 and 82 that satisfy the three conditions of Lemma 1, and thus each Awfiz will converge to a finite value. Since each spectral line is a finite sum of A,” terms, each spectral line converges to a finite value as the interleaver size increases. The convergence of each spectral line to a finite value as the size of the interleaver increases results in spectral thinning. That is, for small interleavers there may be pairs of sets 51 and 52 that do not satisfy the conditions of Lemma 1, but which contribute to the multiplicity of a low-weight spectral line. As the size of the interleaver increases the number of these aberrational sets decreases until the spectral line reaches its final value as determined by Lemma 1. This process of spectral thinning is represented graphically in Fig. 10, which depicts the thinning of low-weight codewords as the size of the interleaver increases for hypothetical distance spectra. It is this thinning of the distance spectrum that enables the free—distance asymptote of a Turbo code to dominate the performance for low SNR and thus to achieve near—capacity performance. A. Primitive Polynomials and Free Distance We now consider the ramifications of Lemma 1 with respect to the free-distance codewords of a Turbo code. That is, what does Lemma 1 imply about the information sequences that generate the free-distance codewords in an “average” Turbo code? ‘ 1707 N Infinite Spectral Thinning Multiplicity, Nd Hamming Distance, d Fig. 10. Graphical representation of spectral thinning for increasing inter- leaver size. For an “average” Turbo code, Lemma 1 states that as the size of the interleaver increases each spectral line is the result of contributions only from pairs of ordered sets of error events in which each error event is caused by a weight— two information sequence. It is reasonable to expect that the free-distance spectral line will be among the first spectral lines to converge to its final value. Thus for reasonably large interleavers, the free distance will be determined by the sets. 5'1 and 52 satisfying the conditions of Lemma 1. Let 31 and 52 be any pair of error events caused by a weight—two information sequence that results in a minimum—weight parity sequence in the first and second constituent encoders, respectively. Note that there may be more than one such pair of minimum-weight error events for the constituent encoders. A free—distance codeword in an “average” Turbo code must be the result of sets 31 and 82 that consist of only those minimum»weight error events 31 and 32, respectively. Furthermore, since each additional error event in either 5'1 and 82 adds weight to the codeword, 51 and 82 must each contain only one minimum-weight error event. (If each error event does not add weight, then a weight—two information sequence exists that generates a zero weight parity sequence and the free distance of the code would be 2.) Thus the free-distance codewords of an “average” Turbo code are caused by weight— two information sequences, provided the interleaver is large enough. Therefore we have the following Lemma. Lemma 2: For an “average” Turbo code, as the size of the interleaver N approaches 00: ' i 1) the free distance codewords are caused by information sequences of weight 2; 2) the free distance of an “average” Turbo code is maxi— mized by choosing constituent encoders that have the largest output weight for weight-two information se- quences. Based on this lemma, we now present an intuitive argument showing that choosing the feedback polynomial h0(D) in a (2, 1, V) systematic feedback encoder to be a primitive polyno- mial maximizes the output weight for weight-two information sequences. It follows that the free distance of an “average” Turbo code is maximized by using a primitive polynomial 1708 as the feedback polynomial in the constituent encoders. This result was independently derived in [9] using the transfer function‘of an “average” Turbo code. The generator matrix of a (2, 1,1/) systematic feedback encoder is given by ' 2:831 an), 2 [1 where hl (D) and h0(D) are referred to as the feedforward and feedback polynomials, respectively, and h0(D) is of degree 1/. Since only information sequences of weight 2 are being considered, the systematic output contributes weight 2 to the overall codeword weight for all the encoders being considered. Therefore, only the weight contributed by the parity sequence, that is, h1(D) MD) = ma?) MD), needs to be maximized. Furthermore, since we are concerned only with the choice of h0(D), MO?) is assumed to be a polynomial such that h0(D) and hl (D) are relatively prime. (There is empirical evidence that the choice of both polyno— mials can affect the performance of the code [28], [30], but we will not address that issue in this paper.) Let 1 + BK, for some finite K , be the shortest input sequence of weight 2 that generates a finite~length codeword. The resultant parity sequence is hiiD) _ hl(D) l10(1)) T hog?) K . K 711(1)) y(D) (1+D) .D MUD). Since y(D) is of finite length, h1(D)/h0(D) must be periodic with period K. Increasing the period K increases the length of the shortest weight 2input sequence that generates a finite- length codeword and therefore increases the length of that codeword. Intuitively, one would expect that increasing its length would result in the codeword gaining weight. That is, on average, half of the added bits would be ones. A strictly proper rational function of two polynomials, like h1(D)/h0(D), is periodic with period K g 2" — 1. The period is maximized, that is, K = 2" — 1, when h0(D) is a primitive polynomial. Since the free distance of an “average” Turbo code is determined by information» sequences of weight 2, for sufficiently large interleavers the free distance will be maximized by maximizing K. Therefore, choosing h0(D)_ to be a primitive polynomial will result in a larger free, distance for an “average” Turbo code. To test this, we compare a (37 , 21,400) Turbo code which has K = 5 and free distance dfree' : 6 to a (23,35,400) Turbo code. Both codes are punctured as in [l]. The feedback polynomial ho : 23 in the second Turbo code is a primitive polynomial of degree V =‘ 4 and thus 1/h0 has a period of K = 2" — 1 = 15. The free distance of this Turbo code was found to be dfree = 10‘ [12], [28]. Fig. 11 shoWs simulation results for these two codes using the iterative decoding algorithm of [l] with 18 iterations. As expected, IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 42,] NO. 6, NOVEMBER 1996 10' E - . w r 'u - » - x - - -<—‘ i e—e_(37,21,400) ‘ i - ~--“ (23,35,400) Primitive 10-1 ' ‘E a 10‘3 — '~ N I a: E In! 5 i5 104 g 10-5 3‘ _l E 10“ .1“ - . 4* I r r r - - - . ._x 1 . . r r . . . . 0.0 10 « 2.0 3.0 rah/man). Fig. 11. Simulation results for a (37,21,400) Turbo code and 21 (23,35,400) primitive Turbo code. the second Turbo code performs better at moderate and high SNR’s because its free distance'asymptote is steeper due to the increased free distance. V. CONCLUSIONS . The excellent performance of Turbo codes may be explained in terms of the distance Spectrum of the code. The “error floor” observed in Simulations of Turbo codes is a manifestation of the free—distance asymptote. Since Turbo codes haverelatively low free distances, the free—distance asymptote has a shallow slope, and thus the performance curves flatten out at moderate to high SNR’s. The “error floor” may be lowered by increasing the size of the interleaver for a fixed free distance, that is, by reducing the effective multiplicity of the code. Altemately, for fixed interleaver lengths, the performance may be‘improved for moderate and high SNR’s by increasing the free distance. Choosing primitive polynomials as the feedback polynomials in the constituent encoders usually results in an increased free distance. _ , The exceptional performance of Turbo codes at low SNR’s is due to the sparse distance spectrum and the resultant ability of the code to follow the free-distance asymptote at -‘ moderate to low SNR’S. The use of systematic feedback encoders and pseudorandom interleavers results in spectral thinning, in which information sequences which generate low— ~ weight parity sequences from the first constituent encoder are interleaved with high probability to information sequences that generate high—weight parity sequences in the second constituent encoder. Spectral thinning is enhanced by increas- ing interleaver lengths. For very large interleavers, spectral thinning results in a sparse distance spectrum in which the first several spectral lines are determined Solely by input sequences of weight two. Thus spectral thinning results in few low—weight codewords and a large number of codewords of “average”. weight. This is very similar to the type of distance spectrum achieved by “random-like” codes [4]. PEREZ et al.: A DISTANCE SPECTRUM INTERPRETATION OF TURBO CODES In a more philosophical light, Turbo codes remind us that information-theoretical arguments imply that long block lengths, but not necessarily large free distances, are required to achieve capacity at moderate ‘BER’s. Thus like convolutional codes, Turbo codes are a class of codes that achieve long block lengths, but without the corresponding increased density of the distance spectrum common to convolutional codes, and for which a practical, albeit nontrivial, decoding algorithm exists. In addition, Turbo codes are time-varying due to the pseudorandom interleaver, and the time—varying structure is essential in achieving the thin distance spectrum that results in near—capacity performance at moderate BER’s. This suggests that some effort should be made to find other classes of time-varying codes, and decoding algorithms, that have thin distance spectra, rather than just large free distances. Finally, since, in fact, long block lengths are required to achieve near- capacity performance at moderate BER’s, only modest coding gains will be achievable in systems that use relatively short block lengths. REFERENCES [l] C. Berrou, A. Glavieux, and P. Thitimajshima, “Near Shannon limit error-correcting coding and decoding: Turbo—codes,” in Proc. 1993 IEEE Int. Conf. on Communication (Geneva, Switzerland, 1993), pp. 1064—1070. [2] C. Berrou and A. Glavieux, “Turbo-codes: General principles and appli— cations,” in Proc 6th Tirrenia Int. Workshop on Digital Communications (Tirrenia, Italy, Sept. 1993), pp. 215—226. [3] G. Battail, C. Berrou, and A. Glavieux, “Pseudo—random recursive convolutional coding for near—capacity performance,” in Proc. Commu— nication Theory Mini-Conf, Globecom ’93 (Houston, TX, Dec. 1993), pp, 23-27. [4] G. Battail, “On random—like codes,” preprint. [5] , “Pseudo—random Turbo—codes,” in Proc. Int. Symp. on Signals, Systems and Electronics (San Francisco, CA, Oct. 25—27, 1995), pp. 419—422. I [6] 'S. Benedetto and G. Montorsi, “Average performance of parallel con— catenated block codes,” Electron. Lett., vol. 31, no. 3, pp. 156—158, Feb. 2, 1995. , “Performance evaluation of TURBO-codes,” Electron. Lett., vol. 31, no. 3, pp. 163—165, Feb. 2, 1995. , “Unveiling turbo codes: some results on parallel concatenated coding schemes,” IEEE Trans. Inform. Theory, vol. 42, no. 2, pp. 409—428, Mar. 1996. [9] , “Design of parallel concatenated convolutional codes,” IEEE Trans. Commun., vol. 44, no. 5, pp. 591—600, May 1996. [10] J. D. Andersen, “The TURBO coding scheme,” Rep. IT—146, Tech. Univ. of Denmark, Lyngby, June 1994. [11] J. D. Andersen, “TURBO coding for deep space applications,” in Proc 1995 IEEE Int. Symp. on Information Theory (Whistler, BC, Canada, Sept. 1995), p. 36. [12] P. Robertson, “Illuminating the structure of parallel concatenated re- cursive systematic (TURBO) codes,” in Proc. GLOBECOM ’94 (San Francisco, CA, Nov. 1994), vol. 3, pp. 1298—1303. [13] J. Hagenauer and L. Papke, “Decoding ‘Turbo’—codes with the soft Output Viterbi algorithm,” in Proc. 1994 IEEE Int.. Symp. on Information * Theory (Trondheim, Norway, June 1994), p. 164. [14] J. Hagenauer, P. Robertson, and L. Papke, “Iterative (TURBO) decoding of systematic convolutional codes with MAP and SOVA algorithms,” in Proc. ITG Conf on “Source and Channel Coding” (Frankfurt, Germany, Oct. 1994), pp. 1—9. [15] J. Hagenauer, E. Offer, and L. Papke, “Iterative decoding of binary block and convolutional codes,” IEEE Trans. Inform. Theory, vol. 42, no. 2, pp. 429—445, Mar. 1996. [16] Y. V. Svirid, “Weight distributions and bounds for turbo codes,” European Trans. Telecommun, vol. 6, no. 5, pp. 543—556, Sept—Oct. 1995. [17] Y. V. Svirid and S. Riedel, “Threshold decoding of turbo codes,” in Proc. 1995 IEEE Int. Symp. on Information Theory (Whistler, BC, Canada, Sept. 1995), p. 39. [7] [8] 1709 [18] D. Divsalar and F. Pollara, “Low-rate turbo codes for deep—space communications,” in Proc. 1995 IEEE Int. Symp. on Information Theory (Whistler, BC, Canada, Sept. 1995), p. 35. [19] D. Divsalar and F. Pollara, “Turbo codes for PCS applications,” in Proc. 1995 IEEE Int. Conf. on Communications (Seattle, WA, June 1995). , “Turbo codes for deep~space communications,” TDA Progr. Rep. 42—120, Jet Propulsion Lab., Pasadena, CA, pp. 29-39, Feb. 15, 1995. [20] [21] , “Multiple turbo codes for deep—space communications,” TDA Progr. Rep. 42—121, Jet Propulsion Lab., Pasadena, CA, pp. 66~77, May 15, 1995. [22] D. Divsalar, S. Dolinar, F. Pollara, and R. J. McEliece, “Transfer function bounds on the performance of turbo codes,” TDA Progr. Rep. 42—122, Jet Propulsion Lab., Pasadena, CA, pp. 44—55, Aug. 15, 1995. [23] S. Dolinar and D. Divsalar, “Weight distributions for turbo codes using » random and nonrandom permutations,” TDA Progr. Rep. 42—122, Jet Propulsion Lab., Pasadena, CA, pp. 56—65, Aug. 15, 1995. [24] D. Divsalar and F. Pollara, “On the design of turbo codes,” TDA Progr. Rep. 42—123, Jet Propulsion Lab., Pasadena, CA, pp. 99—121, Nov. 15, 1995. [25] J .—F. Cheng and R. J. McEliece, “Unit—memory Hamming turbo codes,” in Proc. 1995 IEEE Int. Symp. on Information Theory (Whistler, BC, Canada, Sept. 1995), p. 33. . . [26] R. J. McEliece, E. R. Rodemich, and J.-F. Cheng, “The turbo decision algorithm,” in Proc. 33rd Annu. Allerton Conf. on Communication, Control and Computing (Monticello, IL, Oct. 1995), p. 366. V [27] D. Divsalar and R. J. McEliece, “The effective free distance of turbo codes,” Electron. Lett., submitted for publication. [28] J. Seghers, “On the free distance of TURBO codes and related product codes,” Final Rep., Diploma Project SS 1995, no. 6613, Swiss Federal Institute of Technology, Zurich, Switzerland, Aug. 1995. [29] J. Seghers, L. C. Perez, and D. J. Costello, Jr., “0n selecting code generators for turbo codes,” in Proc. 33rd Annual Allerton Conf. on Communication, Control and Computing (Monticello, IL,'OCt. 1995), pp. 357—361. [30] D. Arnold and G. Meyerhans, “The realization of the the turbo-coding system,” Semester Project Rep, Swiss Federal Institute of Technology, Zurich, Switzerland, July 1995. [31] A. S. Barbulescu and S. S. Pietrobon, “Interleaver design for turbo codes,” Electron. Lett., ,vol. 30, no. 25,,p. 2107, Dec. 8, 1994. , “Terminating the trellis of turbo-codes in the same state,” Electron. Lett., vol. 31, no. 1, pp. 22—23, Jan. 5, 1995. , “Rate compatible turbo codes,” Electron. Lett., vol. 31, no. 7, pp. 535—536, Mar. 30, 1995. [34] _, “Interleaver design for three dimensional turbo codes,” in Proc. 1995 IEEE Int. Symp. on Information Theory (Whistler, BC, Canada, Sept. 1995), pg. 37. [35] M. C. Reed and S. S. Pietrobon, “Turbo—code termination for short frames,” Electron. Lett., submitted for publication. [36] O. Joerssen and H. Meyr, “Terminating the trellis of turbo-codes,” Electron. Lett., vol. 30, no. 16, pp. 1285—1286, Aug. 4, 1994. [37] P. Jung and M. Nthan, “Performance evaluation of turbo codes for short frame transmission systems,” Electron. Lett., vol. 30, no. 2, pp. 111—113, Jan. 20, 1994. , “Dependence of the error performance of turbo—codes on the interleaver structure in short frame transmission systems,” Electron. Lett., vol. 30, no. 4, pp. 285—288, Feb. 17, 1994. [39] P. Jung, “Novel low complexity decoder for turbo—codes,” Electron. Lett., vol. 31, no. 2, pp. 86—87, Jan. 19, 1995. ' [40] R. Podemski, W. Holubowicz, C. Berrou, and A. Glavieux, “Distance spectrum of the turbo-codes,” in Proc. 1995 IEEE Int. Symp. on Infor- mation Theory (Whistler, BC, Canada, Sept. 1995), p. 34. [41] R. Podemski, W. Holubowicz, C. Berrou, and G. Battail, “Hamming distance spectra of turbo—codes,” Ann. Te’le’commun., vol. 50, no. 9-10, pp. 790—797, Sept—Oct. 1995. [42] G. Caire, G. Taricco, and E. Biglieri, “On the convergence of the iterated decoding algorithm,” in Proc. 1995 IEEE Int. Symp. on Information Theory (Whistler, BC, Canada, Sept. 1995), p. 472. [43] N. Wiberg, H.—A. Loeliger, and R. Kotter, “Codes and iterative decoding on general graphs,” European Trans. Telecommun, vol. 6, no. 5, pp. 513—525, Sept—Oct. 1995. [44] S. Lin and D. J. Costello, Jr., Error Control Coding: Fundamentals and Applications. Englewood Cliffs, NJ ; Prentice—Hall, 1983. [45] M. Cedervall and R. Johannesson, “A fast algorithm for computing distance spectrum of convolutional codes,” IEEE Trans. Inform. Theory, vol. 35, no. 6, pp. 1146—1159, Nov. 1989. [321 [33] [38] ...
View Full Document

Page1 / 12

TurboDistanceSpectrum_Costello - 1698 IEEE TRANSACTIONS ON...

This preview shows document pages 1 - 12. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online