sklar_primer - ABSTRACT The goal of this article is to...

Info iconThis preview shows pages 1–9. Sign up to view the full content.

View Full Document Right Arrow Icon
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 2
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 4
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 6
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 8
Background image of page 9
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ABSTRACT The goal of this article is to describe the main ideas behind the new class of codes called turbo codes, whose performance in terms of bit error probability has been shown to be very close to the Shannon limit. In this article, the mathematical measures of a posteriori probability and likelihood are reviewed, and the benefits of turbo codes are explained in this context. Since the algorithms needed to implement the decoders have been well documented by others, they are only referenced here, not described in detail. A numerical example, using a simple concatenated coding scheme, provides a vehicle for illustrating how error performance can be improved when soft outputs from the decoders are used in an iterative decoding process. A Primer on Turbo Code Concepts Bernard Sklar, Communications Engineering Services oncatenated coding schemes were first proposed by Forney [1] for achieving large coding gains by com- bining two or more relatively simple component or building— block codes. The resulting codes had the error correction capability of much longer codes, and were endowed with a structure that permitted relatively easy to moderately complex decoding. A serial concatenation of codes is most often used for power—limited systems such as deep space probes. The most popular of these schemes consists of a Reed—Solomon outer code (applied first, removed last) followed by a convolu- tional inner code (applied last, removed first) A turbo code can be thought of as a refinement of the concatenated encoding structure and an iterative algorithm for decoding the associated code sequence. Turbo codes were introduced by Berrou, Glavieux, and Thitimajshima [3, 4]. For a bit error probability of 10—5 and code rate = 1/2, the authors report a required Eb/NO of 0.7 dB. The codes are constructed by applying two or more com— ponent codes to different interleaved versions of the same information sequence. For any single traditional code, the final step at the decoder yields hard—decision decoded bits (or, more generally, decoded symbols). In order for a concatenat- ed scheme such as a turbo code to work properly, the decod— ing algorithm should not limit itself to passing hard decisions among the decoders. To best exploit the information learned from each decoder, the decoding algorithm must effect an exchange of soft rather than hard decisions. For a system with two component codes, the concept behind turbo decoding is to pass soft decisions from the output of one decoder to the input of the other, and to iterate this process several times to produce better decisions. ' ‘ A REVIEW OF LIKELIHOODS The mathematical foundations of hypothesis testing rests on Bayes’ theorem, which is derived from the relationship between the conditional and joint probability of eventsA and B, as follows: P(A lB) P03) = H3 lA) P(A) = P(Aa) (1) A statement of the theorem yields the a posteriori probability (APP), denoted P(A lB), P(B|A)P(A) (2) P(B) which allows us to infer the APP of an eventA conditioned on B, from the conditional probability, P(B IA), and the a priori probabilities, P(A) and P(B). For communications engineering applications having additive white Gaussian noise (AWGN) in the channel, the most useful form of Bayes’ theorem expresses the APP in terms of a continuousvvalued random variable x in the fol— lowing form: P(AlB) = p(x|d : i)P(d = i) P(d-ilx)— pm i-1,...,M (3) and M , plx) = éplxld = i)P(d = 2') (4) where d = i represents data d belonging to the ith signal class from a set of M classes, and p(x|d = i) represents the proba- bility density function (pdf) of a received continuous—valued data-plus—noise signal, x, conditioned on the signal class at = i. p(x) is the pdf of the received signalx over the entire space of signal classes. In Eq. 3, for a particular received signal, p(x) is a scaling factor since it has the same value for each class. Lower case p is used to designate the pdf of a continuous—val- ued signal, and upper case P is used to designate probability (a priori and APP). Equation 3 can be thought of as the result of an experiment involving a received signal and some statisti— cal knowledge of the signal classes to which the signal may belong. The probability of occurrence of the ith signal Class P(d = 1'), before the experiment, is the a priori probability. As a result of examining a particular received signal, we can com- pute the APP, P(d = ilx), which can be thought of as a “refinement” of our prior knowledge. THE TWO-SIGNAL CLASS CASE Let the binary logical elements 1 and 0 be represented electronically by voltages +1 and — 1, respectively. This pair of transmitted voltages is assigned to the variable d, which may now take on the values d = +1 and d = —1. Let 94 0163—6804/97/$10.00 © 1997 IEEE IEEE Communications Magazine ° December 1997 the binary 0 (or the voltage value —1) be the null element under addition. For an AWGN channel, Fig. 1 shows the conditional pdfs referred to as likelihood functions. The rightmost function p(x|d +1) shows the pdf of the random variable x given the condition that d = +1 was transmitted. The leftmost function p(x|d = —1) illustrates a sim- ilar pdf given that d = — 1 was transmitted. The abscissa represents the full range of possible values of it received at the detector. In Fig. 1 is shown one such arbitrary predetection value xk. A line sub- tended from xk intercepts the two likelihood func— tions yielding two likelihood values XI and lug. A popular hard decision rule, known as maximum likelihood, is to choose the symbol d = +1 or d = —1 associated with the larger of the two intercept values, M or M . This is tanta- mount to deciding dk = +1 if xk falls on the right side of the decision line labeled 70, otherwise deciding dk = —1. A. similar decision rule, known as maximum aposteriori (MAP) or minimum-error rule, takes into account the a priori probabilities, as seen below in Eq. 6. The MAP rule is expressed in terms of APPS as follows: P(d = +1|x): P(d = —1|x) H2 (5) Equation 5 states that one should choose the hypothesis H1 (d : +1) if the APP P(d = +1 |x) is greater than the APP P(d = —1|x). Otherwise, choose hypothesis H2(d = —1). Using the Bayes’ theorem in Eq. 3, the APPS in Eq. 5 can be replaced by their equivalent expressions, yielding: H1 p(x|d = +1)P(d = +1): p(x|d = —1)P(d , H0 -1) (6) Equation 6 is generally expressed in terms of a ratio, called the likelihood ratio test, as follows: Plxld = +1) :11 PM = ~1) poh=4)h,Pw=+u p(x|d : +1) P(d = +1) :11 p(x|d = —1) P(d = —1) 1 712 (7) ‘ Figu-reTliikelihoodjitnctiorls. - Likelihood of d = —l P(x|d = ~1) Likelihood of d = +1 p(x|d = +1) Y0 LOG-LIKELIHOOD RATIO By taking the logarithm of the likelihood ratio in Eq. 7, we obtain a useful metric called the log-likelihood ratio (LLR). It is the real number representing a soft decision out of a detector, designated L(d |x), as follows: (8) _ P(d = +llx) p(xld = +1)P(d = +1) L(dlx)_1°g[1=(d = —l|x)] ogl p(x|d = —1)P(d = -1) weddnhed“dmee> m L(d Ix) = L(xld) + L(d) V (10) where L(x|d) is the LLR of the channel measurements ofx under the alternate conditions that d = +1 or d = —1 may have been transmitted, and L(d) is the a priori LLR of the data bit d. To simplify the notation, we represent Eq. 10 as follows: L’(g) = LAX) + L(d) (11) where the notation Lc(x) emphasizes that this LLR term is the result of a channel measurement made at the detector. Equa- tions 3 through 11 were developed with only a data detector in mind. Next, the introduction of a decoder will typically yield decision—making benefits. For a systematic code, it can be shown [3] that the LLR (soft output) L(d) out of the decoder is equal to W) = L'(c?) + Ltd) (12) where L’(d ) is the LLR of a data bit out of the detector (input to the decoder), and L80? ), called the extrinsic LLR, repre- sents extra knowledge that is gleaned from the decoding pro- cess. The output sequence of a systematic decoder is made up of values representing data and parity. Equation 12 partitions the decoder LLR into the data portion represented Feedback for the next iteration L(d) _ _ _>. _ _ _ _ _ _I Leld) Extrinsic values out soft—out , Detector a post rori decoder LLR value L'( ) = Lc(x) + L(d) Ltd) A posteriori values LC(X) Channel values In by the detector measurement, and the extrinsic por- tion represented by the decoder contribution due to parity. From Eqs. 11 and 12, we write Let) = Lax) + L(d) + Lee?) (13) The soft decision L(d) is a real number that pro— vides a hard decision as well as the reliability of that decision. The sign of L(tt) denotes the hard deci- sion; that is, for positive values of L(d ) decide +1, for negative values decide —1. The magnitude of L(d ) denotes the reliability of that decision. PRINCIPLES .OF ITERATlVE (TURBO) DECODING For the first decoding iteration of the soft input/soft output decoder in Fig. 2, one gener- — Figure 2. Softinput/s’ofi outpitt decoder (for a systematid code). ally assumes the binary data to be equally likely, IEEE Communications Magazine 0 December 1997 95 [(2 n —k2 columns co umns k1, rows d p,, Leh I __ -._ _ _. l m stat. l —__ Ext! i nsic—a vertical I Figure "Two-dimensionalproduct code. yielding an initial a priori LLR value 0fL(d) =0 for the third term in Eq. 9. The channel predetection LLR value, Lc(x), is measured by forming the logarithm of the ratio of 901 and 702, seen in Fig. 1A and appearing as the second term of Eq. 9. The output L(d ) of the Fig. 2 decoder is made up of the LLR from the detector, L13), and the extrinsic LLR output. L461). representing knowledge gleaned from the decoding process. As illustrated in Fig. 2, for iterative decod— ing the extrinsic likelihood is fed back to the decoder input, to serve as a refinement of the a priori value for‘the next iteration. ' .7 Consider the two—dimensional code (product code) depict- ed in Fig. 3. The configuration can be described as a data array made up of k1 rows and k2 columns. Each of the k1 rows contains a code vector made up of k2 data bits and n2 — k2 parity bits. Similarly, each of the k2 columns contains a code vector made up of k1 data 'bits and 711 — k1 parity bits. The var- ious portions of the structure are labeled d for data, ph for horizontal parity (along the rows), and p, for vertical parity (along the columns). Additionally, there are blocks labeled L6}, and L8,, which house the extrinsic LLR values learned from the horizontal and vertical decoding steps, respectively. Notice that this product code is a simple example of a con- catenated code. Its structure encompasses two separate encoding steps, horizontal and vertical. The iterative decoding algorithm for this product code proceeds as follows: 1 Set the a priori information L(d) = 0. 2 Decode horizontally, and using Eq. 13 obtain the hori- zontal extrinsic information as shown below: Lure?) = Ltd )A— Lcrx) —L(d>. 3 Set L(d) = Leh(c_l ). 4 Decode vertically, and using Eq. 13 obtain the vertical extrinsic information as shown below: 1au>=ufirLertwr 5 Set Ltd) = Lad? ). 6 If there have been enough iterations to yield a reliable decision, go to step 7; otherwise, go to step 2. ‘ 7 The soft output is: L0?) = Lax) + Lao? ) + Lent). <14) TWO-DIMENSIONAL SlNGLE-PARITY CODE EXAMPLE At the encoder, let the data bits and parity bits take on the values shown in Fig. 4, where the relationships between the data and parity expressed as binary digits (0,1), are as fol- lows: di (‘9 dj = pij di : ©sz (16) where (9 denotes modulo-2 addition. As shown in Fig. 4, the transmitted symbols are represented by the sequence d1 d2 d3 d4p12p34p13p24. At the receiver input, the received symbols are represented by the sequence {xi, 15,-j}, where x,- = d, + n for each received data signal, xij = pi; + n for each received parity signal, and n represents inde— pendent and identically distributed noise. For notational simplicity, we shall denote the received sequence with a single index, as {xk}, where k can be treated as a time index. Using the relationships developed in Eqs. 9 through 11, and assuming Gaussian noise, we can write the LLR for the channel measurement of a received signal xk as follows: ‘ x d = l L..(xt)=log.[——“p< kl [(71+)] (18) (19) 2 _ 2 1(ka1) :1£lk+1j _ 22 Xk 2 0 2 G o' where the natural logarithm is used. If we further make a simplifying assumption that the nOise variance 02 is unity, then' Lc(Xk) = Zxk (21) Consider the following example, where the data sequence d1 d2 d3 d4 is made up of the binary digits 1 0 0 1, as shown in Fig. 4. By the use of Eq. 15, it is seen that the parity sequence p12p34p13 p24 must be made up of 1 1 1 1. Thus, the transmit- ted sequence is ‘ {d,,p.j}=10011111 which is shown in Fig. 4 as the encoder output. Expressed in terms of bipolar voltage values, the transmitted sequence is +1 —1—1 +1 +1 +1 +1 +1 Let us assume that the noise transforms this data-plus-parity sequence into the received sequence {xk} = 0.75, 0.05, 0.10, 0.15, 1.25, 1.0, 3.0, 0.5 From Eq. 21, the assumed channel measurements yield the LLR values {Lc(xk)} = 1.5. 0.1, 0.20, 0.3. 2.5, 2.0, 6.0, 1.0 which is shown in Fig. 4 as the decoder input measurements. It should be noted that, if hard decisions are made on the {xk} or the {Lc(xk)} values shown aboVe, such a detection pro— cess would result in two errors, since d2 and d3 would each be incorrectly classified as binary 1. LOG-LIKELIHOOD ALGEBRA To best explain the iterative feedback of soft decisions, we introduce the idea of a log—likelihood algebra [5]. For sta- tistically independent d, we define the sum of two log likeli- hood ratios (LLRs) as follows: ' 96 IEEE Communications Magazine ' December 1997 eL(d1) +62%) L011) Q L(d2)AL(d1 @ d2) =10ge W (22) 1+e 1 e 2 s <—1) x sign[L(d1)]~x sign[L(d2)] ‘ (23) X min(lL(d1)2 |L(d2) I) where the natural logarithm is used. There are three addition operations in Eq. 22. + is used for ordinary addition, ® is used for modulo-2 addition of the data expressed as binary digits, and E is used for log-likelihood addition. The sum of two LLRs is denoted by the operator EH, where such an addi- tion is defined as the LLR of the modulo-2 sum of the under- lying data bits. The development of the Eq. 22 equality is shown in Appendix A. Equation 23 is an approximation of Eq. 22 that will prove useful later in a numerical example. The sum of LLRs as described by Eq. 22 or 23 yield the fol- lowing interesting results when one of the LLRs is very large or very small: ~ L(d) 1 oo = —L(d) and L(d) i 0 = 0 Note that the log—likelihood algebra described here dif- fers slightly from Hagenauer’s reference [5] because of choosing the null elemeth differently. In this article, the null element of the binary set (0, 1) has been chosen to be the 0. ‘ EXTRIRNSINC LIKELIHOODS For the product-code example in Fig. 4, we use Eq. 13 to express the soft output L(d1) for the received signal corre- sponding to data d1, as follows: ‘ L(dA 1) =Lc(x1) + L(d1) + ([Lc(xz) + L(d2)lEH Lc(x12)) (24) where the terms ([Lc(x2) + L(d2)] [E Lc(x12)) represent the extrinsic LLR contributed by the code (i.e., the signal corre- sponding to data d2 and its a priori probability, in conjunc- tion with the signal corresponding to parity p12). In general, the soft output L(di) for the received signal corresponding to, data d, is: Max) = Law) + L(di) +,([Lc(x}') + L(dj)] ['3 Lc(xij)) (25) where Lc(x,-), Lc(xj), and Lc(xij) are the channel LLR mea- surements of the received signals corresponding to 61,-, dj, and pi], respectively. L(d,~) and L(dj) are the LLRs of the a priori probabilities of d, and dj, respectively, and ([Lc(xj) + L(dj) E Lc(x,~j)) is the extrinsic LLR contribution from the code. Equations 24 and 25 can best be understood in the context of the decoder in Fig. 4. For example, the soft out- put L(d 1) is represented by the detector LLR measurement of received signal corresponding to data d1, and the extrin— sic LLR gleaned from the fact that data d2 and parity p12 also provide knowledge about data d1, as seen from Eqs. 15 through 17. ’ COMPUTING THE EXTRINSIC LIKELIHOODS or the example in Fig. 4. the horizontal calculations for Leh(d) and the vertical calculations for Lev(d) are expressed as follows: Leh(al) = [Lc(x2) + Lc(x12) Lev(31) = [Lc(x3) + L(d3)l Q Lc(x13) (26a) (26b) Encoder output binary digits H H ~ Decoder input log—likelihood ratios LAX) = 0.3 LLC(X34) = 2.0 174x13) = 6.0 Lc(xz4) = 1.0 - Figure 4. Product code exantple. Lara» = [Law + Lei) E Lc(x12) (27a) Lenin) = [Loom + L(d4)l E Lc(x24) (27b) Len(33) = [Lawn + L<d01E Lc(x34) <28a) Levee = [Law + Leo] E Laxn) (28b) Len(34) = [Law + Lea] E Lc(x34) (292) Lev(a4) = [Lc(x2) + 13012)] i Lc(x24) (29b) The LLR values shown in Fig. 4 are entered into the Leh(d) expressions in Eqs. 26 through 29 and the L(d) values are ini- tially set equal to zero, yielding: Lend/{1) = (0.1 + 0) L8,,(32) = (1.5 + 0) Lana) = (0.3 + 0) Legal.) = (0.2 + 0) 2.5 = ——0.1 = new L(d1) (30) 2.5 = —1.5 = new L(d2) (31) 2.0 = —0.3 = new L(d3) (32) 2.0 = —0.2 = new L(d4) (33) ‘H-H'ZH-H- where the log-likelihood addition has been calculated using the approximation in Eq. 23. We next proceed to calculate the first vertical calculations using the Lev(d) expressions in Eqs. 26 through 29. Now the values of L(d) can be refined by using the new L(d) values gleaned from the first horizontal calcula- tions, shown in Eqs. 30 through 33. Lev(d\1) = (0.2 _ 0.3) :l: 6.0 = 0.1 = new L(d1) (34) Lap?» = (0.3 v 0.2) :t 1.0 = —0.1 = new L(d2) (35) Lev(d3) = (1.5 _ 0.1) E 6.0 = —1.4 = new L(d3) (36) Lena.) = (0.1 — 1.5) 3 1.0 = 1.0 = newL(d4) ‘ (37) The results of the first full iteration of the two decoding steps (horizontal and vertical) are shown in Tables 1—3. Table 1. Table 2. Leh(d ) Table 3. LCVQ ) Original Lc(xk) after first hort— after first vertical measurements. zontal decoding. decoding. Each decoding step improves the original LLR Channel measurements. This is seen by calculating the deCoder output LLR using Eq.14. The original LLR plus the horizontal IEEE Communications Magazine ' December 1997 97 extrinsic LLRs yields, the improvement 'n Table 4 (the extrin— sic vertical terms are not yet being considered). 1.4 —1.4 .401 [0.1 Table 4. Improved LLRS dkle to Leh<d The original LLR plus both the horizontal and vertical extrinsic LLRs yields the improvement in Table 5. 1.5 —1.5 —1.5 1.1 Table 5. Improved 'VLLARS dlle toA _ ‘ Leh(d) + Lev(d) ‘ For this eXample, it is seen that the horizontal parity alone yields the correct hard decisions out of the decoder, but with very low confidence for data bits d3 and d4. After enhancing the decoder LLRs with the vertical extrinsic LLRs, the new LLR solutions have better confidence. Let us pursue one additional horizontal and vertical decoding iteration to see if there are any significant changes in the results. We again use the relationships shown in Eqs. 26 through 29 and proceed with the second horizontal calculations for Leh(d) using the new L(d) from the first vertical. calculations, shown in Eqs. 34 through 37. Lela? 1) = (0.1— 0.1) -— 2.5. ; 0",: new L(d1) (38) Le),(d\2) = (1.5 + 0.1) + 2.5 = —1.6'= new L(d2) (39) L6,,(a3) : (0.3 + 1.0) + 2.0 = —1.3 = new L(d3) 7 (40) Leh(d4) = (0.2 — 1.4) +‘ 2.0 = 1.2 =- new L(d4) (41) Next, we proceed With the second vertical calculations for ‘ Lev(d ) using the new L(d) from the second horizontal calcula— . ' tions, shown in Eqs: 38 through 41. V Lao.) _= (0.2 4 1.3,) + 6.0-: 1.1,: new L(d1) (42) - Lao?» = (0.3+ 1.2) I31 10 = —1.0 = new L(_d2) (43) L433) = (1.5 + 0) m 6.0 4 —1.5 = new L(d3) (44) twain = (0.1 41.6)4 1.0 =' 1,0 = new L(d4) (45) After the second iteration of horizontal and vertical deOOd- ‘ ing calculations, the soft—output likelihood values are again calculated from Eq. 14, rewritten below: ‘ _ L0?) = Lee) + Lara?) +’ are?) ‘ (46) V The final horizontal and vertical extrinsic LLRs, and the resulting decoder LLRs are displayed in Tables 6—8. For this example, the second full iteration of both codes (yield- ing a total of 4 iterations) suggests .a modest improvement over one full iteration. Now, there seems to be a balancing of the confidence values among each of the four data deci- s1ons. ~ The soft output is shown in Table 9. 2.6 —2.5 —1 —2.6 2.5 Table 9. L(d )A= L0(X) + Liah(d) + Lev(d). COMPONENT CODES FOR IMPLEMENTING TURBO CODES We have described the basic concepts of concatenation, iteration, and soft decisions using a simple product-code model. We next apply these ideas to the implementation of turbo codes. We form a turbo Code by the parallel concatena— tion Of component codes (building blocks) [3, '6]. First, a reminder of a simple binary rate—1/2 convolutional encoder with constraint length K and memory K — 1. The input to the encoder at time k is a bit dk, and the correspond.— ing codeword is the bit pair (uk, 11k), where ‘ K—l We 2 Zglidk—i m0d2 gli =0,1 (47) i=0» K—l Vk = ZgZidk—i m0d2 g21?0,1 (48) i=0 G1 = {gli} and G2 = {572,-} are the code generators and dk is represented as a binary digit. This encoder has a finite impulse response (FIR), and gives rise to the familiar non- systematic convolutional code (NSC), an example of which is seen'in Fig. 5. In this example, the constraint length is K = 3, and the two code generators are described by G1 ={111} and G2 = {101}. It is well known that, at large Eb/NO, the error performance of an NSC is better than that Of a system— atic code with the same memory.'At small Eb/No, it is gener— ally the other way around [3]. A class of infinite impulse response (HR) convolutional codes has been proposed [3] for the building blocks Of a turbo code. Such codes are also referr‘edto as recursive systematic convolutional (RSC) codes because previously encoded information bits are continually fed back to the encoder’s input. For high code rates,:RSC codes result in better error performance than [the best NSC codes at any Eb/NO. A binary rate—1/2 RSC code is obtained from an NSC code by using a feedback 100p, and settingfone of the two outputs (uk or Vk) equal tO dk. Figure 6 illustrates an example of such an RSC code, with K = 3, where ink is recursively calculated as ' ' ' ‘ K—1 ak = dk + Zgiak—i i=1 and where g,’ is respectively equal tO gli if uk =. dk, and to g2,- if vk = dk. I We assume that the input bit dk takes on values of 0 or 1 with equal probability. It is stated in [3] that ak exhibits the same statistical properties as dk. The trellis structure is identical for the RSC code of Fig. 6 and the NSC code Of Fig. 5, and these two codes have the same free distance. However, the two output sequences {uk} mod 2 (49) * and {12k} do not correspond to the same input sequence {dk} for RSC and NSC codes. For the same code genera— tors, we can say that RSC codes do not modify the out- put weight distribution Of the output codewords compared to NSC codes. They only change the mapping between input data sequences and output codeword sequences. 1.5 0.1 ' 0 i .—1.6 1.1 —1.0 0.2 _‘ 0.3 ' 41.3 1.2‘ —1.5 1.0 Table 6. Original Table 7. Leh(d) Table 8. LCVG) .Lc(x) after second norl- . after second verti— measuremenls. ‘zonml decoding. cal decoding. 93 ' IEEE Communications Magazine 0 December 1997 CONCATENATION OF i RSC CODES j Consider the parallel concatenation of two RSC encoders of the type shown in Figure 6. Good turbo codes have been constructed with their compo- nent codes having quite short constraint “k words from one encoder with low‘ weight codewords from the other encoder. Many such pairings can be avoided by proper design of the inter‘ leaver. An interleaver that permutes the data in a random fashion provides vk better performance than the familiar J block interleaver [8]. lengths (K = 3 to 5). An example of a such a turbo encoder is shown in Fig. 7, where the switch yielding vk punctures the code, making it rate 1/2. Without the switch, the code would be rate 1/3. Additional concate— nations can be continued with multiple component codes. In general, the component encoders need not be identical with regard to constraint length and rate. In designing turbo codes, the goal is to choose the best component codes by maximizing the effective free distance of the code [7]. At large values of Eb/No, this is tantamount to maximizing the minimum weight codeword. However, at low values of Eb/No (the region of greatest interest) optimizing the weight distri- bution of the codewords is more important than maximizing the minimum weight [6]. The turbo encoder in Fig. 7 produces codewords from each of two component encoders. The weight distribution for the codewords out of this parallel concatenation depends on how the codewords from one of the component encoders are combined with codewords from the other encoder(s). Intuitively, we would like to avoid pairing low-weight code- C2 RSC code InFi—gure 7. Parallel ~cor-z-ca-t-enation-of RSC encoders. - I Fig-ure 5. Nonsystenttztic- convolu- tional (NSC) code. If the encoders are not recursive, a low-weight codeword generated by the input sequence at = (O 00 0 1 0 0 0 0) with a single binary 1 will always appear again at the input of the second encoder for any Choice of interleaver. In other words, the interleaver would not influence the codeword weight distribution if the codes were not recursive. However, if the component codes are recursive, a weight—1 input sequence generates an IIR (infi— nite weight output). Therefore, for the case of recursive codes, the weight—1 input sequence does not yield the mini— mum weight codeword out of the encoder. The encoded output weight is kept finite only by trellis termination, a process that forces the coded sequence to terminate in the zero state. In effect, the convolutional code is converted to a block code. For the Fig. 7 encoder, the minimum weight codeword for each component encoder is generated by the weight-3 input sequence (0 0 0 0 1 1 1 0 0 0 0 0) with three consecutive 1s. Another input that produces fairly low-weight codeWords is the weight-2 sequence (0 0 0 0 1 0 0 1 0 0 0 0). However, after the permutations of an interleaver, either of these dele— terious input patterns is not likely to appear again at the input to another encoder, so it is unlikely that a minimum weight codeword will be combined with another minimum weight codeword. The important aspect of the building blocks used in turbo codes is that they are recursive (the systematic aspect is merely incidental). It is the RSC code’s IIR property that pro- tects against those low-weight encodings which cannot be remedied by an interleaver. One can argue that turbo code performance is determined largely from minimum weight codewords that result from the weight—2 input sequence. The argument is that weight—1 inputs can be ignored since they yield large weights due to the IIR code structure, and for input sequences having weight-3 and larger, a properly designed interleaver makes the number of such input words relatively rare [7—11]. A FEEDBACK DECODER The Viterbi algorithm (VA) is an optimal decoding method for minimizing the probability of sequence error. Unfortu- nately, the VA is not able to yield the APP or soft-decision output for each decoded bit. A relevant algorithm for doing this has been proposed by Bahl et al. [12]. The Bahl algorithm was modified by Berrou et al. [3] for use in decoding RSC codes. The APP of a decoded data bit dk can be derived from the joint probability A,§(m) defined by Adm) = Pidk = 1', Sk = m IRfV} (50) where S k is the state at time k, and R IN is a received sequence from time k = 1 through some time N. Thus, the APP for a decoded data bit dk, represented as a binary digit, is equal to P{dk =i|R1N}= 2N, (m) , 1': 0,1 . (51) The log-likelihood function is written as the logarithm of the ratio of APPS, as follows: IEEE Communications Magazine - December 1997 99 __Foed back loop Z [ L167 k) Lfid n) JDeimep 1:42“? N Fig. 8. For a discrete memoryless Gaus- ki |_ _ _ __J Eaving sian channel and binary modulation, the i a; Decoder tater it Decoder ' : decoder input is made up of a couple Rk I ___ i DEC1 leaving. DECZ F' D , t L2(3 k) of two random variables xk and yk. k _ _ __ _[ --——-- ' ""1 ———— [egcisr’l-~ Where dk and vk are bits, we can express Fifi—i y y ———g~ _:__ the received bit—to—bipolar pulse conver— t?“ 1k 2k _:J sion at time k as follows: I Yk I _— XI.» = (261k - 1) + ik (54) v . Demux ‘ Yk = (2V1: — 1) + 61k (55) Decoded output If y ' where ik and qk are two independent nois— ( k es with the same variance 62. The redun- !_ i_ - 1 .. . o 1 2 3 4 5 Eb/Nowsi‘ I . . . . . .. . . _ I Figure 9. Bit error probability as a function of Eb/No and mul- tiple iterations. (52) The decoder can make a decision by comparing L(c/l\k) to a zero threshold. 3k=i if Lain >0 dk = 0 if L(dk) < 0 For a systematic code, the LLR L(d\k) assoeiated with each decoded bit dk can be described as the sum of the LLR of (1k out of the detector and of other LLRs generated by the decoder (extrinsic information), as was expressed in Eqs. 12 and 13. Consider the detection of a noisy data sequence that stems from the encoder of Fig. 7. The decoder is shown in dant information, yk, is demultiplexed and sent to decoder DEC1 as ylk when vk : vlk, and to decoder DECZ as M when vk = 13%. When the redundant informa- tion of a given encoder (C1 OR C2) is not emitted, the corre— sponding decoder input is set to zero. Note that the output of DEC1 has an interleaver structure identical to the one used at the transmitter between the two encoders. This is because the information processed by DEC1 is the noninterleaved output of C1 (corrupted by channel noise). Conversely, the information processed by DECZ is the noisy output of C2 whose input is the same data going into C1, but permuted by the interleaver. DECZ makes use of the DEC1 output, provided this output is time ordered in the same way as the input to C2. DECODING WITH A FEEDBACK LOOP We rewrite Eq. 13 for the soft decision output at time k, with the a priori LLR L(dk) initially set to zero. This follows from the assumption that the data bits are equally likely. L(&k) = Lc(xk)+Le(tik) = 10g[%]+n (56) where L((/1\k) is the soft decision output, and Lc(xk) is the LLR channel measurement, stemming from the ratio of likelihood functionspggkldk = i) of the discrete memoryless channel. Le(dk) = L(dk) ka = 0 is a function of the redundant informa- tion. It is the extrinsic information supplied by the decoder, and does not depend on the decoder input xk. Ideally, Lc(xk) and/\Le(dk) are corrupted by uncorrelated noise, and thus Le(dk) may be used as a new observation of dk by another ‘ decoder for an iterative process. The fundamental principle for feeding back information to another decoder is: never feed a decoder with information that stems from itself (because the input and output corruption will be highly correlated). For the Gaussian channel, we use the natural logarithm in Eq. 56 to describe the channel LLR, Lc(xk), as was done in Eqs. 18 through 20. We rewrite the Eq. 20 LLR result below: 2 2 . V l xk —1 l xk +1 2 L — + — , C(Xk) 2[ 60 J 2[ 60 j 03 a (57) Both decoders DEC1 and DAECZ use the modified Bahl algorithm [6]. If the inputs L1(dk) and ka toAdecoder DECZ are independent, then the log—likelihood L 2(d k) at the output of DEC2 can be written as Lactic) =flLzrfik>i + Lac?» (58) with 100 IEEE Communications Magazine - December 1997 A 2 A L1 (dk) = 735k +Lel(dk) (59) 00 wheref[ ] indicates/\a functional relationship. The extrinsic information kegajk) out of DBCZ is a function Of the seguence {L1(d,,)},, ,é k. Since L1(c/i\,,) depends on observation R1 , the extrinsic information L62(dk) is correlated with Obser— vations xk and ylk. Nevertheless, the greater |n — k] is, the less correlated are L1(dn) and the observations xk, yk. Thus, due to the interleaving between DECl and DECZ, the extrin- sic information Lez(dk) and the observations xk, ylk are weak- ly correlated. Therefore, they can be jointly used for the decoding of bit dk. 2k = L62(dk) acts as a diversity effect in an iterative pro- cess. In ger/teral, L62(dk) will have the same sign as dk. There- fore, LeZ(dk) may improve the LLR associated with each decoded data bit. A The algorithmic details for computing the LLR, L(dk), of the APP for each bit has been described by several authors [3—5], and suggestions for decreasing the implementational complexity are still ongoing [13—15]. A reasonable way to think of the process that produces APP values for each bit is to imagine computing an MLSE or a VA in two directions over the block of coded bits. Having metrics associated with states in the forward and backward direction allows us to compute the APP of a bit if there is a transition between two given states. Thus, we can proceed with this bidirectional VA and, in a sliding-window fashion, compute the APP for each code bit in the block. With this View in mind, wecan estimate that the complexity of decoding a turbo code is at least two times more complex than decoding one of its component codes using the VA. TURBO CODE ERROR PERFORMANCE EXAMPLE onte Carlo simulations have been presented for a rate-1/2, K = 5 encoder with generators G1 = {11111} and G2 = {10001}, and parallel concatenation. The interleaver was a 256 x 256 array. The modified Bahl algorithm has been used with a data block length of 65,536 bits [3]. For 18 iterations, the bit error probability PB is lower than 10'5 at Eb/NO = 0.7 dB. The error performance improvement as a function of iterations is seen in Fig. 9. Note that, as we approach the Shannon limit of —1.6 dB, the required bandwidth approaches infinity, and the capacity (code rate) approaches zero. For binary modulation, several authors use P3 = 10‘5 and Eb/NO = 0 dB as the Shan- non limit reference for a rate-1/2 code. Thus, with parallel con- catenation of RSC convolutional codes and feedback decoding, the error performance of a turbo code is at 0.7 dB from the Shannon limit. Recently, a class of codes that use serial instead of parallel concatenation of the interleaved building blocks have been proposed. It has been suggested that these codes may have superior performance to turbo codes [14]. SUMMARY In this article basic statistical measures, such as a posteriori probability and likelihood, are reviewed. We then use these measures for describing the error performance of a soft— input/soft-output decoder. A numerical example helps to illustrate how performance is improved when soft outputs from concatenated decoders are used in an iterative decoding process. We next proceed to apply these concepts to the par- allel concatenation of recursive systematic convolutional (RSC) codes, and explain why such codes are the preferred building blocks in turbo codes. A feedback decoder is described in general ways, and its remarkable performance presented. REFERENCES [1] G. D. Forney, Jr., Concatenated Codes, Cambridge, MA: MIT. Press, 1966. [2] J. H. Yuen, et al., "Modulation and Coding for Satellite and Space Communications," Proc. IEEE, vol. 78, no. 7, July 1990, pp. 1250—65. [3] C. Berrou, A. Glavieux, and P. Thitimajshima, “Near Shannon Limit Error- Correcting Coding and Decoding: Turbo Codes," IEEE Proc. ICC ’93, Geneva, Switzerland, May 1993, pp. 1064—1070. [4] C. Berrou and A. Glavieux, "Near Optimum Error Correcting Coding and Decoding: Turbo-Codes," IEEE Trans. Commun., vol. 44, no. 10, Oct. 1996, pp. 126141. [5] J. Hagenauer, "Iterative Decoding of Binary Block and Convolutional Codes," IEEE Trans. Info. Theory, vol. 42, no. 2, Mar. 1996, pp. 429—45. [6] D. Divsalar and F. Pollara, “On the Design of Turbo Codes," TDA progress rep. 42-123, Jet Propulsion Lab., Pasadena, CA, Nov. 15, 1995, pp. 99—121. [7] D. Divsalar and R. J. McEliece, "Effective Free Distance of Turbo Codes," Elect. Lett., vol. 32, no. 5, Feb. 29, 1996, pp. 445—46. [8] S. Dolihar and D. Divsalar, "Weight Distributions for Turbo Codes Using Random and Nonrandom Permutations," TDA progress rep. 42—122, Jet Propulsion Lab., Pasadena, CA, Aug. 15, 1995, pp. 56—65. [9] D. Divsalar and F. Pollara, "Turbo Codes for Deep~Space Communica- tions," TDA progress rep. 42-120, Jet Propulsion Lab., Pasadena, CA, Feb. 15, 1995, pp. 2939. [10] D. Divsalar and F. Pollara, "Multiple Turbo Codes for Deep-Space Com- munications," TDA progress rep. 42-121, Jet Propulsion Lab., Pasadena, CA, May 15, 1995, pp. 66—77. [11] D. Divsalar and F. Pollara, "Turbo Codes for PCS Applications,” Proc. ICC ’95, Seattle, WA, June 18—22, 1995. [12] L. R. Bahl et al., "Optimal Decoding of Linear Codes for Minimizing Symbol Error Rate,” Trans. Info. Theory, vol. IT—20, Mar. 1974, pp. 248—87. [13] S. Benedetto et. al., "Soft Output Decoding Algorithm in Iterative Decoding of Turbo Codes,” TDA progress rep. 42424, Jet Propulsion Lab., Pasadena, CA, Feb. 15, 1996, pp. 63—87. [14] S. Benedetto et al., "A Soft-Input Soft-Output Maximum A Posteriori (MAP) Module to Decode Parallel and Serial Concatenated Codes," TDA progress rep. 42-127, Jet Propulsion Lab., Pasadena, CA, NOV. 15, 1996, pp. 63—87. [15] S. Benedetto et al., “A Soft-Input Soft—Output APP Module for Iterative Decoding of Concatenated Codes," IEEE Commun. Lett, VO|.1, no. 1, Jan. 1997, pp. 22—24. BIOGRAPHY BERNARD SKLAR [LSM] has over 40 years of experience in technical design and management positions at Republic Aviation Corp., Hughes Aircraft, Lit- ton Industries, and The Aerospace Corporation. At Aerospace, he helped develop the MILSTAR satellite system, and was the principal architect for EHF Satellite Data Link Standards. Currently, he is head of advanced sys- tems at Communications Engineering Services, a consulting company he founded in 1984. He has taught engineering courses at several universities, including the University of California, Los Angeles and the University of Southern California, and has presented numerous training programs throughout the world. He has published and presented scores of technical papers. He is the recipient of the 1984 Prize Paper Award from the IEEE Communications Society for his tutorial series on digital communications, and he is the author of the book Digital Communications (Prentice—Hall). He is past chair of the Los Angeles Council IEEE Education Committee. His aca- demic credentials include a 8.5. degree in math and science from the Uni- versity of Michigan, an MS. degree in electrical engineering from the Polytechnic Institute of Brooklyn, New York, and a PHD. degree in engi- neering from the University of California, LOS Angeles. IEEE Communications Magazine ' December 1997 101 APPENDIX A L(d1®d2)=loge Below are presented the algebraic details yielding the results P (‘11 : +1)><P(d2 ‘ 1) I shown in Eq. 22, rewritten as follows: P(dl :+1)X p(d2 +1) [I P( d1 _ 1 MD P(d2 - 11)] A eL(dl) + eL(d2) L(d1) IE L012) "U611 (9 d2) =10ge W (60) L(d1) 1 1 L(d2) > l+e I e 2 e + e (d ) PM 1) 1 1+eL(dl) [1+eL(d2)] (1+eL(dl)] 1+6le2) _ p :+l~ : :+ _ egg 1 v 2 58 L(d)_10g“IP(d=—1)I logtlifpu=11>I (61) ELM 8w ti 1 1 I ( ) l+eL(d1) 1+ W2) 1+eLId‘) 1+eLId2) eud) : P(d=+l) I 1— P(d = +1) (62) 6W1)+ eLtdz) ele) —eL(d) xP(d = +1) = P(d = +1) (63) [mm] mm] ele) = P(d = +1) X [1 + eLl‘m (64) =loge (69) L(d) eLId‘)eL(d2) +1 - ~P d: 1 =——e 65 ‘ —r‘—*“—’ ( 9 new» ( ) [law] [HM] . 6L(d) _ 1 66 ‘ 1 P(d—I1)—1 1+6L(d)—1+6L(d) ( ) _10 eL(dI)+eL(d2) _ ’ 3‘3 fllfiLtdli'e‘Li—d.) Call for Papers. IEEE Network Magazine Special Issue on Network Management:Today and Tomorrow - Guest Editor — Roch H. Glitho Ericsson Research Canada, Town of Mount Royal, Quebec - Canada HAP 2N2 Tel: +I-5I4—345 7900 — Extension: 2266; Fax: +I—5I4-345 oIO5 Email: [email protected] Scope Network management is an emerging discipline and a vibrant area of research. Industry and academia interest is growing. This interest is clearly demonstrated by activities of network management related standards bodies and industry fora, conferences focusing on network man agement, and last but not least, development and deployment of network management products. Most of today's network management sys- tems follow a manager/agent approach. Managers located on dedicated nodes retrieve raw data collected by agents which are usually embedded in the nodes to-be managed. The data is retrieved using dedicated management protocols. The lETF simple network management protocol (SNMP) and the lSO/ITU-T common management information protocol (CMIP) are today’s classical management protocols. Today’s manager/agent approach is being challenged and new approaches are emerging for tomorrow's network management systems. These new approaches include the peer-to—peer approach stipulated by distribution technologies such as CORBA; the use at ubiquitous technologies such as the World Wide Web; and the delegation of classical manager tasks to agents via the use of mobile and/or intelligent agents. Today's management systems must be able interoperate among themselves. The same applies to tomorrow’s systems. Moreover, tomorrow’s manage ment systems, whenever in place, must be able to interoperate with today’s systems, which will be yesterday’s systems of the future. Interoper- abilitywill remain the key issue in network management. This special issue of IEEE Networkwill presentthe presentand future ofnetwork management. High quality tutorial-oriented references are solicited. Topics of interest include: 0 lETF Simple Network Management Protocol (SNMP) 0 Web-based network management 0 ISO/ITU-T OSI systems management ' Mobile agentsbased network management " ITU-T Telecommunication Management Network (TMN) 0 Intelligent agents—basednetwork management 0 CORBA~based network management 0 Interoperability frameworks Instructions to authors Authors are invited to submit four (4) hard copies of their manuscripts to the guest editor. Papers should not exceed 20 double-spaced pages in length, excluding figures. More information for potential authors is available at the IEEE Network home page: http://www.comsoc.org/soc- str/techcom/ntwrk/ Schedule 0 Paper submission deadline: March I5, I998 ' Notification of acceptance: May 30, I998 0 Final manuscripts due: June 30, I998 0 Publication date: September/October I998 ' 102 IEEE Communications Magazine ' December 1997 ...
View Full Document

This note was uploaded on 03/20/2012 for the course EEL 6650 taught by Professor Shea during the Spring '12 term at University of Florida.

Page1 / 9

sklar_primer - ABSTRACT The goal of this article is to...

This preview shows document pages 1 - 9. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online