This preview shows pages 1–9. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: ABSTRACT The goal of this article is to describe the main ideas behind the new class of codes called turbo codes, whose performance in terms of
bit error probability has been shown to be very close to the Shannon limit. In this article, the mathematical measures of a posteriori
probability and likelihood are reviewed, and the benefits of turbo codes are explained in this context. Since the algorithms needed to implement the decoders have been well documented by others, they are only referenced here, not described in detail. A numerical
example, using a simple concatenated coding scheme, provides a vehicle for illustrating how error performance can be improved when
soft outputs from the decoders are used in an iterative decoding process. A Primer on Turbo Code Concepts Bernard Sklar, Communications Engineering Services oncatenated coding schemes were first proposed by Forney [1] for achieving large coding gains by com
bining two or more relatively simple component or building—
block codes. The resulting codes had the error correction
capability of much longer codes, and were endowed with a
structure that permitted relatively easy to moderately complex
decoding. A serial concatenation of codes is most often used
for power—limited systems such as deep space probes. The
most popular of these schemes consists of a Reed—Solomon
outer code (applied first, removed last) followed by a convolu
tional inner code (applied last, removed first) A turbo
code can be thought of as a refinement of the concatenated
encoding structure and an iterative algorithm for decoding the
associated code sequence. Turbo codes were introduced by Berrou, Glavieux, and
Thitimajshima [3, 4]. For a bit error probability of 10—5 and
code rate = 1/2, the authors report a required Eb/NO of 0.7
dB. The codes are constructed by applying two or more com—
ponent codes to different interleaved versions of the same
information sequence. For any single traditional code, the
final step at the decoder yields hard—decision decoded bits (or,
more generally, decoded symbols). In order for a concatenat
ed scheme such as a turbo code to work properly, the decod—
ing algorithm should not limit itself to passing hard decisions
among the decoders. To best exploit the information learned
from each decoder, the decoding algorithm must effect an
exchange of soft rather than hard decisions. For a system with
two component codes, the concept behind turbo decoding is
to pass soft decisions from the output of one decoder to the
input of the other, and to iterate this process several times to
produce better decisions. ' ‘ A REVIEW OF LIKELIHOODS The mathematical foundations of hypothesis testing rests on
Bayes’ theorem, which is derived from the relationship
between the conditional and joint probability of eventsA and
B, as follows: P(A lB) P03) = H3 lA) P(A) = P(Aa) (1) A statement of the theorem yields the a posteriori probability
(APP), denoted P(A lB), P(BA)P(A) (2)
P(B) which allows us to infer the APP of an eventA conditioned on
B, from the conditional probability, P(B IA), and the a priori
probabilities, P(A) and P(B). For communications engineering applications having
additive white Gaussian noise (AWGN) in the channel, the
most useful form of Bayes’ theorem expresses the APP in
terms of a continuousvvalued random variable x in the fol—
lowing form: P(AlB) = p(xd : i)P(d = i) P(dilx)— pm i1,...,M (3)
and
M ,
plx) = éplxld = i)P(d = 2') (4) where d = i represents data d belonging to the ith signal class
from a set of M classes, and p(xd = i) represents the proba
bility density function (pdf) of a received continuous—valued
dataplus—noise signal, x, conditioned on the signal class at = i.
p(x) is the pdf of the received signalx over the entire space of
signal classes. In Eq. 3, for a particular received signal, p(x) is
a scaling factor since it has the same value for each class.
Lower case p is used to designate the pdf of a continuous—val
ued signal, and upper case P is used to designate probability
(a priori and APP). Equation 3 can be thought of as the result
of an experiment involving a received signal and some statisti—
cal knowledge of the signal classes to which the signal may
belong. The probability of occurrence of the ith signal Class
P(d = 1'), before the experiment, is the a priori probability. As
a result of examining a particular received signal, we can com
pute the APP, P(d = ilx), which can be thought of as a
“refinement” of our prior knowledge. THE TWOSIGNAL CLASS CASE Let the binary logical elements 1 and 0 be represented
electronically by voltages +1 and — 1, respectively. This
pair of transmitted voltages is assigned to the variable d,
which may now take on the values d = +1 and d = —1. Let 94 0163—6804/97/$10.00 © 1997 IEEE IEEE Communications Magazine ° December 1997 the binary 0 (or the voltage value —1) be the null
element under addition. For an AWGN channel,
Fig. 1 shows the conditional pdfs referred to as
likelihood functions. The rightmost function p(xd
+1) shows the pdf of the random variable x
given the condition that d = +1 was transmitted.
The leftmost function p(xd = —1) illustrates a sim
ilar pdf given that d = — 1 was transmitted. The
abscissa represents the full range of possible values
of it received at the detector. In Fig. 1 is shown one
such arbitrary predetection value xk. A line sub
tended from xk intercepts the two likelihood func—
tions yielding two likelihood values XI and lug. A popular
hard decision rule, known as maximum likelihood, is to
choose the symbol d = +1 or d = —1 associated with the
larger of the two intercept values, M or M . This is tanta
mount to deciding dk = +1 if xk falls on the right side of
the decision line labeled 70, otherwise deciding dk = —1. A. similar decision rule, known as maximum aposteriori
(MAP) or minimumerror rule, takes into account the a priori
probabilities, as seen below in Eq. 6. The MAP rule is
expressed in terms of APPS as follows: P(d = +1x): P(d = —1x) H2 (5) Equation 5 states that one should choose the hypothesis H1 (d
: +1) if the APP P(d = +1 x) is greater than the APP P(d =
—1x). Otherwise, choose hypothesis H2(d = —1). Using the
Bayes’ theorem in Eq. 3, the APPS in Eq. 5 can be replaced
by their equivalent expressions, yielding:
H1
p(xd = +1)P(d = +1): p(xd = —1)P(d
, H0 1) (6) Equation 6 is generally expressed in terms of a ratio, called
the likelihood ratio test, as follows: Plxld = +1) :11 PM = ~1)
poh=4)h,Pw=+u
p(xd : +1) P(d = +1) :11 p(xd = —1) P(d = —1) 1 712 (7) ‘ FigureTliikelihoodjitnctiorls.  Likelihood of d = —l
P(xd = ~1) Likelihood of d = +1
p(xd = +1) Y0 LOGLIKELIHOOD RATIO By taking the logarithm of the likelihood ratio in Eq. 7, we
obtain a useful metric called the loglikelihood ratio (LLR).
It is the real number representing a soft decision out of a
detector, designated L(d x), as follows: (8) _ P(d = +llx) p(xld = +1)P(d = +1)
L(dlx)_1°g[1=(d = —lx)] ogl p(xd = —1)P(d = 1) weddnhed“dmee> m
L(d Ix) = L(xld) + L(d) V (10) where L(xd) is the LLR of the channel measurements ofx
under the alternate conditions that d = +1 or d = —1 may
have been transmitted, and L(d) is the a priori LLR of the data bit d.
To simplify the notation, we represent Eq. 10 as follows: L’(g) = LAX) + L(d) (11) where the notation Lc(x) emphasizes that this LLR term is the
result of a channel measurement made at the detector. Equa
tions 3 through 11 were developed with only a data detector
in mind. Next, the introduction of a decoder will typically
yield decision—making benefits. For a systematic code, it can
be shown [3] that the LLR (soft output) L(d) out of the
decoder is equal to W) = L'(c?) + Ltd) (12) where L’(d ) is the LLR of a data bit out of the detector (input
to the decoder), and L80? ), called the extrinsic LLR, repre
sents extra knowledge that is gleaned from the decoding pro
cess. The output sequence of a systematic decoder is made up
of values representing data and parity. Equation 12 partitions the decoder LLR into the data portion represented Feedback for the next iteration L(d) _ _ _>. _ _ _ _ _ _I
Leld) Extrinsic
values out soft—out , Detector a post rori
decoder LLR value L'( )
= Lc(x) + L(d) Ltd) A posteriori
values LC(X) Channel
values In by the detector measurement, and the extrinsic por
tion represented by the decoder contribution due to
parity. From Eqs. 11 and 12, we write Let) = Lax) + L(d) + Lee?) (13) The soft decision L(d) is a real number that pro—
vides a hard decision as well as the reliability of that
decision. The sign of L(tt) denotes the hard deci
sion; that is, for positive values of L(d ) decide +1,
for negative values decide —1. The magnitude of
L(d ) denotes the reliability of that decision. PRINCIPLES .OF
ITERATlVE (TURBO) DECODING For the first decoding iteration of the soft
input/soft output decoder in Fig. 2, one gener — Figure 2. Softinput/s’oﬁ outpitt decoder (for a systematid code). ally assumes the binary data to be equally likely, IEEE Communications Magazine 0 December 1997 95 [(2 n —k2
columns co umns k1, rows d p,, Leh
I __ ._ _ _.
l m stat.
l —__ Ext! i nsic—a
vertical I Figure "Twodimensionalproduct code. yielding an initial a priori LLR value 0fL(d) =0 for the third term in Eq. 9. The channel predetection LLR value, Lc(x), is measured by forming the logarithm of the ratio of 901 and 702, seen in Fig. 1A and appearing as the second term of Eq. 9. The output L(d ) of the Fig. 2 decoder is made up of the LLR from the detector, L13), and the extrinsic LLR
output. L461). representing knowledge gleaned from the
decoding process. As illustrated in Fig. 2, for iterative decod—
ing the extrinsic likelihood is fed back to the decoder input,
to serve as a refinement of the a priori value for‘the next iteration. ' .7 Consider the two—dimensional code (product code) depict
ed in Fig. 3. The configuration can be described as a data
array made up of k1 rows and k2 columns. Each of the k1 rows
contains a code vector made up of k2 data bits and n2 — k2
parity bits. Similarly, each of the k2 columns contains a code
vector made up of k1 data 'bits and 711 — k1 parity bits. The var
ious portions of the structure are labeled d for data, ph for
horizontal parity (along the rows), and p, for vertical parity (along the columns). Additionally, there are blocks labeled L6}, and L8,, which house the extrinsic LLR values learned from the horizontal and vertical decoding steps, respectively. Notice that this product code is a simple example of a con catenated code. Its structure encompasses two separate encoding steps, horizontal and vertical. The iterative decoding
algorithm for this product code proceeds as follows: 1 Set the a priori information L(d) = 0. 2 Decode horizontally, and using Eq. 13 obtain the hori
zontal extrinsic information as shown below: Lure?) = Ltd )A— Lcrx) —L(d>. 3 Set L(d) = Leh(c_l ). 4 Decode vertically, and using Eq. 13 obtain the vertical
extrinsic information as shown below:
1au>=uﬁrLertwr 5 Set Ltd) = Lad? ). 6 If there have been enough iterations to yield a reliable
decision, go to step 7; otherwise, go to step 2. ‘ 7 The soft output is: L0?) = Lax) + Lao? ) + Lent). <14) TWODIMENSIONAL SlNGLEPARITY
CODE EXAMPLE At the encoder, let the data bits and parity bits take on the
values shown in Fig. 4, where the relationships between
the data and parity expressed as binary digits (0,1), are as fol
lows: di (‘9 dj = pij di : ©sz (16) where (9 denotes modulo2 addition. As shown in Fig. 4,
the transmitted symbols are represented by the sequence
d1 d2 d3 d4p12p34p13p24. At the receiver input, the
received symbols are represented by the sequence {xi, 15,j},
where x, = d, + n for each received data signal, xij = pi; +
n for each received parity signal, and n represents inde—
pendent and identically distributed noise. For notational
simplicity, we shall denote the received sequence with a
single index, as {xk}, where k can be treated as a time
index. Using the relationships developed in Eqs. 9 through
11, and assuming Gaussian noise, we can write the LLR
for the channel measurement of a received signal xk as
follows: ‘ x d = l
L..(xt)=log.[——“p< kl [(71+)] (18) (19) 2 _ 2
1(ka1) :1£lk+1j _ 22 Xk 2 0 2 G o'
where the natural logarithm is used. If we further make a simplifying assumption that the nOise variance 02 is
unity, then' Lc(Xk) = Zxk (21) Consider the following example, where the data sequence d1
d2 d3 d4 is made up of the binary digits 1 0 0 1, as shown in
Fig. 4. By the use of Eq. 15, it is seen that the parity sequence p12p34p13 p24 must be made up of 1 1 1 1. Thus, the transmit
ted sequence is ‘ {d,,p.j}=10011111 which is shown in Fig. 4 as the encoder output. Expressed in
terms of bipolar voltage values, the transmitted sequence is +1 —1—1 +1 +1 +1 +1 +1 Let us assume that the noise transforms this dataplusparity
sequence into the received sequence {xk} = 0.75, 0.05, 0.10, 0.15, 1.25, 1.0, 3.0, 0.5 From Eq. 21, the assumed channel measurements yield the
LLR values {Lc(xk)} = 1.5. 0.1, 0.20, 0.3. 2.5, 2.0, 6.0, 1.0 which is shown in Fig. 4 as the decoder input measurements.
It should be noted that, if hard decisions are made on the
{xk} or the {Lc(xk)} values shown aboVe, such a detection pro—
cess would result in two errors, since d2 and d3 would each be
incorrectly classified as binary 1. LOGLIKELIHOOD ALGEBRA To best explain the iterative feedback of soft decisions, we
introduce the idea of a log—likelihood algebra [5]. For sta
tistically independent d, we define the sum of two log likeli
hood ratios (LLRs) as follows: ' 96 IEEE Communications Magazine ' December 1997 eL(d1) +62%)
L011) Q L(d2)AL(d1 @ d2) =10ge W (22)
1+e 1 e 2 s <—1) x sign[L(d1)]~x sign[L(d2)] ‘ (23)
X min(lL(d1)2 L(d2) I) where the natural logarithm is used. There are three addition
operations in Eq. 22. + is used for ordinary addition, ® is
used for modulo2 addition of the data expressed as binary
digits, and E is used for loglikelihood addition. The sum of
two LLRs is denoted by the operator EH, where such an addi
tion is defined as the LLR of the modulo2 sum of the under
lying data bits. The development of the Eq. 22 equality is
shown in Appendix A. Equation 23 is an approximation of
Eq. 22 that will prove useful later in a numerical example.
The sum of LLRs as described by Eq. 22 or 23 yield the fol
lowing interesting results when one of the LLRs is very large
or very small: ~ L(d) 1 oo = —L(d)
and
L(d) i 0 = 0 Note that the log—likelihood algebra described here dif
fers slightly from Hagenauer’s reference [5] because of
choosing the null elemeth differently. In this article, the null element of the binary set (0, 1) has been chosen to be
the 0. ‘ EXTRIRNSINC LIKELIHOODS For the productcode example in Fig. 4, we use Eq. 13 to
express the soft output L(d1) for the received signal corre
sponding to data d1, as follows: ‘ L(dA 1) =Lc(x1) + L(d1) + ([Lc(xz) + L(d2)lEH Lc(x12)) (24) where the terms ([Lc(x2) + L(d2)] [E Lc(x12)) represent the
extrinsic LLR contributed by the code (i.e., the signal corre
sponding to data d2 and its a priori probability, in conjunc
tion with the signal corresponding to parity p12). In general, the soft output L(di) for the received signal corresponding to, data d, is:
Max) = Law) + L(di) +,([Lc(x}') + L(dj)] ['3 Lc(xij)) (25) where Lc(x,), Lc(xj), and Lc(xij) are the channel LLR mea
surements of the received signals corresponding to 61,, dj,
and pi], respectively. L(d,~) and L(dj) are the LLRs of the a
priori probabilities of d, and dj, respectively, and ([Lc(xj) +
L(dj) E Lc(x,~j)) is the extrinsic LLR contribution from the
code. Equations 24 and 25 can best be understood in the
context of the decoder in Fig. 4. For example, the soft out
put L(d 1) is represented by the detector LLR measurement
of received signal corresponding to data d1, and the extrin—
sic LLR gleaned from the fact that data d2 and parity p12
also provide knowledge about data d1, as seen from Eqs. 15
through 17. ’ COMPUTING THE EXTRINSIC LIKELIHOODS or the example in Fig. 4. the horizontal calculations for
Leh(d) and the vertical calculations for Lev(d) are expressed as follows:
Leh(al) = [Lc(x2) + Lc(x12)
Lev(31) = [Lc(x3) + L(d3)l Q Lc(x13) (26a)
(26b) Encoder output binary digits H H ~ Decoder input log—likelihood ratios LAX) = 0.3 LLC(X34) = 2.0 174x13) = 6.0 Lc(xz4) = 1.0  Figure 4. Product code exantple. Lara» = [Law + Lei) E Lc(x12) (27a)
Lenin) = [Loom + L(d4)l E Lc(x24) (27b)
Len(33) = [Lawn + L<d01E Lc(x34) <28a)
Levee = [Law + Leo] E Laxn) (28b)
Len(34) = [Law + Lea] E Lc(x34) (292)
Lev(a4) = [Lc(x2) + 13012)] i Lc(x24) (29b) The LLR values shown in Fig. 4 are entered into the Leh(d)
expressions in Eqs. 26 through 29 and the L(d) values are ini
tially set equal to zero, yielding: Lend/{1) = (0.1 + 0)
L8,,(32) = (1.5 + 0)
Lana) = (0.3 + 0)
Legal.) = (0.2 + 0) 2.5 = ——0.1 = new L(d1) (30)
2.5 = —1.5 = new L(d2) (31)
2.0 = —0.3 = new L(d3) (32)
2.0 = —0.2 = new L(d4) (33) ‘HH'ZHH where the loglikelihood addition has been calculated using
the approximation in Eq. 23. We next proceed to calculate the
first vertical calculations using the Lev(d) expressions in Eqs.
26 through 29. Now the values of L(d) can be refined by using
the new L(d) values gleaned from the first horizontal calcula
tions, shown in Eqs. 30 through 33. Lev(d\1) = (0.2 _ 0.3) :l: 6.0 = 0.1 = new L(d1) (34)
Lap?» = (0.3 v 0.2) :t 1.0 = —0.1 = new L(d2) (35)
Lev(d3) = (1.5 _ 0.1) E 6.0 = —1.4 = new L(d3) (36)
Lena.) = (0.1 — 1.5) 3 1.0 = 1.0 = newL(d4) ‘ (37) The results of the first full iteration of the two decoding
steps (horizontal and vertical) are shown in Tables 1—3. Table 1. Table 2. Leh(d ) Table 3. LCVQ )
Original Lc(xk) after ﬁrst hort— after ﬁrst vertical
measurements. zontal decoding. decoding. Each decoding step improves the original LLR Channel
measurements. This is seen by calculating the deCoder output
LLR using Eq.14. The original LLR plus the horizontal IEEE Communications Magazine ' December 1997 97 extrinsic LLRs yields, the improvement 'n Table 4 (the extrin—
sic vertical terms are not yet being considered). 1.4 —1.4 .401 [0.1 Table 4. Improved
LLRS dkle to
Leh<d The original LLR plus both the horizontal and vertical
extrinsic LLRs yields the improvement in Table 5. 1.5 —1.5 —1.5 1.1 Table 5. Improved
'VLLARS dlle toA _ ‘ Leh(d) + Lev(d)
‘ For this eXample, it is seen that the horizontal parity alone
yields the correct hard decisions out of the decoder, but with
very low confidence for data bits d3 and d4. After enhancing
the decoder LLRs with the vertical extrinsic LLRs, the new
LLR solutions have better confidence. Let us pursue one
additional horizontal and vertical decoding iteration to see if
there are any significant changes in the results. We again use
the relationships shown in Eqs. 26 through 29 and proceed
with the second horizontal calculations for Leh(d) using the new L(d) from the first vertical. calculations, shown in Eqs. 34
through 37. Lela? 1) = (0.1— 0.1) — 2.5. ; 0",: new L(d1) (38)
Le),(d\2) = (1.5 + 0.1) + 2.5 = —1.6'= new L(d2) (39)
L6,,(a3) : (0.3 + 1.0) + 2.0 = —1.3 = new L(d3) 7 (40)
Leh(d4) = (0.2 — 1.4) +‘ 2.0 = 1.2 = new L(d4) (41) Next, we proceed With the second vertical calculations for ‘
Lev(d ) using the new L(d) from the second horizontal calcula— . ' tions, shown in Eqs: 38 through 41. V Lao.) _= (0.2 4 1.3,) + 6.0: 1.1,: new L(d1) (42) 
Lao?» = (0.3+ 1.2) I31 10 = —1.0 = new L(_d2) (43)
L433) = (1.5 + 0) m 6.0 4 —1.5 = new L(d3) (44) twain = (0.1 41.6)4 1.0 =' 1,0 = new L(d4) (45) After the second iteration of horizontal and vertical deOOd
‘ ing calculations, the soft—output likelihood values are again
calculated from Eq. 14, rewritten below: ‘ _ L0?) = Lee) + Lara?) +’ are?) ‘ (46) V The final horizontal and vertical extrinsic LLRs, and the
resulting decoder LLRs are displayed in Tables 6—8. For
this example, the second full iteration of both codes (yield
ing a total of 4 iterations) suggests .a modest improvement
over one full iteration. Now, there seems to be a balancing
of the confidence values among each of the four data deci
s1ons. ~ The soft output is shown in Table 9. 2.6 —2.5 —1 —2.6 2.5 Table 9. L(d )A=
L0(X) + Liah(d)
+ Lev(d). COMPONENT CODES FOR
IMPLEMENTING TURBO CODES We have described the basic concepts of concatenation,
iteration, and soft decisions using a simple productcode
model. We next apply these ideas to the implementation of
turbo codes. We form a turbo Code by the parallel concatena—
tion Of component codes (building blocks) [3, '6]. First, a reminder of a simple binary rate—1/2 convolutional
encoder with constraint length K and memory K — 1. The
input to the encoder at time k is a bit dk, and the correspond.—
ing codeword is the bit pair (uk, 11k), where ‘ K—l We 2 Zglidk—i m0d2 gli =0,1 (47)
i=0»
K—l Vk = ZgZidk—i m0d2 g21?0,1 (48)
i=0 G1 = {gli} and G2 = {572,} are the code generators and dk
is represented as a binary digit. This encoder has a finite
impulse response (FIR), and gives rise to the familiar non
systematic convolutional code (NSC), an example of which is
seen'in Fig. 5. In this example, the constraint length is K =
3, and the two code generators are described by G1 ={111}
and G2 = {101}. It is well known that, at large Eb/NO, the
error performance of an NSC is better than that Of a system—
atic code with the same memory.'At small Eb/No, it is gener—
ally the other way around [3]. A class of infinite impulse
response (HR) convolutional codes has been proposed [3]
for the building blocks Of a turbo code. Such codes are also
referr‘edto as recursive systematic convolutional (RSC) codes
because previously encoded information bits are continually
fed back to the encoder’s input. For high code rates,:RSC
codes result in better error performance than [the best NSC
codes at any Eb/NO. A binary rate—1/2 RSC code is obtained
from an NSC code by using a feedback 100p, and settingfone
of the two outputs (uk or Vk) equal tO dk. Figure 6 illustrates
an example of such an RSC code, with K = 3, where ink is
recursively calculated as ' ' ' ‘ K—1
ak = dk + Zgiak—i
i=1
and where g,’ is respectively equal tO gli if uk =. dk, and to g2, if
vk = dk. I We assume that the input bit dk takes on values of 0
or 1 with equal probability. It is stated in [3] that ak
exhibits the same statistical properties as dk. The trellis
structure is identical for the RSC code of Fig. 6 and the
NSC code Of Fig. 5, and these two codes have the same
free distance. However, the two output sequences {uk} mod 2 (49) * and {12k} do not correspond to the same input sequence {dk} for RSC and NSC codes. For the same code genera—
tors, we can say that RSC codes do not modify the out
put weight distribution Of the output codewords
compared to NSC codes. They only change the mapping between input data sequences and output codeword
sequences. 1.5 0.1 ' 0 i .—1.6 1.1 —1.0
0.2 _‘ 0.3 ' 41.3 1.2‘ —1.5 1.0
Table 6. Original Table 7. Leh(d) Table 8. LCVG)
.Lc(x) after second norl . after second verti—
measuremenls. ‘zonml decoding. cal decoding.
93 ' IEEE Communications Magazine 0 December 1997 CONCATENATION OF i
RSC CODES j Consider the parallel concatenation
of two RSC encoders of the type
shown in Figure 6. Good turbo codes
have been constructed with their compo nent codes having quite short constraint “k words from one encoder with low‘
weight codewords from the other
encoder. Many such pairings can be
avoided by proper design of the inter‘
leaver. An interleaver that permutes
the data in a random fashion provides
vk better performance than the familiar
J block interleaver [8]. lengths (K = 3 to 5). An example of a
such a turbo encoder is shown in Fig. 7,
where the switch yielding vk punctures
the code, making it rate 1/2. Without
the switch, the code would be rate 1/3. Additional concate—
nations can be continued with multiple component codes. In
general, the component encoders need not be identical with
regard to constraint length and rate. In designing turbo
codes, the goal is to choose the best component codes by
maximizing the effective free distance of the code [7]. At
large values of Eb/No, this is tantamount to maximizing the
minimum weight codeword. However, at low values of Eb/No
(the region of greatest interest) optimizing the weight distri
bution of the codewords is more important than maximizing
the minimum weight [6]. The turbo encoder in Fig. 7 produces codewords from
each of two component encoders. The weight distribution
for the codewords out of this parallel concatenation depends
on how the codewords from one of the component encoders
are combined with codewords from the other encoder(s).
Intuitively, we would like to avoid pairing lowweight code C2
RSC code InFi—gure 7. Parallel ~corzcatenationof RSC encoders.  I Figure 5. Nonsystenttztic convolu
tional (NSC) code. If the encoders are not recursive, a
lowweight codeword generated by the
input sequence at = (O 00 0 1 0 0 0 0) with a single binary 1 will always
appear again at the input of the second encoder for any
Choice of interleaver. In other words, the interleaver would
not inﬂuence the codeword weight distribution if the codes
were not recursive. However, if the component codes are
recursive, a weight—1 input sequence generates an IIR (infi—
nite weight output). Therefore, for the case of recursive
codes, the weight—1 input sequence does not yield the mini—
mum weight codeword out of the encoder. The encoded
output weight is kept finite only by trellis termination, a
process that forces the coded sequence to terminate in the
zero state. In effect, the convolutional code is converted to
a block code. For the Fig. 7 encoder, the minimum weight codeword for
each component encoder is generated by the weight3 input
sequence (0 0 0 0 1 1 1 0 0 0 0 0) with three consecutive
1s. Another input that produces fairly lowweight codeWords is
the weight2 sequence (0 0 0 0 1 0 0 1 0 0 0 0). However,
after the permutations of an interleaver, either of these dele—
terious input patterns is not likely to appear again at the input
to another encoder, so it is unlikely that a minimum weight
codeword will be combined with another minimum weight
codeword. The important aspect of the building blocks used in
turbo codes is that they are recursive (the systematic aspect is
merely incidental). It is the RSC code’s IIR property that pro
tects against those lowweight encodings which cannot be
remedied by an interleaver. One can argue that turbo code
performance is determined largely from minimum weight
codewords that result from the weight—2 input sequence. The
argument is that weight—1 inputs can be ignored since they
yield large weights due to the IIR code structure, and for
input sequences having weight3 and larger, a properly
designed interleaver makes the number of such input words
relatively rare [7—11]. A FEEDBACK DECODER The Viterbi algorithm (VA) is an optimal decoding method
for minimizing the probability of sequence error. Unfortu
nately, the VA is not able to yield the APP or softdecision
output for each decoded bit. A relevant algorithm for doing
this has been proposed by Bahl et al. [12]. The Bahl algorithm
was modified by Berrou et al. [3] for use in decoding RSC
codes. The APP of a decoded data bit dk can be derived from
the joint probability A,§(m) defined by Adm) = Pidk = 1', Sk = m IRfV} (50) where S k is the state at time k, and R IN is a received sequence
from time k = 1 through some time N. Thus, the APP for a decoded data bit dk, represented as a
binary digit, is equal to P{dk =iR1N}= 2N, (m) , 1': 0,1 . (51) The loglikelihood function is written as the logarithm of
the ratio of APPS, as follows: IEEE Communications Magazine  December 1997 99 __Foed back loop Z [ L167 k) Lﬁd n) JDeimep 1:42“? N Fig. 8. For a discrete memoryless Gaus
ki _ _ _ __J Eaving sian channel and binary modulation, the
i a; Decoder tater it Decoder ' : decoder input is made up of a couple Rk
I ___ i DEC1 leaving. DECZ F' D , t L2(3 k) of two random variables xk and yk.
k _ _ __ _[ —— ' ""1 ———— [egcisr’l~ Where dk and vk are bits, we can express
Fiﬁ—i y y ———g~ _:__ the received bit—to—bipolar pulse conver—
t?“ 1k 2k _:J sion at time k as follows:
I Yk I _— XI.» = (261k  1) + ik (54)
v .
Demux ‘ Yk = (2V1: — 1) + 61k (55) Decoded output If
y ' where ik and qk are two independent nois—
( k es with the same variance 62. The redun !_ i_  1 .. .
o 1 2 3 4 5 Eb/Nowsi‘ I . . . . . .. . . _
I Figure 9. Bit error probability as a function of Eb/No and mul
tiple iterations. (52) The decoder can make a decision by comparing L(c/l\k) to a
zero threshold. 3k=i if Lain >0 dk = 0 if L(dk) < 0 For a systematic code, the LLR L(d\k) assoeiated with each
decoded bit dk can be described as the sum of the LLR of (1k
out of the detector and of other LLRs generated by the
decoder (extrinsic information), as was expressed in Eqs. 12
and 13. Consider the detection of a noisy data sequence that
stems from the encoder of Fig. 7. The decoder is shown in dant information, yk, is demultiplexed
and sent to decoder DEC1 as ylk when vk
: vlk, and to decoder DECZ as M when
vk = 13%. When the redundant informa
tion of a given encoder (C1 OR C2) is not emitted, the corre—
sponding decoder input is set to zero. Note that the output of
DEC1 has an interleaver structure identical to the one used at
the transmitter between the two encoders. This is because the
information processed by DEC1 is the noninterleaved output of
C1 (corrupted by channel noise). Conversely, the information
processed by DECZ is the noisy output of C2 whose input is the
same data going into C1, but permuted by the interleaver.
DECZ makes use of the DEC1 output, provided this output is
time ordered in the same way as the input to C2. DECODING WITH A
FEEDBACK LOOP We rewrite Eq. 13 for the soft decision output at time k,
with the a priori LLR L(dk) initially set to zero. This follows from the assumption that the data bits are equally
likely. L(&k) = Lc(xk)+Le(tik) = 10g[%]+n (56) where L((/1\k) is the soft decision output, and Lc(xk) is the LLR
channel measurement, stemming from the ratio of likelihood
functionspggkldk = i) of the discrete memoryless channel.
Le(dk) = L(dk) ka = 0 is a function of the redundant informa
tion. It is the extrinsic information supplied by the decoder,
and does not depend on the decoder input xk. Ideally, Lc(xk)
and/\Le(dk) are corrupted by uncorrelated noise, and thus
Le(dk) may be used as a new observation of dk by another ‘
decoder for an iterative process. The fundamental principle for
feeding back information to another decoder is: never feed a
decoder with information that stems from itself (because the
input and output corruption will be highly correlated). For the Gaussian channel, we use the natural logarithm
in Eq. 56 to describe the channel LLR, Lc(xk), as was done in Eqs. 18 through 20. We rewrite the Eq. 20 LLR result
below: 2 2 .
V l xk —1 l xk +1 2
L — + — ,
C(Xk) 2[ 60 J 2[ 60 j 03 a (57)
Both decoders DEC1 and DAECZ use the modified Bahl
algorithm [6]. If the inputs L1(dk) and ka toAdecoder DECZ are independent, then the log—likelihood L 2(d k) at the output
of DEC2 can be written as Lactic) =ﬂLzrﬁk>i + Lac?» (58)
with 100 IEEE Communications Magazine  December 1997 A 2 A
L1 (dk) = 735k +Lel(dk) (59)
00 wheref[ ] indicates/\a functional relationship. The extrinsic
information kegajk) out of DBCZ is a function Of the
seguence {L1(d,,)},, ,é k. Since L1(c/i\,,) depends on observation
R1 , the extrinsic information L62(dk) is correlated with Obser—
vations xk and ylk. Nevertheless, the greater n — k] is, the
less correlated are L1(dn) and the observations xk, yk. Thus,
due to the interleaving between DECl and DECZ, the extrin
sic information Lez(dk) and the observations xk, ylk are weak
ly correlated. Therefore, they can be jointly used for the
decoding of bit dk. 2k = L62(dk) acts as a diversity effect in an iterative pro
cess. In ger/teral, L62(dk) will have the same sign as dk. There
fore, LeZ(dk) may improve the LLR associated with each
decoded data bit. A The algorithmic details for computing the LLR, L(dk), of
the APP for each bit has been described by several authors
[3—5], and suggestions for decreasing the implementational
complexity are still ongoing [13—15]. A reasonable way to
think of the process that produces APP values for each bit is
to imagine computing an MLSE or a VA in two directions
over the block of coded bits. Having metrics associated with
states in the forward and backward direction allows us to
compute the APP of a bit if there is a transition between two
given states. Thus, we can proceed with this bidirectional VA
and, in a slidingwindow fashion, compute the APP for each
code bit in the block. With this View in mind, wecan estimate
that the complexity of decoding a turbo code is at least two
times more complex than decoding one of its component
codes using the VA. TURBO CODE ERROR
PERFORMANCE EXAMPLE onte Carlo simulations have been presented for a rate1/2, K = 5 encoder with generators G1 = {11111} and G2 =
{10001}, and parallel concatenation. The interleaver was a 256
x 256 array. The modified Bahl algorithm has been used with a
data block length of 65,536 bits [3]. For 18 iterations, the bit
error probability PB is lower than 10'5 at Eb/NO = 0.7 dB. The
error performance improvement as a function of iterations is
seen in Fig. 9. Note that, as we approach the Shannon limit of
—1.6 dB, the required bandwidth approaches infinity, and the
capacity (code rate) approaches zero. For binary modulation,
several authors use P3 = 10‘5 and Eb/NO = 0 dB as the Shan
non limit reference for a rate1/2 code. Thus, with parallel con
catenation of RSC convolutional codes and feedback decoding,
the error performance of a turbo code is at 0.7 dB from the
Shannon limit. Recently, a class of codes that use serial instead
of parallel concatenation of the interleaved building blocks
have been proposed. It has been suggested that these codes
may have superior performance to turbo codes [14]. SUMMARY In this article basic statistical measures, such as a posteriori
probability and likelihood, are reviewed. We then use these
measures for describing the error performance of a soft—
input/softoutput decoder. A numerical example helps to illustrate how performance is improved when soft outputs
from concatenated decoders are used in an iterative decoding
process. We next proceed to apply these concepts to the par
allel concatenation of recursive systematic convolutional
(RSC) codes, and explain why such codes are the preferred
building blocks in turbo codes. A feedback decoder is
described in general ways, and its remarkable performance
presented. REFERENCES [1] G. D. Forney, Jr., Concatenated Codes, Cambridge, MA: MIT.
Press, 1966. [2] J. H. Yuen, et al., "Modulation and Coding for Satellite and Space
Communications," Proc. IEEE, vol. 78, no. 7, July 1990, pp.
1250—65. [3] C. Berrou, A. Glavieux, and P. Thitimajshima, “Near Shannon Limit Error
Correcting Coding and Decoding: Turbo Codes," IEEE Proc. ICC ’93,
Geneva, Switzerland, May 1993, pp. 1064—1070. [4] C. Berrou and A. Glavieux, "Near Optimum Error Correcting Coding and
Decoding: TurboCodes," IEEE Trans. Commun., vol. 44, no. 10, Oct.
1996, pp. 126141. [5] J. Hagenauer, "Iterative Decoding of Binary Block and Convolutional
Codes," IEEE Trans. Info. Theory, vol. 42, no. 2, Mar. 1996, pp.
429—45. [6] D. Divsalar and F. Pollara, “On the Design of Turbo Codes," TDA
progress rep. 42123, Jet Propulsion Lab., Pasadena, CA, Nov. 15, 1995,
pp. 99—121. [7] D. Divsalar and R. J. McEliece, "Effective Free Distance of Turbo Codes,"
Elect. Lett., vol. 32, no. 5, Feb. 29, 1996, pp. 445—46. [8] S. Dolihar and D. Divsalar, "Weight Distributions for Turbo Codes Using
Random and Nonrandom Permutations," TDA progress rep. 42—122, Jet
Propulsion Lab., Pasadena, CA, Aug. 15, 1995, pp. 56—65. [9] D. Divsalar and F. Pollara, "Turbo Codes for Deep~Space Communica
tions," TDA progress rep. 42120, Jet Propulsion Lab., Pasadena, CA,
Feb. 15, 1995, pp. 2939. [10] D. Divsalar and F. Pollara, "Multiple Turbo Codes for DeepSpace Com
munications," TDA progress rep. 42121, Jet Propulsion Lab., Pasadena,
CA, May 15, 1995, pp. 66—77. [11] D. Divsalar and F. Pollara, "Turbo Codes for PCS Applications,” Proc.
ICC ’95, Seattle, WA, June 18—22, 1995. [12] L. R. Bahl et al., "Optimal Decoding of Linear Codes for Minimizing
Symbol Error Rate,” Trans. Info. Theory, vol. IT—20, Mar. 1974, pp.
248—87. [13] S. Benedetto et. al., "Soft Output Decoding Algorithm in Iterative
Decoding of Turbo Codes,” TDA progress rep. 42424, Jet Propulsion
Lab., Pasadena, CA, Feb. 15, 1996, pp. 63—87. [14] S. Benedetto et al., "A SoftInput SoftOutput Maximum A Posteriori
(MAP) Module to Decode Parallel and Serial Concatenated Codes," TDA
progress rep. 42127, Jet Propulsion Lab., Pasadena, CA, NOV. 15,
1996, pp. 63—87. [15] S. Benedetto et al., “A SoftInput Soft—Output APP Module for Iterative
Decoding of Concatenated Codes," IEEE Commun. Lett, VO.1, no. 1,
Jan. 1997, pp. 22—24. BIOGRAPHY BERNARD SKLAR [LSM] has over 40 years of experience in technical design
and management positions at Republic Aviation Corp., Hughes Aircraft, Lit
ton Industries, and The Aerospace Corporation. At Aerospace, he helped
develop the MILSTAR satellite system, and was the principal architect for
EHF Satellite Data Link Standards. Currently, he is head of advanced sys
tems at Communications Engineering Services, a consulting company he
founded in 1984. He has taught engineering courses at several universities,
including the University of California, Los Angeles and the University of
Southern California, and has presented numerous training programs
throughout the world. He has published and presented scores of technical
papers. He is the recipient of the 1984 Prize Paper Award from the IEEE
Communications Society for his tutorial series on digital communications,
and he is the author of the book Digital Communications (Prentice—Hall). He
is past chair of the Los Angeles Council IEEE Education Committee. His aca
demic credentials include a 8.5. degree in math and science from the Uni
versity of Michigan, an MS. degree in electrical engineering from the
Polytechnic Institute of Brooklyn, New York, and a PHD. degree in engi
neering from the University of California, LOS Angeles. IEEE Communications Magazine ' December 1997 101 APPENDIX A L(d1®d2)=loge Below are presented the algebraic details yielding the results P (‘11 : +1)><P(d2 ‘ 1) I
shown in Eq. 22, rewritten as follows: P(dl :+1)X p(d2 +1) [I P( d1 _ 1 MD P(d2  11)] A eL(dl) + eL(d2)
L(d1) IE L012) "U611 (9 d2) =10ge W (60) L(d1) 1 1 L(d2)
> l+e I e 2 e + e
(d ) PM 1) 1 1+eL(dl) [1+eL(d2)] (1+eL(dl)] 1+6le2)
_ p :+l~ : :+ _ egg 1 v 2 58
L(d)_10g“IP(d=—1)I logtlifpu=11>I (61) ELM 8w ti 1 1 I ( )
l+eL(d1) 1+ W2) 1+eLId‘) 1+eLId2)
eud) : P(d=+l) I
1— P(d = +1) (62) 6W1)+ eLtdz)
ele) —eL(d) xP(d = +1) = P(d = +1) (63) [mm] mm]
ele) = P(d = +1) X [1 + eLl‘m (64) =loge (69)
L(d) eLId‘)eL(d2) +1 
~P d: 1 =——e 65 ‘ —r‘—*“—’
( 9 new» ( ) [law] [HM]
. 6L(d) _ 1 66 ‘ 1 P(d—I1)—1 1+6L(d)—1+6L(d) ( ) _10 eL(dI)+eL(d2) _ ’ 3‘3 ﬂlﬁLtdli'e‘Li—d.)
Call for Papers.
IEEE Network Magazine
Special Issue on Network Management:Today and Tomorrow
 Guest Editor — Roch H. Glitho
Ericsson Research Canada, Town of Mount Royal, Quebec  Canada HAP 2N2
Tel: +I5I4—345 7900 — Extension: 2266; Fax: +I—5I4345 oIO5
Email: [email protected]
Scope Network management is an emerging discipline and a vibrant area of research. Industry and academia interest is growing. This interest is
clearly demonstrated by activities of network management related standards bodies and industry fora, conferences focusing on network man
agement, and last but not least, development and deployment of network management products. Most of today's network management sys
tems follow a manager/agent approach. Managers located on dedicated nodes retrieve raw data collected by agents which are usually
embedded in the nodes tobe managed. The data is retrieved using dedicated management protocols. The lETF simple network management
protocol (SNMP) and the lSO/ITUT common management information protocol (CMIP) are today’s classical management protocols. Today’s
manager/agent approach is being challenged and new approaches are emerging for tomorrow's network management systems. These new
approaches include the peerto—peer approach stipulated by distribution technologies such as CORBA; the use at ubiquitous technologies such
as the World Wide Web; and the delegation of classical manager tasks to agents via the use of mobile and/or intelligent agents. Today's
management systems must be able interoperate among themselves. The same applies to tomorrow’s systems. Moreover, tomorrow’s manage
ment systems, whenever in place, must be able to interoperate with today’s systems, which will be yesterday’s systems of the future. Interoper
abilitywill remain the key issue in network management. This special issue of IEEE Networkwill presentthe presentand future ofnetwork management.
High quality tutorialoriented references are solicited. Topics of interest include: 0 lETF Simple Network Management Protocol (SNMP) 0 Webbased network management
0 ISO/ITUT OSI systems management ' Mobile agentsbased network management " ITUT Telecommunication Management Network (TMN) 0 Intelligent agents—basednetwork management
0 CORBA~based network management 0 Interoperability frameworks Instructions to authors Authors are invited to submit four (4) hard copies of their manuscripts to the guest editor. Papers should not exceed 20 doublespaced pages in length, excluding figures. More information for potential authors is available at the IEEE Network home page: http://www.comsoc.org/soc
str/techcom/ntwrk/ Schedule 0 Paper submission deadline: March I5, I998 ' Notification of acceptance: May 30, I998 0 Final manuscripts due: June 30, I998
0 Publication date: September/October I998 ' 102 IEEE Communications Magazine ' December 1997 ...
View
Full
Document
This note was uploaded on 03/20/2012 for the course EEL 6650 taught by Professor Shea during the Spring '12 term at University of Florida.
 Spring '12
 Shea

Click to edit the document details