This preview shows pages 1–5. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: A Comparison of Optimal and SubOptimal MAP Decoding
Algorithms Operating in the Log Domain Patrick Robertson, Emmanuelle Villebrun and Peter lloeher Institute for Communications Technology, German Aerospace Research Establishment (DLR)
D—82230 Oberpfaifenhofen, Germany, Tel.: ++49 8153 28 2808; Email: Patrick.R0berts0n©dlrde Abstract For estimating the states or outputs of a Markov process, the
symbolby—symbol MAP algorithm is optimal. However, this al
gorithm, even in its recursive form, poses technical difﬁculties be—
cause of numerical representation problems, the necessity of non
linear functions and a high number of additions and multiplica—
tions. MAP like algorithms operating in the logarithmic domain
presented in the past solve the numerical problem and reduce the
computational complexity, but are suboptimal especially at low
SNR (a common example is the MaxeLog—MAP because of its use
of the max function). A further simpliﬁcation yields the softoutput
Viterbi algorithm (SOVA). In this paper, we present a Log—MAP algorithm that avoids the ap—
proximations in the Max—LogMAP algorithm and hence is equiv
alent to the true MAP, but without its major disadvantages. We
compare the (Log—)MAP, MaxLogMAP and SOVA from a theo
retical point of View to illuminate their commonalities and differ—
ences. As a practical example forming the basis for simulations,
we consider Turbo decoding, where recursive systematic convolu—
tional component codes are decoded with the three algorithms,
and we also demonstrate the practical suitability of the Log—MAP
by including quantization effects. The SOVA is, at 10—4, approxi—
mately 0.7 dB inferior to the (Log—)MAP, the Max—Log—MAP lying
roughly in between. We also present some complexity comparisons
and conclude that the three algorithms increase in complexity in
the order of their optimality. 1 Introduction We will consider trellis—based softoutput decoding algorithms
delivering additional reliability information together with
hard decisions. The Bahl—Jelinek algorithm, also known as the
symbolhy—symbol MAP algorithm (MAP algorithm for short),
is optimal for estimating the states or outputs of a Markov
process observed in white noise However, this algorithm is
perhaps too difﬁcult in practice, basically because of the nu
merical representation of probabilities, non—linear functions
and because of mixed multiplications and additions of these
values. Some approximations of the MAP algorithm have been de—
rived, such as the softoutput Viterbi algorithm (SOVA) [2]
and the MaeLog—MAP algorithm [3, 4, 5]. In both algo—
rithms, processing is exclusively in the logarithmic domain;
values and operations (addition and max—function) are easier
to handle. However, both algorithms are suboptimal at low
signal—to—noise ratios, where we use TurboeCodes [6, 7, 8, 9],
for example. In this paper, we will modify the Max~l.og—MAP algorithm
through the use of a simple correction function at each max— 0—78032486—2/95 $4.00 © 1995 IEEE operation This algorithm, to be called LogMAP algo—
rithm, is equivalent to the MAP algorithm in terms of perfore
mance, but without its problems of implementation. The cor—
rection needs just an additional onedimensional table look—up
and an addition per max—operation. The organization of the paper is as follows: After reviewing
the MAP and Max—lLog—MAP algorithms, we will derive the
LogeMAP algorithm in Section 2. In Section 3, we will com
pare these algorithms with the VA and the SOVA. Complexity
comparisons and quantization issues are covered in Section 4.
Finally, numerical results are presented in Section 5 by ap
plying the addressed algorithms in a Turbo—Code system. 2 Deﬁnition of the MAP, MaxLog
MAP, and LogMAP Algorithms 2.1 The Encoder and Notation Since we will study the behaviour of the (Max—)Log—MAP al
gorithm applied to the decoding of convolutional codes (and
in particular recursive systematic convolutional (RSC) codes),
we will choose a notation that complies with such an en
coder, for example one with four memory elements as shown in
Fig. 1. Since the MAP algorithm is essentially block—oriented,
we shall represent the binary input (information) data se— quence by (i: (d1, ..., dN). :l ﬂ _
Puncturing _j Xp Figure 1: Two identical recursive systematic convolutional
encoders employed in a Turbo coding scheme. The encoder has M memory elements. In our example
(the RSC code is of rate 1/2) it has two outputs, one is 1009 the sequence of information bits 233 : (xi, : J, since
the encoder is systematic. The other is the ‘parity informa— . . M
tion’ sequence 573 : (a:§,...,:c§v), With (131}: : Zizo gifak_za ,gfw) is the feed—forward generator and ak : where (g5, . .. M . . .
dk®Zi21gfak_i;s1mIlarly, (g3... erator of the encoder. The sequences is and :31" may be punctured, are then mod—
ulated and transmitted over a channel. In this work, we have
assumed BPSK modulation and an AWGN channel with one—
sided noise power spectral density No. The corresponding
received sequences be g” and For brevity, the sequence
3;? : (373,32?) will refer to the sequence of pairs of received systematic and parity symbols. , 9514) is the feedback gen— 2.2 The MAP Algorithm We will not repeat the derivation of the MAP algorithm1 but
only state the results. For more detail see [1, 6, 8]. Let the
state of the encoder at time k be Sk, it can take on values
between 0 and 2M — 1. The bit dk is associated with the tran
sition from step k — l to step k. Goal of the MAP algorithm
is to provide us with the logarithm of the ratio of the a pos—
teriori probability (APP) of each information bit dk being 1
to the APP of it being 0. We obtain: ;52 71(yk,Sk—1,Sk)'Otk—1(Sk—1)‘ﬁk(5k)
Z Z 70(9):: Sic—1,50 'ak—itsk—i) 'ﬁkt’SkV Sk Sk—l
. (1)
where the forward recurs10n of the MAP can be expressed as: A(dk) : ln 1
Z Z 717(yk;Sk—1;Sk) ' ale—1(Slc—1) Sk_1i:0 1 .
E E wakﬁk—nsklak—1(5k—1)
5k 5k_1i:0 1 f S :0
wow = {o :1; 0 and the backward recursion as: 01451) — ; (2) 1
Z Z 7/r(yk+1,5k, SicH) 'ﬁk+i(5k+1) ﬂktsk) : Skﬂizol ;
Z Z :7i(yk+135k:S/c+1) 'ak(5k)
5k 5H1 i=0
'1 1 : v
mm = {0 £th 0 e The branch transition probabilities aregiven by 7i((yiayfl,Sk—i,5k) = (1(dk = ilskask—l)‘ Z i) ﬂyildk : 2',Sk,5k_1) P7’{Sklskn1}. (4) The value of q(d;c : ilSk,Sk_1) is either one or zero de—
pending on whether bit 1' is associated With the transition
from state Sk_1 to Sk or not. It is in the last component
that we use a priori information for bit dk [10]: In our case
of no parallel transitions, Pr{SkISk_1} : Pr{dk : 1} if
q(dk 1: 1Sk,Sk_1) I 1; and P1”{Sk5k_1}1 P1’{dk Z 0} if
(1011: = Olskzsk—l) = 1 2.3 The lVIax—LogMAP Algorithm As we already said in the introduction, the MAP algorithm
is likely to be considered too complex for implementation in
a real system To avoid the number of complicated opera—
tions and also number representation problems, one does not calculate way?“ yi), Shh 3k), ak(Sk) and ﬂk(Skl any longer.
One computes and works with the logarithms of these values instead [5]. By taking the logarithm of 7i((y,:, yg), Sk_1, Sh) derived in
(4) and inserting 1 (ﬁgwgwxgoshshaﬂ (5) «mo ' ' we obtain the following expression for : 1: :iisvk)SVk—1): 218:1:3 ln7i((y29y£):Sk—15Sk) : ink—{34)
QyZI11:(i,Sk,S/C_1) N 0 + +111 Pr{SkISk_1} + K. (6) We can ignore the constant K, indeed it cancels out in the calculation of ln ak(Sk) and in B14516). We must remember
that NC must be estimated to correctly weight the channel information with the a priori probability Pr{S;c Sk_1}.
For ln ak(5k), we get: 1
1n ak(Sk) : ln( 2: Z eln7:((yiyy:),5k—175k)+ln ak—1(5k—1)) Sk_1i:0 1
m]n(Z Z Z:cln71((yiyy:)vSk—lysk)+lnak—lfsk—ll). (7) Sk Sk—l 1:0 To obtain a simple solution, we use the following approxima
tion: ln(e<51 + + 65") 2 max 61;
i€{l..n} (8) maxi E 61 can be calculated by successively using n — 1 maximum functions over only two Values. From now on, we deﬁne Tittyiayg):sk_—1;Skl : 1n7i((yZ>y£)7Sk—1:Sk)a
&k(:5k) : 1n ak(Sk) and ﬁk(5k) : lnﬁk(Sk). We eventually obtain: @ktsk) % (EinJWinZV 911;), Sic—1; Sic) + (it—1(Sk—1» — max , (Ti(:(yi;:yZ);Sk—1;Sk) + @k—ltsk—1»;
(5k,Sk—1,1) (9)
and similarly, ﬁllets/c) 3 max (?i((yi+11y§;+1)7 Sky Sk+1) + Bk+1lsk+1>> (Sklli'l) (7z((yi+1,y£+1), 5k, Sk+1) + aktskll (10) "" max
(5): 151mm 17;) The second terms are a consequence of the derivation from
(2) and (3); they are needed for numerical reasons. Omitting
them has no effect on the value of the output of the Max—Log— MAP algorithm, since these normalization terms will cancel
out in (11). 1010 In the same way we can give an approximation of the log—
likelihood reliability of each bit dk:
A(dk) z (11)
max (11(31):, Sic—1, 5k) + ale—1(Sk—1) + 5k(5k)) — (Ski'sk—l) (Smgx )(i‘otyk, SIC—17 Sic) + Eric—1(Sk—i) + kv k—l To be used in a Turbo decoder, we want the output of the
MaxLog—MAP algorithm, A(dk), to be split into three terms
(extrinsic, a priori and systematic components) as shown in
[6, 7, 8]; it can easily be shown that this is possible here.
The extrinsic component will be used as a priori informa
tion in the next decoding stage. This a priori log—likelihood
ratio (LHL) for bit dk is called L(dk) for short We need to determine the a priori information in If q(d1C =
113k,Sk_1) 2 1, then _ Pr{dk :1} _ Pr{Sle _ }
it (m) a 02> hence ln P7‘{SkSk_1} : L(dk) — ln(l + eL(d’°)). An approxi
mation for P1'{SkSk_1} can be easily found using (8): lnPr{Slek_1}z L(dk) — max(0, L(dk)).
If q(dk : OISk, Sk_1) : 1 then
w P1'{dk :1} _ 1—PT{Slek_1}
“d” *1“ (m) *1“ (m) ’ “4) hence ln PT‘{Slek—1} = —ln(1 «1— end”). Similarly we can
approximate ln Pr{.5';c 5151} N —max(0, L(dk)). (13) 2.4 Correction of the Approximation: The
Log—MAP Algorithm Because of the approximation (8) we applied, the MaxLog
MAP algorithm is suboptimal and yields an inferior soft—
output than the MAP algorithm. The problem is to exactly
calculate ln(exp 61 + + exp 6”). This problem can be solved
by using the Jacobian logarithm [3, 4]: ln(e‘” + 6"?) = max(61,62)+1n(1+ e"l62‘61l) max(61,62)+ fc(i51 ~52I), (15) where is a correction function. Let us now prove re—
cursively that the expression ln(exp 61 + + exp 5”) can be
computed exactly. The recursion is initialized with (15). Sup— pose that 6 : ln(el"1 + + can—1) is known. Hence, ln(e‘s1 + + 66")
= ln(A + 6'5") with A : 661 + + eén" : e."
: max(lnA, 5n) + lllA — (Sni)
: mam”) +fc(la — 6,,l) (16) When deriving the Log—MAP algorithm, we now augment all
maximizations over two values with the correction function.
As a consequence, by correcting at each step the approxi—
mation made by the Max—Log—MAP, we have preserved the
original MAP algorithm. By calculating fc(.), we lose some of the lower complexity of
the Max—LogMAP algorithm. That is why we approximate by a pre—computed table. Since the correction only de
pends on 152 — 61], this table is one dimensional. We shall see
that only very few values need to be stored. q.e.d. 3 Comparison of the (Max—)Log
MAP and SoftOutput Viterbi
Algorithms 3.1 Hard Decisions In [5], it was claimed that the hard decision of the Max—Log
MAP provides exactly the same hard decision as the Viterbi
algorithm. We now present a mathematical proof of this re
sult for our example of rate 1/2 RSC codes; extensions are
rudimentary. We assume for simplicity that the deﬁnitions
of (ik(Sk) and do not include the normalization term.
Remember that the Viterbi algorithm selects that path with
the largest metric: M: max N 1
— 3—:03 ath 2
vpaﬂls{2[ Nook up )> 7,15% — WW1:th+1n(Pv'{Slek—1})]} . (17) The Max—Log—MAP output (11) for the last bit LIN can be
written as A(dN) : M1(dN) — MU(dN), Where Mi(dN) = gnax {z(yN,SN1,SN = 10) + melanin. (18)
By applying the recursive deﬁnition of o‘zN_1(SN_1) (without
normalization) we obtain: Mi(dN) = maxi1i(yN,5N_1,5N = 0) + (19)
SNv—l (ijN_1(yN—1,SN_2,SN_1) + (EN—2(SN_2))}. max
(SN—2 YIN—1) There exists an SN_2,ma$ and an jN_;_),mw such that Miidiv) = {IIBX{‘/i(yN,SN—1,SN = 0) + N—l (20) ijN_1,mw(yN—1,SN—2,mam,5iv_i) + aN—2(SN—2,ma:c)}~ This is repeated N —— 2 times, yielding:
Mi(dN) = ﬂax {Ti(3/N:SN—1)SN = 0)} +
N—i (21) Viknnax (yNi Sk—1,maxi Samar),
111:1 since we assume ("10(0) 2 0. By making a decision for the bit
dN, the Max—Log—MAP algorithm selects the largest M¢(dN).
By inserting (6) into the maximum over 'i of (21) and com—
paring with (17), we can easily see that the Max—Log—MAP
and Viterbi algorithm make the same decision for bit dN, since
maxde{o,1}i\1g(dN): iM—i—C = M(dN), where C is a constant.
To continue, let us deﬁne a metric for the Max—Log—MAP as: M(dk) : GHQ/k, Sic1) 5k) + dk71(sk#1) + EMS)». (22)
The MaxLogMAP will choose that bit cl;c that maximizes
(22). We now suppose that ilk/(dig) :: .M + C. Let us ﬁrst
prove that M(dk) = M(dk_1): max
(SkySk—l 11') max. {:y’i(yk—1;Sk—2:Sk—1) +
(Sh—~2yl) Cindi—2)} + Bic—1(Sk—l)}v M(dk_1) = max{
Sic—.1 (23) 1011 We apply the recursive deﬁnitions of &k_1(S;¢_1) and
ﬂk_1(5k_1), so that we obtain: M(dk_1) = max 1" 15—15 7— 5—,
(Shh—112,){7tyk k 1 k)+ak 1( k 1)+ man} : M(d;,). (24) We can deduce from this recursion step and from the initial
step where k z N that Vk E {1, . . .,N} MMax—Log—MAP = MUN) : Midi) : M + C (25)
For each bit elk, the Max—LogMAP algorithm calculates two
Viterbi metrics and takes the largest one, This proves that
the Max—LogeMAP algorithm makes the same hard decisions
as the Viterbi algorithm. 3.2 Soft Outputs As we have already explained, the MaxeLog—MAP algorithm
and the SOVA work with the same metric. If we consider
only the hard decisions, they are identical. But, they behave
in diifcrent ways in computing the information returned about
the reliability of decoded bit dk. The SOVA considers only
one competing path per decoding step. That is to say, for
each bit dj it does not consider all the competing paths but
only the survivors of the Viterbi algorithm. To be taken into
account in the reliability estimation, a competing path must
join the path chosen by the Viterbi algorithm without being
eliminated. The differences between the (LongAP, MaxALogeMAP
and SOVA are illustrated in Fig. 2. The MAP takes all paths
into its calculation, but splits them into two sets: those that
have an information bit one at step j and those that have a
zero; it returns the LLB of these two sets. All that changes
from step to step, is the classiﬁcation of the paths into the
respective sets. Due to the Markov properties of the trellis,
the computation can be done relatively easily. In contrast,
the Max—Log—MAP looks at only two paths per step: the best
with bit zero and the best with bit one at transition j; it then
outputs the difference of the log—likelihoods, However, from
step to step one of these paths can change, but one will always be the maximumelikelihood (ML) path. The SOVA will al
ways correctly ﬁnd one of these two paths (the ML path), but
not necessarily the other, since it may have been eliminated
before merging with the ML path. There is no bias on the
SOVA output when compared to that of the Max—Log—MAP
algorithm, only the former will be more noisy. 4 Complexity and Quantization 4.1 As mentioned earlier, the correction function in (15‘) used by
the Log—MAP can be implemented using a look—up table. We
found that excellent results can be obtained with 8 stored
values and 61 —62 ranging between 0 and 5. No improvement
is achieved when using a ﬁner representation. We now present
the result of complexity analyses in the following table, taking
into account the additional complexity of including a priori
information: Complexity Comparisons all paths are
considered two paths are
considered two paths are
' considered but
the competing path
may not be the best
competing path V. ,
competing path /’; that now determines
reliability (will survive to
merge with ML path) Here path    —
is eliminated Figure 2: Comparison between (Log)MAP, Mavaog—MAP
and SOVA. The MAP uses all paths in the trellis to optimally
determine the reliability of bit dj. The MaxLog—MAP makes
its decision (and soft output) based on the best two paths with different dj. The SOVA also takes two paths, but not
necessarily both the same as for the MaxeLog—MAP. Operation l I Max—Log—MAP Log—MAP  SOVA
maxops 5x2M——2 5x2M—2 3(M+B+2M
additions 10 x 2M + 11 15 x 2V+9 2 x 2 +“8 "J
mult. by i] , 8 8 8
r bit comps 6(M + 1)
lookups 5 >( W— 2 If we assume that one bit comparison costs as much as one
addition, this table allows us to conclude that the Max—Log—
MAP algorithm is more than twice as complex as the SOVA, for memory 2% = 4 and less than 2 times as complex for
NI : 2. 4.2 Quantization We shall now present the quantization ranges that were used
in simulations of the LogMAP, they are based on observa—
tions of the distributions of the pertinent variables. We have
attempted to take into account the large variations that are
the result ofthe iterative decoding process in a Turbo decoder. Variable Number of levels Limit values
gag?  16 ‘i 1.83 2.16
L(dkj— 64  15.5 16.0 ?¢I,yk,5k,5k_1 256 I 15.875 16.0 MM 256  —15.875 L 16.0
ﬁrst) 256  —127.0 A 128.0
A103“ 256  731.75 32.0 1012 5 Simulation Results for Applica
tion in a Turbo—Code System Figure 3 shows the BER for decoding TurboCodes with the
MAP, Log—MAP, MaerogvMAP, and SOVA, respectively,
taking no quantization effects into account. However, the
Log—MAP is applied with an 8—Values correction table; a loss due to the 3 bit quantization of the correction table is not
visible. Results for the SOVA are taken from 0 10 I _1 9—0 MAP
10 MaxLog—MAP  <>—<> LogAMAP
1072 3K \\\A\ H SOVA “ 10'3 — \ \ — \ \
\A
E 10" ~ \ \\ 
m \
10'5 L \\ A
\ \ \s\\\ 10“ — \ E1
\6 10'7 — ‘
10’8 7,7,... . l . ...i..,,.i,M
1.0 1.5 2.0 2.5 Eb/No, dB Figure 3: BER for Turbo—decoding with MAP, Log—VIAP,
Max—Log—MAP and SOVA (8 iterations). N : 1024, It! : 4. Finally, Figure 4 shows the corresponding BER for the Log
MAP with and without quantization. We set NO to 2, and
used an 8 values table. The step—size of the correction table
is equal to 1/8. 10 T i
10" v 
10.2 b\ A
A
10.. >\\ 7
5 10" ~ «
d]
10'5 7 \‘ED
106 Le—oMAP \\ _
i I: LogMAP \f
107 ;<>——<>quamized LogMAP _
10‘8 J 1
1.0 1.5 2.0 2.5 Eb/NO, dB Figure 4: BER for Turbo—decoding with MAP, L0g~MAP, and
quantized Log—MAP (8 iterations). 6 Conclusions We have demonstrated a Log—MAP algorithm that is equiv—
alent to the (true) symbol—bysymbol MAP algorithm, i.e.,
is optimum for estimating the states or outputs of a Markov
process However, the novel implementation works exclusively
in the logarithmic domain, thus avoiding the basic problems
of the symbol—by—symbol MAP algorithm. The difference be—
tween our LogMAP algorithm and the known Max—Log—MAP
algorithm, which just approximates the NIAP algorithm, is the substitution of logarithms by the Jacobian logarithm.
This correction is simple to implement and quite insensitive
to a quantization; a loss due to quantization of the correc—
tion function is not visible. Further implementation issues
concerning continuous data (“real—time implementation”) and
simpliﬁcations valid for feedforward trellises are still ap—
plicable. We have compared the MAP, (Max—)Log—MAP and
SOVA from a theoretical point of View to illuminate their
commonalities and differences. As a practical example form—
ing the basis for simulations, we considered Turbo—decoding,
where recursive systematic convolutional (RSC) component
codes have been decoded with the three algorithms. Quanti—
zation of the whole LogMAP algorithm was also investigated:
the loss is about 0.5 dB. This can probably be improved, but
Turbo—Codes have a huge variation of variables” ranges as a
result of iterative decoding. Finally, we have compared the
complexity of the three algorithms. The number of operations
of the Log—MAP is about twice the number of operations of
the SOVA, however, the former is more suited to parallel pro—
cessing We conclude that the Log—MAP is particularly suitable for
decoding TurboCodes, since in this challenging application
we have a very low signal—to—noise ratio but a small number ofstatcs, so that the additional complexity is less pronounced. Acknowledgements The authors would especially like to thank Dr. J. Hagenauer
and Dr. J. Huber for valuable discussions and comments. References [1_ L. Bahl, J. Cocke, F. Jelinek, and J. Raviv, “Optimal decoding
of linear codes for minimizing symbol error rate,” IEEE Trans.
Inform. Theory, vol. lT—ZD, pp. 284r287, March 1974. 2. J. Ha enauer and P. Hoeher, “A Viterbi al orithm with soft
. g o n n u  g decision outputs and its applications,” in Proc. GLOBECOM
’89, pp. 1680—1686, November 1989. [3 J. A. Erfanian, S. Pasupathy, and G. Gulak, “Reduced com— plexity symbol detectors with parallel structures for isi chan—
nels,” IEEE Trans. Commun., vol. 42, pp. 166171671, Febru— ary/March/April 1994.
[4 W. Koch and A. Baier, “Optimum and sub—optimum detec tion of coded data disturbed by timeivarying intersymbol in
terference,” in Proc. GLOBECOM ’90, pp. 167971684, De— cember 1990. [5' J. Petersen, “lmplementierungsaspekte zur Symbolby
Symbol MAP Decodierung von Faltungscodes,:’ in Proc.
ITG Tagung, Codierung fn'r Quelle, Kanal und Ubertragung,
pp. 41748, October 1994. [6: C. Berrou, A. Glavieux, and P. Thitirnajshima, “Near Shan— non limit errorcorrecting coding and decoding: Turbo—
codes,’7 in Proc. ICC ’93, pp. 1064~1070, May 1993.
7 J. Ha ’enauer, P. Robertson, and L. Papke, “Iterative
_ 8 (“Turbo”) decoding of systematic convolutional codes with the MAP and SOVA algorithms,” in Proc. ITG Tagung, Codierung fiir Quelle, Kanal and Ubertragung, pp. 21729, Oc
tober 1994. [8] P. Robertson, “Illuminating the structure of code and decoder for parallel concatenated recursive systematic (turbo) codes,”
in Proc. GLOBECOr’l/I ’94, pp. 12984303, December 1994. [9] E. Villebrun, “Turbo—decoding with close‘to—optimal MAP al—
gorithms.” Diploma thesis, TU Munich, September 1994. G. D. Forney, “The Viterbi algorithm,” Proc. of the IEEE,
vol. 61, pp. 2684278, March 1973. 1013 ...
View
Full
Document
This note was uploaded on 03/20/2012 for the course EEL 6650 taught by Professor Shea during the Spring '12 term at University of Florida.
 Spring '12
 Shea

Click to edit the document details