SuboptimalMAPDecoding_Robertson

SuboptimalMAPDecoding_Robertson - A Comparison of Optimal...

Info iconThis preview shows pages 1–5. Sign up to view the full content.

View Full Document Right Arrow Icon
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 2
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Background image of page 4
Background image of page 5
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: A Comparison of Optimal and Sub-Optimal MAP Decoding Algorithms Operating in the Log Domain Patrick Robertson, Emmanuelle Villebrun and Peter lloeher Institute for Communications Technology, German Aerospace Research Establishment (DLR) D—82230 Oberpfaifenhofen, Germany, Tel.: ++49 8153 28 2808; Email: Patrick.R0berts0n©dlrde Abstract For estimating the states or outputs of a Markov process, the symbol-by—symbol MAP algorithm is optimal. However, this al- gorithm, even in its recursive form, poses technical difficulties be— cause of numerical representation problems, the necessity of non- linear functions and a high number of additions and multiplica— tions. MAP like algorithms operating in the logarithmic domain presented in the past solve the numerical problem and reduce the computational complexity, but are suboptimal especially at low SNR (a common example is the MaxeLog—MAP because of its use of the max function). A further simplification yields the soft-output Viterbi algorithm (SOVA). In this paper, we present a Log—MAP algorithm that avoids the ap— proximations in the Max—Log-MAP algorithm and hence is equiv alent to the true MAP, but without its major disadvantages. We compare the (Log—)MAP, Max-Log-MAP and SOVA from a theo- retical point of View to illuminate their commonalities and differ— ences. As a practical example forming the basis for simulations, we consider Turbo decoding, where recursive systematic convolu— tional component codes are decoded with the three algorithms, and we also demonstrate the practical suitability of the Log—MAP by including quantization effects. The SOVA is, at 10—4, approxi— mately 0.7 dB inferior to the (Log—)MAP, the Max—Log—MAP lying roughly in between. We also present some complexity comparisons and conclude that the three algorithms increase in complexity in the order of their optimality. 1 Introduction We will consider trellis—based soft-output decoding algorithms delivering additional reliability information together with hard decisions. The Bahl—Jelinek algorithm, also known as the symbol-hy—symbol MAP algorithm (MAP algorithm for short), is optimal for estimating the states or outputs of a Markov process observed in white noise However, this algorithm is perhaps too difficult in practice, basically because of the nu- merical representation of probabilities, non—linear functions and because of mixed multiplications and additions of these values. Some approximations of the MAP algorithm have been de— rived, such as the soft-output Viterbi algorithm (SOVA) [2] and the Mae-Log—MAP algorithm [3, 4, 5]. In both algo— rithms, processing is exclusively in the logarithmic domain; values and operations (addition and max—function) are easier to handle. However, both algorithms are suboptimal at low signal—to—noise ratios, where we use TurboeCodes [6, 7, 8, 9], for example. In this paper, we will modify the Max~l.og—MAP algorithm through the use of a simple correction function at each max— 0—7803-2486—2/95 $4.00 © 1995 IEEE operation This algorithm, to be called Log-MAP algo— rithm, is equivalent to the MAP algorithm in terms of perfore mance, but without its problems of implementation. The cor— rection needs just an additional one-dimensional table look—up and an addition per max—operation. The organization of the paper is as follows: After reviewing the MAP and Max—lLog—MAP algorithms, we will derive the LogeMAP algorithm in Section 2. In Section 3, we will com- pare these algorithms with the VA and the SOVA. Complexity comparisons and quantization issues are covered in Section 4. Finally, numerical results are presented in Section 5 by ap- plying the addressed algorithms in a Turbo—Code system. 2 Definition of the MAP, Max-Log- MAP, and Log-MAP Algorithms 2.1 The Encoder and Notation Since we will study the behaviour of the (Max—)Log—MAP al- gorithm applied to the decoding of convolutional codes (and in particular recursive systematic convolutional (RSC) codes), we will choose a notation that complies with such an en- coder, for example one with four memory elements as shown in Fig. 1. Since the MAP algorithm is essentially block—oriented, we shall represent the binary input (information) data se— quence by (i: (d1, ..., dN). :l fl _ Puncturing _j Xp Figure 1: Two identical recursive systematic convolutional encoders employed in a Turbo coding scheme. The encoder has M memory elements. In our example (the RSC code is of rate 1/2) it has two outputs, one is 1009 the sequence of information bits 233 : (xi, : J, since the encoder is systematic. The other is the ‘parity informa— . . M tion’ sequence 573 : (a:§,...,:c§v), With (131}: : Zizo gifak_z-a ,gfw) is the feed—forward generator and ak : where (g5, . .. M . . . dk®Zi21gfak_i;s1mIlarly, (g3... erator of the encoder. The sequences is and :31" may be punctured, are then mod— ulated and transmitted over a channel. In this work, we have assumed BPSK modulation and an AWGN channel with one— sided noise power spectral density No. The corresponding received sequences be g” and For brevity, the sequence 3;? : (373,32?) will refer to the sequence of pairs of received systematic and parity symbols. , 9514) is the feed-back gen— 2.2 The MAP Algorithm We will not repeat the derivation of the MAP algorithm1 but only state the results. For more detail see [1, 6, 8]. Let the state of the encoder at time k be Sk, it can take on values between 0 and 2M — 1. The bit dk is associated with the tran- sition from step k — l to step k. Goal of the MAP algorithm is to provide us with the logarithm of the ratio of the a pos— teriori probability (APP) of each information bit dk being 1 to the APP of it being 0. We obtain: ;52 71(yk,Sk—1,Sk)'Otk—1(Sk—1)‘fik(5k) Z Z 70(9):: Sic—1,50 'ak—itsk—i) 'fikt’SkV Sk Sk—l . (1) where the forward recurs10n of the MAP can be expressed as: A(dk) : ln 1 Z Z 717(yk;Sk—1;Sk) ' ale—1(Slc—1) Sk_1i:0 1 . E E wakfik—nskl-ak—1(5k—1) 5k 5k_1i:0 1 f S :0 wow = {o :1; 0 and the backward recursion as: 01451) — ; (2) 1 Z Z 7/r(yk+1,5k, Sic-H) 'fik+i(5k+1) flktsk) : Skflizol ; Z Z :7i(yk+135k:S/c+1) 'ak(5k) 5k 5H1 i=0 '1 1 : v mm = {0 £th 0 e The branch transition probabilities aregiven by 7i((yiayfl,Sk—i,5k) = (1(dk = ilskask—l)‘ Z i) flyildk : 2',Sk,5k_1) P7’{Sklskn1}. (4) The value of q(d;c : ilSk,Sk_1) is either one or zero de— pending on whether bit 1' is associated With the transition from state Sk_1 to Sk or not. It is in the last component that we use a priori information for bit dk [10]: In our case of no parallel transitions, Pr{SkISk_1} : Pr{dk : 1} if q(dk 1: 1|Sk,Sk_1) I 1; and P1”{Sk|5k_1}1 P1’{dk Z 0} if (1011: = Olskzsk—l) = 1- 2.3 The lVIax—Log-MAP Algorithm As we already said in the introduction, the MAP algorithm is likely to be considered too complex for implementation in a real system To avoid the number of complicated opera— tions and also number representation problems, one does not calculate way?“ yi), Shh 3k), ak(Sk) and flk(Skl any longer. One computes and works with the logarithms of these values instead [5]. By taking the logarithm of 7i((y,:, yg), Sk_1, Sh) derived in (4) and inserting 1 (figwgwxgoshshafl (5) «mo ' ' we obtain the following expression for : 1: :iisvk)SVk—1): 218:1:3 ln7i((y29y£):Sk—15Sk) : ink—{34) QyZI11:(i,Sk,S/C_1) N 0 + +111 Pr{SkISk_1} + K. (6) We can ignore the constant K, indeed it cancels out in the calculation of ln ak(Sk) and in B14516). We must remember that NC must be estimated to correctly weight the channel information with the a priori probability Pr{S;c Sk_1}. For ln ak(5k), we get: 1 1n ak(Sk) : ln( 2: Z eln7:((yiyy:),5k—175k)+ln ak—1(5k—1)) Sk_1i:0 1 m]n(Z Z Z:cln71((yiyy:)vSk—lysk)+lnak—lfsk—ll). (7) Sk Sk—l 1:0 To obtain a simple solution, we use the following approxima- tion: ln(e<51 + + 65") 2 max 61; i€{l..n} (8) maxi E 61- can be calculated by successively using n — 1 maximum functions over only two Values. From now on, we define Tittyiayg):sk_—1;Skl : 1n7i((yZ>y£)7Sk—1:Sk)a &k(:5k) : 1n ak(Sk) and fik(5k) : lnfik(Sk). We eventually obtain: @ktsk) % (EinJWinZV 911;), Sic—1; Sic) + (it—1(Sk—1» — max , (Ti(:(yi;:yZ);Sk—1;Sk) + @k—ltsk—1»; (5k,Sk—1,1) (9) and similarly, fillets/c) 3 max (?i((yi+11y§;+1)7 Sky Sk+1) + Bk+1lsk+1>> (Sk-l-li'l) (7z((yi+1,y£+1), 5k, Sk+1) + aktskll (10) "" max (5): 151mm 17;) The second terms are a consequence of the derivation from (2) and (3); they are needed for numerical reasons. Omitting them has no effect on the value of the output of the Max—Log— MAP algorithm, since these normalization terms will cancel out in (11). 1010 In the same way we can give an approximation of the log— likelihood reliability of each bit dk: A(dk) z (11) max (11(31):, Sic—1, 5k) + ale—1(Sk—1) + 5k(5k)) — (Ski'sk—l) (Smgx )(i‘otyk, SIC—17 Sic) + Eric—1(Sk—i) + kv k—l To be used in a Turbo decoder, we want the output of the Max-Log—MAP algorithm, A(dk), to be split into three terms (extrinsic, a priori and systematic components) as shown in [6, 7, 8]; it can easily be shown that this is possible here. The extrinsic component will be used as a priori informa- tion in the next decoding stage. This a priori log—likelihood ratio (LHL) for bit dk is called L(dk) for short We need to determine the a priori information in If q(d1C = 113k,Sk_1) 2 1, then _ Pr{dk :1} _ Pr{Sle _ } it (m) a 02> hence ln P7‘{Sk|Sk_1} : L(dk) — ln(l + eL(d’°)). An approxi- mation for P1'{Sk|Sk_1} can be easily found using (8): lnPr{Slek_1}z L(dk) — max(0, L(dk)). If q(dk : OISk, Sk_1) : 1 then w P1'{dk :1} _ 1—PT{Slek_1} “d” *1“ (m) *1“ (m) ’ “4) hence ln PT‘{Slek—1} = —ln(1 «1— end”). Similarly we can approximate ln Pr{.5';c 515-1} N —max(0, L(dk)). (13) 2.4 Correction of the Approximation: The Log—MAP Algorithm Because of the approximation (8) we applied, the Max-Log- MAP algorithm is suboptimal and yields an inferior soft— output than the MAP algorithm. The problem is to exactly calculate ln(exp 61 + + exp 6”). This problem can be solved by using the Jacobian logarithm [3, 4]: ln(e‘” + 6"?) = max(61,62)+1n(1+ e"l62‘61l) max(61,62)+ fc(i51 ~52I), (15) where is a correction function. Let us now prove re— cursively that the expression ln(exp 61 + + exp 5”) can be computed exactly. The recursion is initialized with (15). Sup— pose that 6 : ln(el"1 + + can—1) is known. Hence, ln(e‘s1 + + 66") = ln(A + 6'5") with A : 661 + + eén" : e." : max(lnA, 5n) + lllA — (Sni) : mam”) +fc(la — 6,,l) (16) When deriving the Log—MAP algorithm, we now augment all maximizations over two values with the correction function. As a consequence, by correcting at each step the approxi— mation made by the Max—Log—MAP, we have preserved the original MAP algorithm. By calculating fc(.), we lose some of the lower complexity of the Max—Log-MAP algorithm. That is why we approximate by a pre—computed table. Since the correction only de- pends on 152 — 61], this table is one dimensional. We shall see that only very few values need to be stored. q.e.d. 3 Comparison of the (Max—)Log- MAP and Soft-Output Viterbi Algorithms 3.1 Hard Decisions In [5], it was claimed that the hard decision of the Max—Log- MAP provides exactly the same hard decision as the Viterbi algorithm. We now present a mathematical proof of this re- sult for our example of rate 1/2 RSC codes; extensions are rudimentary. We assume for simplicity that the definitions of (ik(Sk) and do not include the normalization term. Remember that the Viterbi algorithm selects that path with the largest metric: M: max N 1 — 3—:03 ath 2 vpaflls{2[ Nook up )> 7,15% — WW1:th+1n(Pv'{Slek—1})]} . (17) The Max—Log—MAP output (11) for the last bit LIN can be written as A(dN) : M1(dN) — MU(dN), Where Mi(dN) = gnax {z(yN,SN-1,SN = 10) + melanin. (18) By applying the recursive definition of o‘zN_1(SN_1) (without normalization) we obtain: Mi(dN) = maxi1i(yN,5N_1,5N = 0) + (19) SNv—l (ijN_1(yN—1,SN_2,SN_1) + (EN—2(SN_2))}. max (SN—2 YIN—1) There exists an SN_2,ma$ and an jN_;_),mw such that Miidiv) = {IIBX{‘/i(yN,SN—1,SN = 0) + N—l (20) ijN_1,mw(yN—1,SN—2,mam,5iv_i) + aN-—2(SN—2,ma:c)}~ This is repeated N —— 2 times, yielding: Mi(dN) = flax {Ti(3/N:SN—1)SN = 0)} + N—i (21) Vikn-nax (yNi Sk—1,maxi Samar), 111:1 since we assume ("10(0) 2 0. By making a decision for the bit dN, the Max—Log—MAP algorithm selects the largest M¢(dN). By inserting (6) into the maximum over 'i of (21) and com— paring with (17), we can easily see that the Max—Log—MAP and Viterbi algorithm make the same decision for bit dN, since maxde{o,1}i\1g(dN): iM—i—C = M(dN), where C is a constant. To continue, let us define a metric for the Max—Log—MAP as: M(dk) : GHQ/k, Sic-1) 5k) + dk71(sk#1) + EMS)». (22) The Max-Log-MAP will choose that bit cl;c that maximizes (22). We now suppose that ilk/(dig) :: .M + C. Let us first prove that M(dk) = M(dk_1): max (SkySk—l 11') max. {:y’i(yk—1;Sk—2:Sk—1) + (Sh—~2yl) Cindi—2)} + Bic—1(Sk—l)}v M(dk_1) = max{ Sic—.1 (23) 1011 We apply the recursive definitions of &k_1(S;¢_1) and flk_1(5k_1), so that we obtain: M(dk_1) = max 1" 15—15 7— 5—, (Shh—112,){7tyk k 1 k)+ak 1( k 1)+ man} : M(d;,). (24) We can deduce from this recursion step and from the initial step where k z N that Vk E {1, . . .,N} MMax—Log—MAP = MUN) : Midi) : M + C (25) For each bit elk, the Max—Log-MAP algorithm calculates two Viterbi metrics and takes the largest one, This proves that the Max—LogeMAP algorithm makes the same hard decisions as the Viterbi algorithm. 3.2 Soft Outputs As we have already explained, the MaxeLog—MAP algorithm and the SOVA work with the same metric. If we consider only the hard decisions, they are identical. But, they behave in diifcrent ways in computing the information returned about the reliability of decoded bit dk. The SOVA considers only one competing path per decoding step. That is to say, for each bit dj it does not consider all the competing paths but only the survivors of the Viterbi algorithm. To be taken into account in the reliability estimation, a competing path must join the path chosen by the Viterbi algorithm without being eliminated. The differences between the (LongAP, MaxALogeMAP and SOVA are illustrated in Fig. 2. The MAP takes all paths into its calculation, but splits them into two sets: those that have an information bit one at step j and those that have a zero; it returns the LLB of these two sets. All that changes from step to step, is the classification of the paths into the respective sets. Due to the Markov properties of the trellis, the computation can be done relatively easily. In contrast, the Max—Log—MAP looks at only two paths per step: the best with bit zero and the best with bit one at transition j; it then outputs the difference of the log—likelihoods, However, from step to step one of these paths can change, but one will always be the maximumelikelihood (ML) path. The SOVA will al- ways correctly find one of these two paths (the ML path), but not necessarily the other, since it may have been eliminated before merging with the ML path. There is no bias on the SOVA output when compared to that of the Max—Log—MAP algorithm, only the former will be more noisy. 4 Complexity and Quantization 4.1 As mentioned earlier, the correction function in (15‘) used by the Log—MAP can be implemented using a look—up table. We found that excellent results can be obtained with 8 stored values and |61 —62| ranging between 0 and 5. No improvement is achieved when using a finer representation. We now present the result of complexity analyses in the following table, taking into account the additional complexity of including a priori information: Complexity Comparisons all paths are considered two paths are considered two paths are ' considered but the competing path may not be the best competing path V. , competing path /’; that now determines reliability (will survive to merge with ML path) Here path - - - — is eliminated Figure 2: Comparison between (Log)MAP, Mavaog—MAP and SOVA. The MAP uses all paths in the trellis to optimally determine the reliability of bit dj. The Max-Log—MAP makes its decision (and soft output) based on the best two paths with different dj. The SOVA also takes two paths, but not necessarily both the same as for the MaxeLog—MAP. Operation l I Max—Log—MAP Log—MAP | SOVA maxops 5x2M——2 5x2M—2 3(M+B+2M additions 10 x 2M + 11 15 x 2V+9 2 x 2 +“8 "J mult. by i] , 8 8 8 r bit comps 6(M + 1) look-ups 5 >( W— 2 If we assume that one bit comparison costs as much as one addition, this table allows us to conclude that the Max—Log— MAP algorithm is more than twice as complex as the SOVA, for memory 2% = 4 and less than 2 times as complex for NI : 2. 4.2 Quantization We shall now present the quantization ranges that were used in simulations of the Log-MAP, they are based on observa— tions of the distributions of the pertinent variables. We have attempted to take into account the large variations that are the result ofthe iterative decoding process in a Turbo decoder. Variable Number of levels Limit values gag? || 16 ‘i -1.83 2.16 L(dkj— 64 | 15.5 16.0 ?¢I,yk,5k,5k_1 256 I -15.875 16.0 MM 256 | —15.875 L 16.0 first) 256 | —127.0 A 128.0 A103“ 256 | 731.75 32.0 1012 5 Simulation Results for Applica- tion in a Turbo—Code System Figure 3 shows the BER for decoding Turbo-Codes with the MAP, Log—MAP, MaerogvMAP, and SOVA, respectively, taking no quantization effects into account. However, the Log—MAP is applied with an 8—Values correction table; a loss due to the 3 bit quantization of the correction table is not visible. Results for the SOVA are taken from 0 10 I _1 9—0 MAP 10 Max-Log—MAP - <>—<> LogAMAP 1072 3K \\\A\ H SOVA “ 10'3 — \ \ — \ \ \A E 10" ~ \ \\ - m \ 10'5 L \\ A \ \ \s\\\ 10“ — \ E1 \6 10'7 — ‘ 10’8 7,7,... . l . ...i..,,.i,M 1.0 1.5 2.0 2.5 Eb/No, dB Figure 3: BER for Turbo—decoding with MAP, Log—VIAP, Max—Log—MAP and SOVA (8 iterations). N : 1024, It! : 4. Finally, Figure 4 shows the corresponding BER for the Log- MAP with and without quantization. We set NO to 2, and used an 8 values table. The step—size of the correction table is equal to 1/8. 10 T i 10" v - 10.2 b\ A A 10.. >\\ 7 5 10" ~ « d] 10'5 7 \‘ED 10-6 Le—oMAP \\ _ i I: Log-MAP \f 10-7 ;<>——<>quamized Log-MAP _ 10‘8 J 1 1.0 1.5 2.0 2.5 Eb/NO, dB Figure 4: BER for Turbo—decoding with MAP, L0g~MAP, and quantized Log—MAP (8 iterations). 6 Conclusions We have demonstrated a Log—MAP algorithm that is equiv— alent to the (true) symbol—by-symbol MAP algorithm, i.e., is optimum for estimating the states or outputs of a Markov process However, the novel implementation works exclusively in the logarithmic domain, thus avoiding the basic problems of the symbol—by—symbol MAP algorithm. The difference be— tween our Log-MAP algorithm and the known Max—Log—MAP algorithm, which just approximates the NIAP algorithm, is the substitution of logarithms by the Jacobian logarithm. This correction is simple to implement and quite insensitive to a quantization; a loss due to quantization of the correc— tion function is not visible. Further implementation issues concerning continuous data (“real—time implementation”) and simplifications valid for feed-forward trellises are still ap— plicable. We have compared the MAP, (Max—)Log—MAP and SOVA from a theoretical point of View to illuminate their commonalities and differences. As a practical example form— ing the basis for simulations, we considered Turbo—decoding, where recursive systematic convolutional (RSC) component codes have been decoded with the three algorithms. Quanti— zation of the whole Log-MAP algorithm was also investigated: the loss is about 0.5 dB. This can probably be improved, but Turbo—Codes have a huge variation of variables” ranges as a result of iterative decoding. Finally, we have compared the complexity of the three algorithms. The number of operations of the Log—MAP is about twice the number of operations of the SOVA, however, the former is more suited to parallel pro— cessing We conclude that the Log—MAP is particularly suitable for decoding Turbo-Codes, since in this challenging application we have a very low signal—to—noise ratio but a small number ofstatcs, so that the additional complexity is less pronounced. Acknowledgements The authors would especially like to thank Dr. J. Hagenauer and Dr. J. Huber for valuable discussions and comments. References [1_ L. Bahl, J. Cocke, F. Jelinek, and J. Raviv, “Optimal decoding of linear codes for minimizing symbol error rate,” IEEE Trans. Inform. Theory, vol. lT—ZD, pp. 284r287, March 1974. 2. J. Ha enauer and P. Hoeher, “A Viterbi al orithm with soft- . g o n n u - g decision outputs and its applications,” in Proc. GLOBECOM ’89, pp. 1680—1686, November 1989. [3 J. A. Erfanian, S. Pasupathy, and G. Gulak, “Reduced com— plexity symbol detectors with parallel structures for isi chan— nels,” IEEE Trans. Commun., vol. 42, pp. 166171671, Febru— ary/March/April 1994. [4 W. Koch and A. Baier, “Optimum and sub—optimum detec- tion of coded data disturbed by timeivarying intersymbol in- terference,” in Proc. GLOBECOM ’90, pp. 167971684, De— cember 1990. [5' J. Petersen, “lmplementierungsaspekte zur Symbol-by- Symbol MAP Decodierung von Faltungscodes,:’ in Proc. ITG Tagung, Codierung fn'r Quelle, Kanal und Ubertragung, pp. 41748, October 1994. [6: C. Berrou, A. Glavieux, and P. Thitirnajshima, “Near Shan— non limit error-correcting coding and decoding: Turbo— codes,’7 in Proc. ICC ’93, pp. 1064~1070, May 1993. 7 J. Ha ’enauer, P. Robertson, and L. Papke, “Iterative _ 8 (“Turbo”) decoding of systematic convolutional codes with the MAP and SOVA algorithms,” in Proc. ITG Tagung, Codierung fiir Quelle, Kanal and Ubertragung, pp. 21729, Oc- tober 1994. [8] P. Robertson, “Illuminating the structure of code and decoder for parallel concatenated recursive systematic (turbo) codes,” in Proc. GLOBECOr’l/I ’94, pp. 12984303, December 1994. [9] E. Villebrun, “Turbo—decoding with close‘to—optimal MAP al— gorithms.” Diploma thesis, TU Munich, September 1994. G. D. Forney, “The Viterbi algorithm,” Proc. of the IEEE, vol. 61, pp. 2684278, March 1973. 1013 ...
View Full Document

This note was uploaded on 03/20/2012 for the course EEL 6650 taught by Professor Shea during the Spring '12 term at University of Florida.

Page1 / 5

SuboptimalMAPDecoding_Robertson - A Comparison of Optimal...

This preview shows document pages 1 - 5. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online