Advanced channel coding techniques using bit-level soft information

(1)

A Dissertation by JING JIANG

Submitted to the Office of Graduate Studies of Texas A&M University

in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY

August 2007

(2)

A Dissertation by JING JIANG

Submitted to the Office of Graduate Studies of Texas A&M University

in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY

Approved by:

Chair of Committee, Krishna R. Narayanan Committee Members, Costas N. Georghiades

Andrew K. Chan Jianxin Zhou

Head of Department, Costas N. Georghiades

August 2007

(3)

ABSTRACT

Advanced Channel Coding Techniques Using Bit-level Soft Information. (August 2007) Jing Jiang, B.S., Shanghai Jiao Tong University Chair of Advisory Committee: Dr. Krishna R. Narayanan

In this dissertation, advanced channel decoding techniques based on bit-level soft information are studied. Two main approaches are proposed: bit-level probabilistic iterative decoding and bit-level algebraic soft-decision (list) decoding (ASD).

In the first part of the dissertation, we first study iterative decoding for high density parity check (HDPC) codes. An iterative decoding algorithm, which uses the sum product algorithm (SPA) in conjunction with a binary parity check matrix adapted in each decoding iteration according to the bit-level reliabilities is proposed. In contrast to the common belief that iterative decoding is not suitable for HDPC codes, this bit-level reliability based adaptation procedure is critical to the conver-gence behavior of iterative decoding for HDPC codes and it significantly improves the iterative decoding performance of Reed-Solomon (RS) codes, whose parity check matrices are in general not sparse. We also present another iterative decoding scheme for cyclic codes by randomly shifting the bit-level reliability values in each iteration. The random shift based adaptation can also prevent iterative decoding from getting stuck with a significant complexity reduction compared with the reliability based parity check matrix adaptation and still provides reasonable good performance for short-length cyclic codes.

In the second part of the dissertation, we investigate ASD for RS codes using level soft information. In particular, we show that by carefully incorporating

(4)

bit-level soft information in the multiplicity assignment and the interpolation step, ASD can significantly outperform conventional hard decision decoding (HDD) for RS codes with a very small amount of complexity, even though the kernel of ASD is operating at the symbol-level. More importantly, the performance of the proposed bit-level ASD can be tightly upper bounded for practical high rate RS codes, which is in general not possible for other popular ASD schemes.

Bit-level soft-decision decoding (SDD) serves as an efficient way to exploit the potential gain of many classical codes, and also facilitates the corresponding per-formance analysis. The proposed bit-level SDD schemes are potential and feasible alternatives to conventional symbol-level HDD schemes in many communication sys-tems.

(5)

(6)

ACKNOWLEDGMENTS

Looking back to my graduate years, I would like to thank many people who have contributed to this dissertation. First of all, I wish to take this opportunity to express my deepest gratitude to my advisor Professor Krishna R. Narayanan. It was his fascinating advanced coding course that inspired me to work in the area of channel coding for communications. His brilliant insight, patient guidance and countless encouragement throughout the last five years helped me to grow quickly from an inexperienced student to a sophisticated researcher. Working with him was truly a pleasant, stimulating and rewarding experience. Without his continuous support, the completion of this dissertation would have never become possible.

My heartfelt thanks go to Professor Marc Fossorier and Professor Ralf Koetter. Two main ingredients of this dissertation, reliability based iterative decoding and algebraic list decoding of Reed-Solomon (RS) codes were largely inspired by their seminal works. Their invaluable comments and constructive criticisms lead to sig-nificant improvements in many aspects of this dissertation. It is my real fortune to have their opinions on my work. Besides, the visit to Professor Koetter’s group at the University of Illinois was a fruitful and enjoyable experience to me.

I would also like to thank Professor Costas Georghiades, Professor Andrew Chan and Professor Jianxin Zhou, for serving as my dissertation committee and being supportive in different stages of my doctoral studies.

Moreover, I want to thank Professor Narayanan for generously offering me many academic opportunities: attending top conferences, visiting world-renowned research institutes, working as a summer intern in industry and lecturing a senior undergradu-ate DSP course, which motivundergradu-ated and inspired many interesting research works. The

(7)

main results in Chapter V were initiated by some interesting discussions with Rat-nakar Niranjan, Ivana Djurdjevic and Jorn Justesen during my visit in Illinois and during my internship at Seagate Research. I would like to thank many other friends during my study and visit the last five years for making my graduate life more enjoy-able. I am very grateful to my group mates, Jing Li, Dung Doan, Yongzhe Xie, Guosen Yue, Jun Zheng, Wenyan He, Vivek Gulati, Janath Peiris, Hari Sankar, Nitin Nangare, Angelos Liveris, Vladimir Stankovic, Abhiram Prabhkar, Kapil Bhattad, Qiang Li, Lingjia Liu, Salim El Rouayheb, Makesh Pravin, Anantharaman Balasaubaramanian, Weiyu Chen, Sabaresan Mothi and Karthik Nagasubramanian in the WCL group. I would also like to thank my friends and colleagues, Dake He, Ashish Jagmohan, Jian Lou, Jun Chen, Vadim Sheinin, Ivana Djurdjevic, Ching He, Guanjun Zhang, Xueshi Yang, Erozan Kurtas, Alexander Kuznetsov, Fatih Erden, Haiyun Yang, Lei Chen, Lin Yang, Jinghu Chen, Xiangyu Tang, Fang Bao, Tianyu Zhan, Ratnakar Niranjan, Wai Fong, Jun Xu, Wei Zeng, Suquan Ding and Lifeng Su, during my stay at IBM, Seagate and Legend Silicon, UIUC, Institute Eurecom, etc..

Finally, this acknowledgement would be incomplete without mentioning my fam-ily. I sincerely thank my parents, my grandmother and my wife, Yiqing, for their un-conditional love. Without their support, my dream of this dissertation would never have become true. This dissertation is dedicated to them.

(8)

III ITERATIVE SOFT-INPUT-SOFT-OUTPUT (SISO) DECOD-ING OF HIGH DENSITY PARITY CHECK CODES BY ADAPTING THE PARITY CHECK MATRIX . . . . 20

A. Iterative Decoding Algorithm by Adapting the Parity Check Matrix . . . 21

B. Geometric Interpretation of the Proposed Algorithm . . . . 25

C. Variations to the Generic Algorithms and Complexity Analysis . . . 31

1. Degree-2 Random Connection . . . 31

2. Various Groupings of Unreliable Bits . . . 32

3. Incorporated Hard Decision Decoding . . . 33

4. Partial Reliable Bits Updating . . . 33

5. Symbol-level Adaptation . . . 34

6. SISO Decoding for Turbo Equalization Systems . . . . 35

(9)

CHAPTER Page

D. Simulation Results . . . 37

1. AWGN Channels . . . 39

2. Rayleigh Fading Channels . . . 43

3. Inter-Symbol Interference Channels . . . 44

E. Discussions and Conclusion . . . 47

IV ITERATIVE SOFT-DECISION DECODING OF CYCLIC CODES BASED ON STOCHASTIC SHIFT . . . . 52

A. Proposed Iterative Decoding By Stochastic Shift . . . 52

B. Experimental Results and Discussions . . . 54

C. Conclusion . . . 59

V ALGEBRAIC SOFT-DECISION DECODING OF REED-SOLOMON CODES USING BIT-LEVEL SOFT INFORMA-TION. . . . 60

A. Algebraic Soft Decision Decoding of Reed-Solomon Codes . 61 1. Algebraic Soft-Decision Decoding . . . 62

2. Performance of Algebraic Soft-decision Decoding . . . 63

B. Performance Analysis of Algebraic Soft-decision Decod-ing over Erasure Channels . . . 66

1. Algebraic Soft-decision Decoding over the Binary Erasure Channels (BEC) . . . 66

2. Extension to 2u_{-ary Erasure Channels . . . .} ₇₃

C. Performance Analysis of ASD over Binary Symmetric Channels (BSC) . . . 74

D. Bit-level Decoding Region of Algebraic Soft-decision Decoding Algorithm . . . 79

1. Proposed MAS for a Mixed Bit-level Error and Erasure Channel . . . 80

2. Infinite Cost Performance Analysis . . . 85

3. Finite Cost Performance Analysis . . . 86

4. Monotonicity . . . 90

E. Bit-Level Generalized Minimum Distance Decoding Algorithm 92 1. The Generic BGMD Algorithm . . . 92

2. Performance Analysis of BGMD . . . 93

3. Discussions . . . 97

F. Simulation Results . . . 99

(10)

CHAPTER Page

VI SUMMARY AND FUTURE WORKS . . . . 106

A. Iterative Decoding of High Density Parity Check Codes . . 106

B. Bit-level Algebraic Soft-decision List Decoding . . . 107

C. Future Works . . . 107 REFERENCES . . . . 109 APPENDIX A . . . . 119 APPENDIX B . . . . 121 APPENDIX C . . . . 124 APPENDIX D . . . . 127 APPENDIX E . . . . 129 VITA . . . . 132

(11)

LIST OF FIGURES

FIGURE Page

1 Upper and Lower Bound on the Performance of an RS(31, 25)

Code under MLD . . . . 15 2 Performance Comparison of an RS(255,239) Code under Different

Decoders . . . . 17 3 Performance Comparison of an RS(255,127) Code under Different

Decoders . . . . 18 4 Performance Comparison of an RS(255,127) Code under Different

Decoders in the Practical SNR Region . . . . 19 5 Form of the Parity Check Matrix Suitable for Iterative Decoding

Obtained through Row Operations . . . . 21 6 Convergence Behavior of Iterative Decoding with and without

Adaptation of an RS(31, 25) Code . . . . 30 7 Performance of an RS (31,25) Code over an AWGN Channel . . . . . 40 8 Performance of an RS (63,55) Code over an AWGN Channel . . . . . 41 9 Performance of an RS (255,239) Code over an AWGN Channel . . . 42 10 Performance of an RS (31,15) Code over a Fully Interleaved Slow

Fading Channel . . . . 44 11 Performance of an RS (128,64) Code over a Fully Interleaved Slow

Fading Channel . . . . 45 12 Performance of an RS (204,188) Code Modulated Using 256-QAM

over a Rayleigh Fading Channel . . . . 46 13 TE Performance of an RS(31,25) Code over an EPR4 Channel . . . . 47

(12)

FIGURE Page 14 TE Performance of an RS(63,55) Code and an RS(255,239) Code

over an EPR4 Channel . . . . 48

15 Iterative Decoding Performance of an RS(255,239) Code over a Perpendicular Recording Channel (Ds = 2.5) . . . . 49

16 Fraction of Detectable Errors as a Function of Frame Error Rate . . 50

17 Performance of an RS (15,7) Code over an AWGN Channel under Different Updating Schemes . . . . 56

18 Performance of an RS (15,7) Code over a Rayleigh Fading Channel Using SSID Decoding . . . . 57

19 Performance of an RS (31,25) Code over an AWGN Channel Using Different Decoding Schemes . . . . 58

20 Bounds and Simulation Results of ASD for an RS(255,239) over a BEC 72 21 Bit-level Decoding Radius of an RS (255,55) Code . . . . 79

22 Upper Bounds of ASD for an RS (7,3) Code over a BSC . . . . 80

23 Bit-level Decoding Region of ASD for an RS(255, 239) Code . . . . . 88

24 Bit-level Decoding Region of ASD for an RS(63, 23) Code . . . . 89

25 Performance Bounds of Various Decodings for an RS (255, 239) Code over an AWGN Channel . . . . 96

26 Performance Bounds of Error and Erasure Decoding for an RS (255, 239) Code over an AWGN Channel . . . . 97

27 Performance of ASD for an RS (31, 25) Code over an AWGN Channel 100 28 Performance of ASD for an RS (255, 239) Code over an AWGN Channel101 29 Performance of ASD for an RS (63, 12) Code over an AWGN Channel 102 30 Performance of ASD for an RS (255, 175) Code over a Rayleigh Fast Fading Channel . . . . 103

(13)

FIGURE Page 31 Performance of ASD for an RS (255, 239) Code over a

Longitudi-nal Recording Channel, (Ds = 2.0, 90% Jitter) . . . . 104 32 Performance of ASD for an RS (255, 239) Code over a

(14)

LIST OF ALGORITHMS

ALGORITHM Page

1 Iterative Decoding Algorithm by Adapting the Parity Check Matrix. . . 24

2 Deg-2 Random Connection Algorithm. . . 32

3 Turbo Equalization Based on the ADP-SISO Algorithm. . . 36

4 SSID Algorithm for Cyclic Codes. . . 55

5 Bit-level Generalized Minimum Distance Decoding Based on Algebraic Soft De-cision Decoding for Reed-Solomon Codes . . . 94

(15)

CHAPTER I

INTRODUCTION

Reed-Solomon (RS) codes are one of the most popular error correction codes in many state-of-the-art communication and data storage systems. In most applications, RS codes are decoded via algebraic hard decision decoding (HDD), which does not fully exploit the error correcting capability of RS codes. When soft information about the channel output is available (for example, in an additive white Gaussian noise (AWGN) channel), HDD can incur a significant performance loss compared with optimal soft decision decoding (SDD). For the AWGN channel, the loss is about 2-4 dB for practical high rate RS codes at the target error rate where they are usually operating. More importantly, in many practical systems, RS codewords are mapped into their binary image expansions and then transmitted through channels using binary modulation formats. Therefore, it is of both theoretical and practical value to study decoding schemes for RS codes using bit-level soft information.

In this dissertation, we propose two advanced channel decoding techniques, namely, probabilistic iterative decoding [1][2] and algebraic soft-decision (list) de-coding (ASD) [3] [4] for RS codes, using bit-level soft information. We have shown that, in spite of the fact that RS codes are non-binary codes, efficiently exploiting bit-level soft information plays a crucial role in achieving the potential gain of the codes with a moderate complexity and facilitates the corresponding performance analysis as well.

The rest of this dissertation is organized as follows: we first review some back-ground on RS codes, which will be used throughout this dissertation in Chapter II.

(16)

We study iterative decoding schemes in Chapter III and Chapter IV. In Chapter III, we propose an iterative sum product algorithm (SPA) based decoding by adapting the parity check matrix in each iteration according to the bit reliabilities. In Chap-ter IV, another iChap-terative decoding method based on stochastic shifting the bit-level reliability values of the coded bits in each iteration is presented. In Chapter V, we study the performance of ASD for RS codes using bit-level soft information. Finally, we summarize the main contributions of this dissertation and discuss potential future works of the bit-level advanced channel coding techniques in Chapter VI.

(17)

CHAPTER II

BACKGROUND ON REED-SOLOMON CODE

In this chapter, we briefly review background materials on RS codes. We first give a historic survey of RS codes and the symbol-level code structure in Section A. Com-monly used algebraic HDD algorithms for RS codes are introduced in Section B. The binary image expansion of RS codes over GF(2m_{) is presented in Section C. Two} ad-vanced channel coding techniques, i.e. iterative decoding and algebraic list decoding are discussed in Section D and Section E respectively. Other popular SDD algorithms of RS codes are introduced in Section F. Finally, the ensemble average performance of the binary image expansion of RS codes over GF(2m_{) under maximum likelihood} decoding (MLD) is analyzed in Section G.

A. Symbol-level Structure of Reed-Solomon Codes

In this section, we give a brief review of the symbol-level code structure of RS codes.

1. Evaluation Form of Reed-Solomon Codes

RS codes were invented in 1960 by Reed and Solomon [5] as the name stands for. The code was initially defined as a set of message polynomials evaluated at N distinct points, which is usually referred to as the evaluation form of RS codes.

Consider the evaluation form of an (N, K) RS code over GF(q). Define the message vector f as:

f = (f0, f1,· · ·fK−1), fi ∈GF(q). (2.1) The polynomial form of the message is:

(18)

Let D = {x1, x2,· · ·, xN} be a set of distinct elements in GF(q). An RS code is defined by evaluating the message polynomial at N points:

C(N, K) ={(f(x1),· · · , f(xN))} (2.3) for all message polynomials f(x). If the evaluation points take the form xi =βi(i= 0,· · · , N −1), where β is a primitive element in GF(q), the RS code is evaluated using a set of fixed order points.

Reed and Solomon showed many nice properties of RS codes. For instance, RS codes are shown to be maximum distance separable (MDS) codes at the symbol-level. That is, for an (N, K) RS code, its minimum distance dmin = N −K+ 1, which is the maximum possible at the symbol-level. The initial paper also proposed encoding and decoding schemes for RS codes, however, they are not very efficient for practical implementations.

2. Non-binary BCH Code Form of Reed-Solomon Codes

RS codes are closely related to another class of popular error correcting codes, Bose Chaudhuri and Hocquenghem (BCH) codes [6]. Gorenstein and Zierler showed that RS codes are equivalent to q-ary BCH codes over GF(q) [7].

Consider a BCH code over GF(q). The generator polynomial of the code is: g(x) = (x−βb_)(x₋_βb+1₎_{· · ·}_(x₋_βb+N−K−1₎ _(2.4) where β is a primitive element in GF(q) and b is an integer number. When b = 1, the code becomes a narrow sense BCH code.

Since the generator polynomial has N−K consecutive roots, the corresponding parity check matrix can be represented as follows [8]:

(19)

Hs =          1 βb _{· · ·} _β(N−1)b 1 βb+1 _{· · ·} _β(N−1)(b+1) · · · 1 β(b+δ−2) _{· · ·} _β(N−1)(b+δ−2)          (2.5)

where δ =dmin =N −K+ 1. In addition, due to the cyclic property of BCH codes, it can be shown that the dual code of an RS code is also an RS code. Therefore the generator matrix of an RS code can also be represented in the form of (2.5). Comparing (2.5) when b = 0 with (2.2) evaluated by a set of fixed order points xi = βi (i = 0,· · ·, N −1), we can see that RS codes and q-ary BCH codes over GF(q) are equivalent.

B. Hard Decision Decoding of Reed-Solomon Codes

Many efficient HDD schemes have been proposed to decode RS codes up to t =

bdmin−1

2 csymbol-level errors. Due to the equivalence of RS codes and non-binary BCH codes, the bounded distance HDD for non-binary BCH codes proposed by Peterson [9] can be applied to RS codes. However, the computational complexity of Peterson’s algorithm is still quite large for long codes, mainly at the error location step. It was not until Berlekamp’s seminal work [10], HDD of RS codes became truly efficient. Several years later, Massey showed that Berlekamp’s decoding scheme is equivalent to the problem of synthesizing the shortest linear feedback shift register (LFSR) to generate a given sequence and HDD of RS can be realized efficiently using a set of shift registers in hardware [11]. The algorithm is therefore called Berlekamp and Massey (BM) decoding. Besides BM decoding, Sugiyama et al. showed that Euclid’s algorithm can also be used for error location in RS HDD [12]. Frequency domain encoding and decoding methods of RS codes have been studied in depth in [13]. In

(20)

1986, Berlekamp and Welch proposed an interpolation based decoding algorithm for RS codes [14], which paved way for the later algebraic list decoding algorithms [3].

When error locations are known to the decoder, the error magnitudes can be obtained by solving a set of linear equations. Forney proposed a fast algorithm to calculate the error magnitudes [15], which is referred to as Forney’s algorithm. BM algorithm can also be easily modified to accommodate erasures. Since the error location of erased symbols are known, BM algorithm can decode more erasures than errors. In general, HDD of RS codes will succeed if the number of errors e and the number of erasures g satisfies 2e+g < dmin. In other words, if twice the number of erroneous symbols plus the number of erased symbols does not exceed the designed minimum distance of the code, we are guaranteed to be able to recover the original codeword. Besides, erasure decoding can also be used as a systematic way to encode RS codes. For more decoding techniques of RS codes, we refer interested readers to [8, 13, 16].

C. Binary Image Expansions of Reed-Solomon Codes

In this section, we consider RS codes over GF(2m_{), which are most commonly used.} It is known that all the 2m _{elements in} _GF₍₂m_{), 0, 1,} _β, _β2_, _{· · ·}_, _β2m₋₂

, can be represented by anm-dimensional binary vector overGF(2) using a basis which spans GF(2m_{). Addition operation in} _GF₍₂m_{) is nothing but component wise addition of} the vector over GF(2). Similarly, multiplication can be carried out by multiplying a binary vector with a binary multiplication matrix. Therefore, each entry in the parity check matrix Hs can be replaced by an m×m matrix over GF(2) for the purpose of multiplication. For instance, consider RS codes over GF(4) and let β be a root of the primitive polynomial p(x) = x2₊_x_{+ 1.} _β _{has the binary vector representation}

(21)

[0,1] and the multiplication operation ×β corresponds the binary multiplication of the vector expansion with a multiplication matrix:

   0 1 1 1   .

Hence, Hs has an equivalent binary image expansion Hb and the RS code has a binary linear block code representation. Let n = N ×m and k = K ×m be the length of the codeword and the message at the bit level, respectively. Hence, Hb is an (n−k)×n binary parity check matrix. In other words, an RS code over GF(2m_)can also be viewed as a binary linear block code.

D. Iterative Decoding

As discussed in Section C, RS codes can be represented using their binary image ex-pansions. Therefore, RS decoding problem is turned into a general decoding problem for binary linear block codes.

Ever since the invention of turbo codes [1] and the rediscovery of low-density-parity-check (LDPC) codes [2], belief propagation (BP) based iterative decoding has been a panacea for many coding and communication problems. Iterative decoding of linear block codes and the sum product algorithm (SPA) was studied in [17] [18] [19]. However, BP algorithm is usually not considered to be suitable for high den-sity parity check (HDPC) codes [18], since iterative decoding can easily get stuck at pseudo-equilibrium points due to the large number of short cycles in the corre-sponding bipartite graph of the code. Some research works have been focused on the construction of proper parity check sets for iterative decoding [18] [19], since the performance of iterative decoding will be different with the choice of parity check matrix even if the code is the same. Lucas et al. [18] suggested using minimum weight parity check sets for iterative decoding. Some algorithms for the small weight parity check sets search are also discussed. However, in general, finding a minimum

(22)

parity check vector is NP-complete. Moreover, for MDS codes, such as RS codes, we cannot expect to get a very small weight parity check vector. Since the parity check matrix is nothing but the generator matrix of the dual code, which is also an MDS code, the minimum weight of every parity check must be at least (K+ 1) (typically much larger). For high rate RS codes, where K is large, the parity check matrix is necessarily non-sparse, which is unsuitable for iterative decoding.

Yedidiaet al.[20] established the inherent connection between BP and statistical physics. In [19], Yedidia et al. proposed the “generalized belief propagation (GBP)” algorithm, which introduces hidden states in the bipartite graph of the code to help to improve the performance of iterative decoding. However, GBP still has problems in decoding HDPC codes over AWGN channels. Nevertheless, GBP inspired our study of BP algorithms for HDPC codes such as RS codes.

Recently, iterative decoding of RS codes have been studied by several research groups. Ungerboeck proposed a sub-trellis based iterative decoding scheme [21]. Yedidia proposed a factor graph based iterative decoding of RS codes using Galois-field fast Fourier transform (GFFT) as an instance of GBP [22]. Unfortunately, the proposed iterative decoding schemes only work well for short length RS codes. It is suggested that directly applying iterative decoding to HDPC codes does not give good results since the parity check matrix of an RS code is not sparse so that iterative decoding can quickly get stuck.

E. Algebraic Soft-decision List Decoding

In Section B, we have discussed many HDD schemes for RS codes. However, all the HDD algorithms can only decode up to t = bN−K

2 c. In 1995, based on the previous work by Berlekamp [23] and Berlekamp and Welch [14], Sudan [3] built

(23)

the first algebraic HDD algorithm which can correct beyond half dmin errors for low rate RS codes with polynomial time complexity. Two years later, Guruswami and Sudan [24] improved Sudan’s decoder [3] and enlarged the decoding radius up to tGS = bN−pN(K −1)cfor all rates with polynomial time complexity. Koetter and Vardy [4] generalized Guruswami and Sudan (GS) decoding algorithm and presented a multiplicity assignment strategy (MAS) for the GS list decoder, which can take into account the soft information available at the decoder input. Algebraic soft-decision decoding (ASD) algorithms have since then gained great research interest. In contrast to the iterative decoding algorithms in Section D, ASD is algebraic in nature. However, soft information can still be incorporated in the algebraic decoding procedure by appropriate multiplicity assignment.

From a theoretical perspective, optimal MAS for ASD and corresponding perfor-mance analysis still remains an open problem. In [4], Koetter and Vardy presented an asymptotically optimal MAS that maximizes the transmission rate for a given channel such that the probability of error can be made arbitrarily small as the code length goes to infinity. Multiplicity assignment optimization for finite length RS codes has been considered in [25–27] using numerical algorithms. In [28], a general framework has been studied for channels with additive cost [28]. In [27], MAS for general discrete memoryless channels (DMC) has been investigated and an upper bound based on a Chernoff bounding technique has been derived. However, the Chernoff-type bound [27] largely relies on numerical computations and gives little insight into the decoding region of ASD under a certain MAS. Besides, the bound becomes loose for practical high rate RS codes. More recently, the decoding region and typical error patterns of ASD with infinite cost over some basic DMC’s have been studied independently in

(24)

[29]1 _{and [30]. Based on the analysis in [29], the performance of practical high rate} RS codes can be tightly bounded over erasure channels. However, as suggested in [30], even with infinite cost, ASD has a significant gain over BM only for RS codes of low rates or when the number of candidate symbols at the decoder input is small.

From a practical perspective on the other hand, many techniques have been pro-posed for ASD, for example, [31–38] and references therein. We also refer interested readers to [39] for a comprehensive tutorial of the generic GS algorithm. Most of ASD schemes, such as the KV algorithm, can significantly outperform HDD for low rate RS codes. However, to achieve large coding gain for high rate practical RS codes, the complexity can be prohibitively large [34]. Large computational complexity and the limited performance gain for high rate RS codes of ASD becomes the main obstacle to steer ASD decoding towards an implementable alternative to conventional HDD.

F. Other Soft Decision Decoding Schemes for Reed-Solomon Codes

In this section, we review some other existing SDD techniques for RS codes.

1. Enhanced Algebraic Hard Decision Decoding

The idea to take advantage of the soft information of the received bits to improve the decoding performance of RS codes can be dated back to Forney’s 1966 work [40]. In the original paper, Forney suggested successively erasing some of the unreliable symbols in the received signals and use algebraic HDD to decode. Thus, if twice the number of erroneous symbols plus the number of erased symbols does not exceed the designed minimum distance of the code, i.e., 2e+g < dmin, we can recover the original codeword. Forney proved that GMD guarantees to return an estimated codeword.

(25)

Since GMD has to run the generic HDD each time after erasing some symbols, the complexity of the GMD is about dmin/2 times as large as that of the generic HDD. Since GMD is based on the algebraic HDD and also takes advantage of the reliability information, it is called enhanced algebraic HDD.

Another enhanced algebraic HDD is Chase decoding [41]. The most popular version of Chase decoding, Chase type-II decoding (which is also usually abbreviated as Chase decoding) is also a reliability-assisted decoder, i.e., the decoder exhaustively flips up to d least reliable symbols and incorporates an algebraic HDD in each step. If the number of errors in the rest of the symbols is within the HDD error correction radius, the codeword can be recovered. Since Chase decoding involves exhaustively flipping symbols, its complexity is exponential in d. A hybrid of Chase type-II and GMD (Chase-GMD) has been proposed in [42]. Enhanced HDD usually gives a mod-erate performance improvement over HDD with reasonable complexity for practical applications.

2. Reliability Based Decoding

As shown in Section C, RS codes can be represented using their binary expansions. Efficient decoding methods for the general binary linear block codes such as the reli-ability based ordered statistics decoding (OSD) [43][44] can be applied to RS codes as well. The main idea of OSD is to propose reprocessing based on the most reliable basis (MRB) of the received signal. Since the reprocessing involves exhaustively flip-ping pbits in the MRB, the complexity increases exponentially with p. Nevertheless, with some small p, the OSD decoder provides good performance for short to medium length block codes. Recently, a variation of box and match (BMA) decoding [45] has been proposed by Fossorier and Valembois to trade-off memory for time complexity. Another variation of OSD which does not need large memory and turns out to be

(26)

effi-cient has been proposed by Wu et al. [46] [47]. More recent works on BMA decoding for linear block codes can be found in [48] and [49].

3. Trellis Based Coset Decoding

All the soft decision decoding methods discussed above are list decoding based soft input hard output (SIHO) algorithms. In some situations, it is desirable to obtain soft output from the decoder. A typical example is when turbo equalization (TE) [50] is employed at the receiver and soft outputs from the decoder have to be fedback to the equalizer. Consequently, soft-input-soft-output (SISO) decoding algorithms for RS codes are of research interest. Though averaging over all returned codewords in the list can generate the soft output [51], a natural SISO decoder is still favorable in some applications.

In the early 90s, Vardy and Be’ery suggested a binary decomposition scheme of RS codes into BCH subfield subcodes and glue vectors [52], which essentially reduces the overall trellis complexity of MLD of RS codes. Some recent works [53] further reduce the complexity and modify the algorithm to be able to generate soft outputs efficiently. Nevertheless, this trellis based scheme still has an exponentially increasing complexity and works only for short RS codes or RS codes with small dmin.

G. Performance of Reed Solomon Codes under Maximum Likelihood Decoding Maximum likelihood decoding (MLD) of RS codes is a non-trivial task, especially for long codes. It has recently been shown that MLD of RS codes is NP-complete [54]. Consequently, analytical bounds on the performance of RS codes under MLD are of interest as benchmarks for suboptimal decoders. However, the performance of the RS code under MLD using a particular binary image expansion is still difficult to analyze,

(27)

since the weight enumerator of an RS code using a specific binary image expansion is in general unknown. In this section, we study the ensemble average performance of RS codes under MLD [55][56] using Divsalar bound [57], which is tighter than the union bound in the low signal to noise ratio (SNR) region. The ensemble average of the RS code is taken by averaging over all binary representations of the RS code expanded using all possible binary bases for each symbol.

Due to the maximum distance separable (MDS) property of RS codes, the symbol-level weight enumerator is well known [8] and given by:

Rw = µ N w ¶ (q−1) w−dXmin i=0 (−1)i µ w−1 i ¶ qw−dmin−i _(2.6)

where w is the symbol-level weight of a valid codeword and Rw is the corresponding weight enumerator at the symbol-level.

Whereas, the bit-level weight enumerator of an RS code is not unique, i.e., each symbol of the RS codeword can be expanded into different binary vectors using dif-ferent bases. The binary image expansion of an RS code depends on the specific basis chosen to expand the m-dimensional binary vectors for each symbol. Though the weight enumerator given the specific basis is difficult to obtain, Retter [55] computed the averaged weight enumerator over the ensemble of all possible binary expansions for all symbols, i.e, of the generalized RS ensemble (GRSE). Recently, El-Khamy and McEliece [56] got exactly the same bit-level weight enumerator expression by assuming that each the bit-level Hamming weight of each symbol in the RS codeword is binomially distributed. The overall weight enumerator polynomial can then be

(28)

expressed as: G(x) = Nm X i=0 Gixi = N X w=0 Rw " _m X j=1 ¡_m j ¢ xj 2m₋₁ #_w = N X w=0 Rw Nw((x+ 1) m₋₁₎w _(2.7)

Expanding G(x), we can get: Gi = N X w=0 Rw Nw w X j=0 (−1)w−j µ w j ¶µ jm i ¶ = min(N,i)_X w=dmin Rw Nw w X j=0 (−1)w−j µ w j ¶µ jm i ¶ (2.8) The second equation holds by the observation that there is no non-zero codeword of symbol-level weight w ≤ dmin and there are no codewords of symbol-level weight w > icontributing to the enumerator of bit weighti. Each code in the GRSE inherits all the properties of RS codes, e.g., designed minimum distance, symbol-level MDS property, etc.. Therefore, it is reasonable to evaluate the performance of the GRSE, which gives some idea about the performance of a specific RS code under MLD.

Here, we use standard bounding techniques to study the performance of the GRSE under MLD. Define the normalized weight δ = i/n, the normalized weight enumerator r(δ),(lnGi)/n and the normalized SNR per coded bit asρ=R E_Nb₀. We have the union-type bound on the frame error rate (F ER) as

F ER≤

n

X

i=dmin

e−nE(ρ,δ) _(2.9)

where the exponent term E(ρ, δ) depends on the specific bound chosen. For the standard union Chernoff bound, we have the exponent:

(29)

0 1 2 3 4 5 6 7 8 9 10 10−6 10−5 10−4 10−3 10−2 10−1 100 FER E_b/N₀(dB) RS(31, 25) HDD RS(31, 25) ML UB RS(31, 25) ML LB

Fig. 1. Upper and Lower Bound on the Performance of an RS(31, 25) Code under MLD

In this dissertation, we choose a modified union-type bound derived by Divsalar [57], which is tight in the low SNR region, where standard union bound is loose. For this bound, the exponent is:

E(ρ, δ) = −γ(δ) + 1 2ln[β+ (1−β)e 2γ(δ)_{] +} δβ 1−δ(1−β)ρ (2.11) where β is: β = r ρ1−δ δ 2 1−e−2γ(δ) + ( 1−δ δ )2[(1 +ρ)2−1]− 1−δ δ (1 +ρ) (2.12) Since the above upper bound is an average over the ensemble of binary expan-sions, it is not cleara priori, how tight it is in predicting the performance of a specific RS code. Therefore, we also compute a simulation based lower bound for medium

(30)

length codes. The lower bound is obtained using the following procedure: First, we run a soft decision decoder (e.g. some of the proposed decoding algorithms discussed in the following chapters). When the received vector is closer to the estimated code-word than to the transmitted codecode-word, an error is counted for the ML decoder. Otherwise, we assume that the ML decoder does not make an error. This provides a lower bound always, the tightness of which depends on how good the decoder is. The upper and lower bounds of an RS(31,25) code under MLD along with the performance of HDD are shown in Figure 1. It can be seen that the ML upper bound of the GRSE is tight in the high SNR region, which is 0.5dB away from the simulation-based ML lower bound of a specific RS code (expanded using a fixed basis) at an FER = 10−4_. Besides, the HDD performance is 2-3dB away from the optimal performance under ML decoding, which is consistent with the commonly believed potential performance gain using SDD over HDD. Hence, we will use the GRSE upper bound under the MLD as a performance benchmark for long RS codes where simulation based lower bounds are difficult to obtain.

We first investigate the performance of a widely used high rate code, i.e., RS (255,239). In Figure 2, we plot the performance of the upper bound over GRSE under MLD, HDD with error correction radius t = (dmin−1)/2 and a hypothetical “genie decoder”, which can correct up to t = (dmin − 1) symbol errors. We can see that, the HDD is asymptotically 3dB worse than the performance under MLD (the largest gap is about 4dB, which appears at around an F ER = 10−20_{). The} “genie decoder” is optimal for asymptotically large SNRs. However, this happens only at very low FERs (say, at an FER = 10−200_{), which is impractical for most of} the applications. For practical SNRs, it has approximately 2dB loss compared with the performance under ML decoding.

(31)

4 6 8 10 12 14 16 10−300 10−250 10−200 10−150 10−100 10−50 100

comparison of performance bounds of RS(255,239)

E

b/N0

FER

hard decision decoder t=(d

min−1)/2

hard decision decoder t=(d

min−1)

bounds on RS code ensemble

Fig. 2. Performance Comparison of an RS(255,239) Code under Different Decoders

Figure 3. We can see that the performance of the RS ensemble under MLD reaches an FER = 10−4 _{at an}_E

b/N0 = 1.2dB and outperforms the hypothetical decoder and HDD by 2.5dB and 5dB respectively. The performance of RS ensemble under MLD is only about 0.6 dB away from the sphere packing bound [58] (as can be seen from Figure 4), making it comparable to the best known turbo and LDPC codes. Note that for this code, all known decoders up to now are still away from the performance under MLD, making it difficult to obtain good simulation based lower bounds to estimate the MLD performance of the RS code.

The above examples reveal two important facts. First, the RS code itself is a good code, which can perform close to capacity under MLD as the codeword length goes long. Second, symbol-level bounded distance decoding does not fully exploit

(32)

0 2 4 6 8 10 12 14 16 10−300 10−250 10−200 10−150 10−100 10−50 100

E b/N0 FER HDD t=(d min−1)/2 HDD t=(d min−1) rand ensemble ml RS ensemble ml

Fig. 3. Performance Comparison of an RS(255,127) Code under Different Decoders

the error correction capability of the code. Recently, there has been very significant developments in bounded distance decoding beyond half the minimum distance (see the seminal work [24][59]). However, the above example shows that even the “genie decoder”, which decodes up to t = dmin −1, performs far away from ML decoding. It suggests that the efficient RS decoder should be able to decode far beyond the minimum distance to take full advantage of the error correction capability of the code, since the number of low weight codewords can be very small. This motivates alternative design philosophies for RS soft-decision decoders, that is, decoding RS codes by taking advantage of bit-level soft information.

(33)

0 1 2 3 4 5 6 7 8 9 10−8 10−7 10−6 10−5 10−4 10−3 10−2 10−1 100

E b/N0 FER HDD t=(d min−1)/2 HDD t=(d min−1) RS ensemble ml

Fig. 4. Performance Comparison of an RS(255,127) Code under Different Decoders in the Practical SNR Region

(34)

CHAPTER III

ITERATIVE SOFT-INPUT-SOFT-OUTPUT (SISO) DECODING OF HIGH DENSITY PARITY CHECK CODES BY ADAPTING THE PARITY CHECK

MATRIX

It is commonly believed that BP decoding is not suitable for high density parity check (HDPC) codes [18], since iterative decoding can easily get stuck at pseudo-equilibrium points due to the large number of short cycles in the factor graph.

In this chapter, we present an SPA based SISO iterative decoding algorithm for HDPC codes. The main novelty in the proposed scheme is to adapt the parity check matrix at each iteration according to the bit reliabilities such that the unreliable bits correspond to a sparse submatrix and SPA is then applied to the adapted parity check matrix. The proposed algorithm can be geometrically interpreted as a two-stage gradient descent with an adaptive potential function. This adaptive procedure is crucial to the convergence behavior of the gradient descent algorithm since it prevents the gradient descent procedure from getting stuck at pseudo-equilibrium points and, hence, significantly improves the convergence behavior of the iterative decoder

In principle, the proposed algorithm can be applied to any linear block code; however, we restrict our attention to RS codes in this dissertation because of the practical interest in SDD of RS codes and the fact that the gain from this adaptive procedure is likely to be significant for codes with dense parity check matrices such as RS codes. Simulation results show that the proposed iterative decoding scheme performs well for RS codes with reasonable decoding complexity, even though their original parity check matrices are not sparse.

The rest of the chapter is organized as follows: The generic iterative decoding algorithm is presented in Section A. A geometric interpretation of the proposed

(35)

.

1

.

0

.

0

.

0

.

0

.

1

.

0

.

0

.

0

.

1

0

.

0

.

0

.

0

1

.

0

.

0

.

0

.

0

.

1

.

i

1

i

2

i

3

i

4

i

5

i

6

Dense part

Fig. 5. Form of the Parity Check Matrix Suitable for Iterative Decoding Obtained through Row Operations

algorithm is given in Section B. Several variations of the generic algorithm are inves-tigated in Section C. In Section D, simulation results of the proposed algorithm are presented and compared with popular RS soft decoding algorithms. Discussions and conclusions are presented in Section E.

A. Iterative Decoding Algorithm by Adapting the Parity Check Matrix

We will use underlined letters to denote vectors and bold face letters to denote ma-trices. Let c = [c1, c2, . . . , cn] be the binary representation of an RS codeword. In the description of the generic algorithm, we first assume that the bits are modulated

(36)

using BPSK (with 0 mapped to +1 and 1 mapped to −1) and transmitted over an AWGN channel (extension to other channels is straightforward). The received vector is given by

y = (−2c+ 1) +n, (3.1)

Thus, the initial reliability of each bit in the received vector can be expressed in terms of the log-likelihood ratios (LLR) as observed from the channel:

L(0)(ci) = logP(ci = 0|yi) P(ci = 1|yi)

, (3.2)

The proposed algorithm is composed of two stages: the matrix updating stage and the bit-reliability updating stage. In the matrix updating stage, the magnitude of the received LLRs |L(ci)| are first sorted and let i1, i2, . . . , iN−K, . . . , in denote the position of the bits in terms of ascending order of |L(ci)|, i.e., the bit ci1 is the least

reliable and cin is the most reliable. We begin with the original parity check matrix

Hb and first reduce the ith

1 column of Hb to a form [1 0. . .0]T. Then, we try to reduce the ith

2 column of Hb to a form [0 1 0. . .0]T and so on. We can be guaranteed to proceed until the ith

(N−K) column, since there are at least (N −K) independent columns in Hb. Then we try to reduce the ithN−K+1 to [0| {z }. . .0

(N−K)

1,0, . . . ,0]T_{. However,} there is no guarantee we can do this. If we are unable to do so, we will leave that particular column and try to reduce ith

(N−K+2) column to the above form and so on. Finally, we can reduce (n−k) columns among then columns of Hb to be the identity matrix, since the matrix is (n−k)×n and is full rank. The matrix is thus reduced to a form as shown in Fig. 5. We denote the set of unreliable bits corresponding to the sparse submatrix as B_L.

(37)

of LLRs as:

L(j) = [L(j)(c1), L(j)(c2),· · · , L(j)(cn)] (3.3) where initially L(0) _{is determined from the channel output. Then, the parity check} matrix is reduced to a desired form based on L(j)_:

H(j)_b =φ(Hb,|L(j)|). (3.4) Henceforth, in the bit-reliability updating stage, the extrinsic LLR vector L(j)_ext is first generated according to L(j) _{using SPA [17] based on the adapted parity check matrix} H(j)_b :

L(j)_ext =ψ(H(j)_b ,L(j)₎ _(3.5) That is for each bit, the extrinsic LLR is updated according to:

L(j)_ext(ci) = n−k X l=1 Hli=1 2 tanh−1     n Y p=1 p6=i,Hlp=1 tanh Ã L(j)(cp) 2 !    (3.6)

The bit-reliability is then updated as:

L(j+1) ₌_L(j)₊_α_L(j)

ext (3.7)

where 0 < α ≤ 1 is a damping coefficient. This is continued until a predetermined number of times jmax = N1 or until all the parity checks are satisfied. A detailed description of the algorithm is given in Algorithm 1.

The proposed adaptive algorithm is inspired by the OSD [43]. However, instead of reprocessing the most reliable basis (MRB), we adapt the parity check matrix according to the bit reliability. It can also be viewed as a generalization of the iterative

(38)

Algorithm 1 Iterative Decoding Algorithm by Adapting the Parity Check Matrix Step1. Initialization: set α, jmax = N1, j = 0 and the coded bits LLR from the

channel: L(0) = 2 σ2y

Step2. Reliability based parity check matrix adaptation: H(j)_b =φ(Hb,|L(j)|). a) Order the coded bits according to the absolute value of the LLRs |L(j)| and record the ordering indices.

b) Implement Gaussian elimination to systematize the (n−k) unreliable posi-tions which are independent in the parity check matrix. (The submatrix can also be made to be degree-2 connected, see Section C-1).

Step3. Generating extrinsic information: Apply SPA to generate the extrinsic LLR for each bit using the adapted parity check matrix H(j)_b :

L(j)_ext =ψ(H(j)_b ,L(j)) (according to (3.6)). Step4. Bit-level reliabilities Update:

L(j+1) =L(j)+αL(j)_ext, where 0 < α≤1.

Step5. Hard decision: ˆci =

     0, L(j+1)_(c i)>0; 1, L(j+1)_(c i)<0.

Step6. Termination criterion: If all the checks are satisfied, output the estimated bits; else if j =jmax, declare a decoding failure; otherwise set j ←j+ 1 and go to Step2for another iteration.

(39)

a posterior probability (APP) decoding algorithm based on a set of minimum weight parity check vectors by Lucaset al. [18]. In [18], the iterative algorithm is interpreted as a gradient descent. The adaptive algorithm generalizes the idea of gradient descent and extends it to be a two-stage gradient descent algorithm with an adaptive potential function. The damping coefficient α serves as the step size in the gradient descent process to control the dynamics of convergence. In the following section, we look into the geometric interpretation of this algorithm.

B. Geometric Interpretation of the Proposed Algorithm

In this section, a geometric interpretation of the proposed algorithm as a two-stage optimization procedure is presented. The idea of using optimization methods, such as gradient descent, to solve decoding problems can be dated back to Farrell et al. [60]. The belief propagation (BP) based algorithms by Gallager [2] and Pearl [61] were also shown to be special cases of the gradient descent algorithm. More similar to the bit reliability updating algorithm in this dissertation, Lucas et al. [18] proposed the APP decoding algorithm using the minimum weight parity check sets. They also interpreted the proposed APP algorithm as a gradient descent. Here, we generalize Lucas’ idea and interpret the proposed adaptive algorithm as a two-stage gradient descent algorithm with an adaptive potential function.

Define the operator ν : [−∞,+∞]→[−1,1] as a mapping from LLR domain to tanh domain: ν(L) = tanh µ L 2 ¶ = eL−1 eL_{+ 1} (3.8)

where the mapping is one-to-one and onto.

(40)

expressed as: ν−1_{(t) = ln} µ 1 +t 1−t ¶ , t∈[−1,+1] (3.9)

We apply the one-to-one tanh transform on the LLRs and get the reliability measure of the received signal in the tanh domain as:

T= [T1, T2,· · · , Tn] = [ν(L(c1)),· · · , ν(L(cn))] (3.10) As in [17], we can measure the reliability of a parity check node in tanh domain as:

ν(L(s)) =ν(L(c1⊕c2⊕ · · · ⊕cl)) = l

Y

p=1

ν(L(cp)). (3.11) Following the concept of generalized weighted syndrome proposed by Lucas et al. ((20) in [18]), we define a cost function J, which characterizes the reliability of the received vector Twith a particular parity check matrix Hb.

Definition 1 Define the potential function J as:

J(Hb,T) =− (n−k)_X i=1 ν(L(si)) =− (n−k)_X i=1 n Y j=1,Hij=1 Tj (3.12)

where J is a function of both the parity check matrix Hb and the received soft information T.

The operator ν maps the original n-dimensional unbounded real space into an n-dimensional cube (since the output of the tanh function is confined to [-1, 1]). The potential functionJ is minimized iff a valid codeword is reached, that is all the checks are satisfied and |Tj| = 1 for j = 1,· · · , n, where Jmin = −(n−k). Besides, points with all |Tj| = 1 correspond to vertex of the n-dimensional cube. Therefore, valid codewords lie on the minimum potential vertices of the n-dimensional cube. The decoding problem can be interpreted as searching for the most probable minimum potential vertex given the initial point observed from the channel.

(41)

Note that the potential functionJ is minimized iff a valid codeword is reached. It is quite natural to apply the gradient descent algorithm to search for the minimum potential vertex, with the initial value T observed from the channel. Consider the gradient of J with respect to the received vector T. From (3.12), it can be seen that:

∇J(Hb,T) = µ ∂J(Hb,T) ∂T1 , ∂J(Hb,T) ∂T2 ,· · ·, ∂J(Hb,T) ∂Tn ¶ (3.13) where the component wise partial derivative with respect to Ti is given by:

∂J(Hb,T) ∂Ti =− (n−k)_X l=1 Hli=1 n Y p=1 p6=i,Hlp=1 Tp (3.14)

Thus, the gradient descent updating rule can be written as:

T(j+1) _←_T(j)₋_α_∇_J(H

b,T(j)) (3.15)

whereαis a damping coefficient as in standard gradient descent algorithms to control the step width.

Note that the reliabilities in tanh domain are confined to Ti ∈[−1,1]. However, the updating rule (3.15) does not guarantee this. Therefore, we use the following modified updating rule to guarantee the validity of the updated values:

T_i(j+1) ←ν  _ν−1³_T(j) i ´ −α  ₋ X Hli=1 ν−1   Y p6=i,Hlp=1 T(j) p       _(3.16)

whereν−1_{(x) = 2 tanh}−1_{(x). Recall that the above non-linear updating rule is exactly} the same as the Step 3-Step 4 in Algorithm 1.

When iterative decoding is applied to an HDPC code, with very high probability, the iterative algorithm will reach some local minimum points where ∇J(Hb,T) is zero or is close to zero (since a few unreliable symbols will render the components of ∇J(Hb,T) to be small or close to zero). We refer to these as pseudo-equilibrium

(42)

points. Since gradient descent gets stuck at these points, while these points do not correspond to valid codewords.

From (3.12), we observe that sinceJ is also a function of Hb, different choices of the parity check matrices Hb (though span the same dual space), results in different potential functions J. More importantly, each Hb results in a different gradient

∇J(Hb,T). The proposed algorithm exploits this fact and when a pseudo equilibrium point is reached, by adapting the parity check matrix based on the bit reliabilities, we switch to another Hb such that it allows the update in (3.16) to proceed rather than getting stuck at the pseudo-equilibrium point. However, note that the potential function that we want to minimize does not involve the Euclidean distance between the received codeword and current estimate at all. That is, the adaptive algorithm attempts merely to find a codeword that satisfies all the parity checks, without really enforcing that it be the one at minimum distance from the received word. Since small steps are taken in the gradient descent, very often we converge to the codeword at small distance from the received vector as well. However, there is no guarantee of convergence.

We use the following examples to show the operation of the adaptive algorithm and its difference from directly applying iterative decoding to an HDPC code.

Example 1: Consider codewords transmitted through a binary erasure channel (BEC). We first apply the gradient descent algorithm directly to HDPC codes (assume that each entry in the parity check matrix is i.i.d. with 0 or 1 equal probable) without reliability based adaptation. Assume that the erasure fraction is ², therefore the number of erased bits is n². Consider a particular parity check vector, any code bit will participate in that check with probability 1/2 (according to the i.i.d. equal probable assumption). A check is not erased iff all the participated bits are not erased. Therefore, the probability that a check is erased is

(43)

P rob{L(s) = 0}= 1. −(1 2)

n²_{, n}_{→ ∞} _(3.17)

Assume that we have r parity check vectors that the ith _{bit participates in. The} gradient with respect to the ith _{coordinate becomes zero (i.e.,} ∂J(Hb,T)

∂Ti = 0) iff the

extrinsic LLRs from all the checks it participates are erased. The probability of zero gradient in the ith _{bit is:}

P rob{∂J(Hb,T) ∂Ti = 0}= [1. −2−(n²−1)_]r . = 1−r2−(n²−1)₊_o(2−(n²−1)₎ . = 1−r2−(n²−1) _→_{1, n} _{→ ∞} _(3.18) which suggests that unless the number of parity checks grows exponentially with n², iterative decoding quickly gets stuck at a pseudo-equilibrium point.

On the other hand, for BEC, it is known that by adapting the parity check matrix corresponding to less reliable bits (i.e. the erased bits), ML decoding performance can be achieved [62] in one iteration. Notice that Gaussian elimination cannot proceed iff some of the columns corresponding to the erased bits are dependent. In this case, there is ambiguity between two or more valid codewords. In such a case, the ML decoder also fails.

In conclusion, for BEC, gradient descent without adaptation tends to get stuck at a pseudo-equilibrium point, while the reliability based adaptation will help gradient descent to converge to the ML solution in one iteration.

Example 2: The idea of reliability based parity check matrix adaptation can nat-urally be extended to AWGN channels and the insight remains the same. Though adapting the parity check matrix based on the channel output does not guarantee to converge to the ML decision for AWGN channels, it does avoid iterative decoding

(44)

get-0 20 40 60 80 100 120 140 160 180 200 −30 −25 −20 −15 −10 −5 0 number of iterations potential function J E b/N0 = 3.0dB, without adaptation E b/N0 = 3.0dB, with adaptation E b/N0 = 4.0dB, without adaptation E b/N0 = 4.0dB, with adaptation

Fig. 6. Convergence Behavior of Iterative Decoding with and without Adaptation of an RS(31, 25) Code

ting stuck at pseudo-equilibrium points and thus improves the convergence behavior. We give a numerical example of the convergence behavior of iterative decoding of an RS(31,25) code in Figure 6. A typical realization of iterative decoding is simulated. The potential functionJ is plotted against the number of iterations. Since there are 30 parity checks for RS(31, 25), the minimum value of the potential function is J =−30 (corresponding to valid codewords). We can see that due to the high density of the parity check matrix of the RS code, iterative decoding without matrix adaptation (Algorithm 1 without Step 2) will easily get stuck at some pseudo-equilibrium. On the other hand, when the iterative algorithm is applied in conjunction with reliability based parity check matrix adaptation (Algorithm 1), the value of J quickly goes to

(45)

the global minimum as the number of iteration increases. Consequently, reliability based parity check matrix adaptation improves the convergence behavior of iterative decoding significantly. We will show in Section D that the adaptive algorithm also largely improves the error performance.

C. Variations to the Generic Algorithms and Complexity Analysis

In this section, several variations of the proposed algorithm are discussed either to improve the performance or to reduce the decoding complexity.

1. Degree-2 Random Connection

One problem with the proposed approach is that each bit in the unreliable part B_L

participates in only one check, it receives extrinsic information from one check only. If there is a bit error in the dense part participating in that check, the bit in B_L

tends to be flipped and the decoder tends to converge to a wrong codeword. To overcome this drawback, we can reduce the matrixHb to a form where the submatrix corresponding to the less reliable bits is sparse (say column weight 2 rather than 1). This can improve the performance since each less reliable bit now receives more extrinsic information while the submatrix corresponding to the unreliable bits still does not form any loops (i.e., unreliable bits themselves do not participate in any loops with each other). We can obtain this via a degree-2 random connection algorithm. The details are presented in Algorithm 2.

After the Deg-2 random connection, all the (n-k-1) positions in the parity check matrix are of degree-2 except thepth

1 column. The last columnp1 can be connected to some row or just left to be degree-1, which will not significantly affect the performance. This appears to improve the performance of the proposed algorithm especially in the

(46)

Algorithm 2 Deg-2 Random Connection Algorithm

Step1. Apply Gaussian elimination to the parity check matrix and obtain an identity matrix in the unreliable part.

Step2. Generate a random permutation of numbers from 1 to n-k. Record all the indices, i.e., p1, p2, p3,· · · , pn−k.

Step3. Adapt the parity check matrix according to the follow rule: Add pth

i+1 row to pthi row, for i = 1 to n-k-1.

high SNRs.

2. Various Groupings of Unreliable Bits

Another variation that can help to further improve the performance is to run the proposed algorithm several times each time with the same initial LLRs from the channel but a different grouping of the less reliable bits. It is possible that some bits with |L(cj)| close to those in the unreliable set BL are also of the wrong sign and vice-versa. Hence, we can run the proposed algorithm several times each time with a different grouping of the less reliable bits. That is, we can swap some bits in the reliable part with those in the unreliable part near the boundary and run the matrix adaptation all over again, which gives a newHb. We then run the proposed algorithm on that new matrixHb. Each time the proposed algorithm is run, a different estimate of codeword may be obtained due to the difference in the parity check matrixHb. All the returned codewords are kept in a list and finally the one that minimizes Euclidean distance from the received vector is chosen. We will see from simulation results that this method can significantly improve the asymptotic performance, but also increases

(47)

the worst case complexity. Similar techniques have been used in conjunction with OSD by Fossorier [63] and Wu [47]. The way of grouping reliable bits used here is similar to the grouping scheme by Wu [47]. We refer interested readers to [47] for a detailed description and asymptotic performance analysis.

3. Incorporated Hard Decision Decoding

A hard decision decoder can be used during each iteration in the proposed algorithm to improve the performance and accelerate decoding as well. Since the HDD may return a codeword which is different from the ML codeword, we do not stop the decoder once a codeword is returned by the HDD. Rather, we still iterate up to a maximum number of iterations to obtain all the codewords returned by HDD during each iteration and finally pick up the most likely codeword. This guarantees to perform no worse than the proposed algorithm or HDD. In practice, error detection schemes such as CRC or other test techniques as discussed in [64] can serve as a practical stopping criterion to reduce the average decoding complexity. Combining the adaptive scheme with other SIHO algorithms such as the KV algorithm have recently been investigated by [65].

4. Partial Reliable Bits Updating

The complexity in the iterative decoding part can be further reduced via “partial reliable bits updating” scheme.

The main floating-point operation complexity comes from the computation of the extrinsic information in the reliable part (where the submatrix is dense). However, in the adaptation of the parity check matrix, only some bits in the boundary will be switched from the reliable part to the unreliable part. Therefore, in the bit reliability updating stage, we only update the bits in the unreliable setB_Land some reliable bits with |L(cj)| close to those in the unreliable set BL. For example, at each iteration,

(48)

we may update the ith

1 , · · ·, ithn−k+M LLRs rather than all of them (where i1 through in are sorted in ascending reliability). The number of bits in the reliable part M can be adjusted to control the complexity.

On the other hand, in the computation of the tanh of each check, we can also make approximations to reduce the complexity. For instance, min-sum can be used instead of SPA in Step 3 Algorithm 1 [66]. Furthermore, since the bit reliabilities are first ordered, the minimum of the absolute value of the LLRs in the dense part of the parity check matrix is known. Thus, we can approximate the tanh of all the bits in the reliable part using the tanh of the minimum value. This modification can significantly reduce the floating point operation complexity while retaining most of the performance gain.

More sophisticated updating schemes can also reduce the complexity of matrix adaptation. El-Khamy and McEliece proposed a scheme that adapts the parity check matrix from previous ones, which reduces the overall complexity by 75% [67].

5. Symbol-level Adaptation

Gaussian elimination requires serial update of the rows and is difficult to parallelize. Here we propose an alternative algorithm that is parallelizable. The idea is to take advantage of the structure of RS codes and adapt the parity check matrix at the symbol level. Let SL={i1, i2, . . . , i(N−K)}be a set of (N−K) least reliable symbols (symbol-level reliability can be computed by taking the tanh product of bit-level reliabilities or taking the minimum of the bit-level reliabilities). In order to update the parity check matrix at the symbol level, we need to find a valid parity check matrix for which the submatrix corresponding to the symbols in SL is an identity matrix. The detailed procedure is as follows: first, the submatrix corresponding to the symbols in SL is filled with an (N −K)×(N −K) identity matrix and the rest

(49)

of the matrix with unknowns (erasure). The key idea is that computing the unknown symbols in the parity check matrix is equivalent to finding (N −K) valid codewords of the dual code which will be the rows of the parity check matrix for the original code. For the jth _{row, the}_ith

j entry is 1 and theith1 , ith2 , . . . , ithj−1, ithj+1, . . . , ithN−K entries are 0s and all other entries are erasures E (i.e., all the positions corresponding to the reliable symbols are erased). Since the dual code is an (N, N −K) RS code with dmin =K+ 1 and there are exactly K erasures in each row, Forney’s algorithm [15][8] can be used to compute the values in the erased positions. Each decoded codeword corresponds to one row in the original parity check matrix. By repeating this procedure for all (N−K) rows, we can get a systematic parity check matrix over GF(2m_{), where the submatrix corresponding to unreliable symbols is the identity} matrix. Using the binary expansion, we can then get the binary parity check matrix and thereafter apply the SPA using it. Unlike Gaussian elimination, each entry of the parity check matrix can be computed independently and, hence, the whole procedure can be parallelized. This provides a computationally efficient way to obtain a parity check matrix in the desired form for hardware implementation. Related concepts such as re-encoding have also been used to reduce the complexity of KV decoding (see [33]).

6. SISO Decoding for Turbo Equalization Systems

With slight modification, the generic iterative decoding algorithm can be embedded in a turbo equalization (TE) system. In a TE system, the inputs to the decoder are obtained from the frontend SISO equalizer rather than directly observing from the channel. After several decoding iterations, the decoder outputs the soft information and feeds it back to the equalizer. The proposed TE algorithm based on this SISO algorithm is given in Algorithm 3.

(50)

Algorithm 3 Turbo Equalization Based on the ADP-SISO Algorithm

Step1. Initialization: set maximum turbo rounds M2,M = 0 and the received value as observed from the channel.

Step2. Apply the BCJR algorithm (or other equa