A Study of Low Density Parity-Check Codes Using Systematic Repeat-Accumulate Codes

(1)

SYSTEMATIC REPEAT-ACCUMULATE CODES

_______________

A Thesis Presented to the

Faculty of

San Diego State University

_______________

In Partial Fulfillment

of the Requirements for the Degree Master of Science in Electrical Engineering _______________ by Jose Ruvalcaba Summer 2015

(2)

(3)

(4)

For I know the plans that I have for you”, declares the Lord, “plans to prosper you and not to harm you, plans to give you hope and a future.

(5)

ABSTRACT OF THE THESIS

A Study of Low Density Parity-Check Codes Using Systematic Repeat-Accumulate Codes

by Jose Ruvalcaba

Master of Science in Electrical Engineering San Diego State University, 2015

Low Density Parity-Check, or LDPC, codes have been a popular error correction choice in the recent years. Its use of soft-decision decoding through a message-passing algorithm and its channel-capacity approaching performance has made LDPC codes a strong alternative to that of Turbo codes. However, its disadvantages, such as encoding complexity, discourages designers from implementing these codes.

This thesis will present a type of error correction code which can be considered as a subset of LDPC codes. These codes are called Repeat-Accumulate codes and are named such because of their encoder structure. These codes is seen as a type of LDPC codes that has a simple encoding method similar to Turbo codes. What makes these codes special is that they can have a simple encoding process and work well with a soft-decision decoder. At the same time, RA codes have been proven to be codes that will work well at short to medium lengths if they are systematic. Therefore, this thesis will argue that LDPC codes can avoid some of its encoding disadvantage by becoming LDPC codes with systematic RA codes.

This thesis will also show in detail how RA codes are good LDPC codes by

comparing its bit error performance against other LDPC simulation results tested at short to medium code lengths and with different LDPC parity-check matrix constructions. With an RA parity-check matrix describing our LDPC code, we will see how changing the interleaver structure from a random construction to that of a structured can lead to improved

performance. Therefore, this thesis will experiment using three different types of interleavers which still maintain the simplicity of encoding complexity of the encoder but at the same time show potential improvement of bit error performance compared to what has been previously seen with regular LDPC codes.

(6)

LIST OF TABLES

PAGE

Table 3.1. Comparison Between Irregular and Regular LDPC Codes ...19

Table 6.1. N = 96 Simulation Points ...68

(10)

LIST OF FIGURES

PAGE

Figure 2.1. Digital communication system. ...4

Figure 2.2. Linear block code with k = 4 and n = 7. ...8

Figure 2.3. Systematic codeword structure. ...9

Figure 2.4. A rate 1/3 convolutional code. ...12

Figure 2.5. Viterbi algorithm example. ...13

Figure 2.6. Turbo code encoder and decoder. Encoder on the top and decoder on the bottom. ...15

Figure 3.1. Tanner graph for a LDPC code. Note the squares represent check nodes and circles bit nodes. ...21

Figure 4.1. Block diagram structure of an RA code. ...32

Figure 4.2. Block diagram structure of a systematic RA code. ...33

Figure 4.3. Block diagram of a non-systematic RA code. ...34

Figure 4.4. A systematic RA code tanner graph. ...42

Figure 4.5. Block diagram with an RA encoder and a SPA LDPC decoder...43

Figure 4.6. Equation for L-type interleavers ...49

Figure 4.7. Block diagram of the encoding circuit for a combined q = 3 repetition code and modified L-type interleaver. ...52

Figure 5.1. MATLAB simulation block diagram. ...55

Figure 5.2. Block diagram of our systematic RA encoder. ...56

Figure 5.3. RA parity check matrix w/ random interleaver (N = 408). ...57

Figure 5.4. RA parity check matrix w/ L-type interleaver. ...57

Figure 5.5. RA parity check matrix w/ modified L-type interleaver. ...58

Figure 5.6. Gallager parity-check matrix with N = 408. ...58

Figure 6.1. Simulation results for [1]. ...62

Figure 6.2. The original simulation results computed by MacKay...62

(11)

Figure 6.4. SNR vs. BER plot with random interleaver (N = 96)...64

Figure 6.5. SNR vs. BER plot with L-type interleaver (L = 8, N = 96). ...65

Figure 6.6. SNR vs. BER plot with modified L-type interleaver (L = 8, N = 96). ...66

Figure 6.7. SNR performance with L-type interleaver (L = 30, N = 96). ...67

Figure 6.8. SNR performance with modified L-type interleaver (L = 30, N = 96). ...68

Figure 6.9. Gallager parity-check matrix BER vs. SNR plot (N = 204). ...69

Figure 6.10. SNR vs. BER plot with random interleaver (N = 204). ...69

Figure 6.11. SNR vs BER plot with L-type interleaver (L = 8, N = 204). ...70

Figure 6.15. Gallager code SNR vs. BER plot (N = 408). ...72

Figure 6.16. SNR vs. BER plot for RA code with random interleaver (N = 408). ...73

Figure 6.18. SNR vs. BER plot with an L-type interleaver (L = 8, N = 408). ...75

(12)

ACKNOWLEDGEMENTS

To begin I want to thank God for this great blessing he has given me. For without God, no part of this thesis would be possible. Second I want to give thanks to Professor Harris, my advisor, who guided me throughout this whole project. To Professor Nagaraj and Professor Sarah Johnson from the University of Newcastle in Australia whose help on the topic allowed for the completion of this project. To Professor O’Sullivan who agreed to be part of my panel. I want to give special thanks to my mom, dad and sister who had strong faith in me and who went out of their way for me, in their own way, to help me get this research completed. To my girlfriend, Leslie Flores, who was my greatest cheerleader and has supported me through this process, even when at times it seemed like I was never going to finish. To all my old Broadcom colleagues especially, Mrs. Ana Ramos, who always kept me accountable about my thesis, even though work sometimes interfered with its completion. However, I am mostly in debt to Mr. Juan Garcia who along with God has been there with me in every step of the way. He dealt with my personal lows and highs from this project and whose wise words and heavy prayers always kept me going, even if it felt there was no end to this. To all of these people and friends who cheered me on, I thank you so much for keeping me going. Thank you all for your support and know that I feel so blessed that God put so many people in my path to get through this journey in my academic career. Again Thank you and may God bless you all!

(13)

CHAPTER 1 INTRODUCTION

Today’s form of information transmission is in a digital form and is available in many of the gadgets we use. From cell phones to satellite TV, consumers desire information

transmission to be as fast and error-free as possible, hence, when discussing about digital systems, error-correcting methods are necessary to mention. In 1948, Claude Shannon theorized that communications over a noisy channel can be improved by the use of a channel code C. Shannon stated that given a discrete channel with capacity C and a source with entropy per second H that:

If 𝐻 ≤ 𝐶 there exists a coding system such that the output of the source can be transmitted over the channel with an arbitrarily small frequency of errors (or an arbitrarily small equivocation). If 𝐻 > 𝐶 it is possible to encode the source so that the equivocation is less than H. [1]

In other words, if we have a channel code rate R that is equal to or less than the channel capacity, then it is possible to find an error-correction code able to achieve any given probability of error. However, he did not give many details on how to achieve these codes. Ever since, researchers have pushed the limits on finding error-correction codes that give us improved performance without becoming limited to the available communication techniques implemented.

In 1960, Robert Gallager, a PhD student at MIT, developed Low Density Parity-Check Codes, or LDPC codes, in his doctoral dissertation. He proposed using sparse parity check matrices as linear block codes along with soft decision iterative decoding for error correction. Due to lack of advanced computer processing the ability for further research and evaluation on this topic was stopped and left idle. As the years went on, some work on LDPC Codes was considered, such was the major work developed by Professor Michael Tanner in 1981 and his work on an effective graphical representation of LDPC codes with so called bipartite graphs, or Tanner graphs, which provided a simpler way of understanding how

(14)

iterative decoding worked. It was not until the introduction of Turbo Codes by Berrou, Glavieux, and Thitimajshima in 1993 which re-ignited interest in error correction codes that could reach close to Shannon capacity that gave way to the rediscovery of LDPC codes. Independently, David MacKay, a professor at the University of Wales, rediscovered LDPC codes and proved that these codes can achieve SNR level close to the Shannon Limit and to an extend similar to those found in Turbo Codes.

As turbo codes and LDPC codes have been arising in popularity in recent years, researchers began to look into ways in which we could improve these two methods or can create a new set of codes which fit into the family of either “turbo-like” codes or “LDPC-like” codes. Repeat-Accumulate codes, or RA codes for short, are a set of codes that can be considered as part of both “like” type codes due to its ability to be represented both as serially concatenated Turbo codes and as an LDPC code, depending on how it’s viewed. This is because RA codes have an interesting implementation efficiency of being encoded using a Turbo code representation while at the same time being decoded using a message-passing algorithm such as it is done in LDPC codes.

In this thesis, we will look into RA codes and the construction of practical repeat-accumulate parity-check matrices. Our goal is to show through simulations that utilizing RA codes can have similar, if not better, bit error rate (BER) performance as those with LDPC codes. Our thesis will also focus on the construction of the interleaver building block within the RA encoder which can become essential to the encoder complexity and performance. Overall, the idea to obtain from this project is that by utilizing an RA parity-check matrix and encoder as an error correction code can achieve good performance at low encoding and decoding complexity leaving us to ponder if these RA codes can potentially end the debate between LDPC and Turbo codes and eventually replace them.

1.1 O

UTLINE OF

T

HESIS

The outline of this thesis will be as follows:

Chapter 2 will recap the basic building blocks of a communication system. It will briefly discuss the modulator, demodulator, channel as well as channel encoder and decoder typically used.

Chapter 3 will overview Low Density Parity-Check (LDPC) codes. This chapter will look into the definition of LDPC codes, the various encoding methods, the soft and

(15)

hard-decision decoding algorithms used, and some of the advantages and disadvantages in using LDPC codes.

Chapter 4 will discuss Repeat-Accumulate codes. The chapter will present its encoding methods as well as the construction of its parity-check matrix. It also mentions how it uses the soft-decision algorithm, belief propagation, to have good decoding performance.

Chapter 5 gives the layout used for our MATLAB simulations between regular Gallager LDPC codes and systematic Repeat-Accumulate codes.

Chapter 6 is the continuation of the previous chapter and will show the results from the simulations. It will show various BER performance curves compared against an uncoded BPSK performance curve at different code lengths and using different interleavers.

Finally, chapter 7 will conclude this thesis and gives a summary on the study performed and an interpretation on the results obtained.

(16)

CHAPTER 2 BACKGROUND

2.1 D

IGITAL

C

OMMUNICATIONS

A digital communication system can be described by a set of building blocks, such as in Figure 2.1, in which transmitted information goes through. The system starts by having information come out from the source. This information source can come either from a person or machine, for example a voice signal or digital computer. It then goes through the

source encoder where it transforms the source output into a sequence of binary digits (bits) called the information sequence. It is in this block that an A/D converter is found to convert data from analog to digital. The source encoder is ideally designed so that

1. The number of bits per unit time required to represent the source output is minimized

2. The source output can be ambiguously reconstructed from the information sequence.

Figure 2.1. Digital communication system.

Next, each bit of the information sequence is encoded with redundant bits to create a binary sequence called a codeword. A channel encoder is needed to protect our information sequence from noisy environments that can distort our signal. Our channel decoder is used to recover the code sequence after it passes through the channel. Section 2.3 and 2.4 will talk in more in detail about channel encoders and decoders.

(17)

Following encoding comes modulating our encoded codeword. The modulator is responsible for transforming each output symbol from the channel encoder into a waveform suitable for transmission over a channel. Our Channel introduces noise to our signal and then it’s received by our receiver which then demodulates our waveform. The demodulator processes each received waveform and produces either a discrete or continuous output.

2.2 M

ODULATOR

/D

EMODULATOR

Modulation involves converting the original information signal (baseband signal) into another signal with a frequency convenient for transmission. To achieve this we may vary amplitude, frequency or phase of the signal to “modulate” our signal. PSK (Phase Shift Keying), FSK (Frequency Shift Keying) and ASK (Amplitude Shift Keying) are some methods used to modulate our signals.

Our baseband signals are transmitted as pulse trains which are generated by the voltage of electrical signals. Because our data stream consist of 0’s and 1’s, we map them to a “Bipolar” pulse train such that our 0 and 1 correspond to voltages -1 and 1, respectively, that we call Non-Return to Zero (NRZ). To modulate this signal we select a waveform of duration T seconds that is suitable for transmission for each encoded output symbol. For a wideband channel, we will have:

𝑠1(𝑡) = √2𝐸_𝑇𝑠cos 2𝜋𝑓0𝑡 where 0 ≤ 𝑡 ≤ 𝑇

and

𝑠2(𝑡) = √2𝐸_𝑇𝑠cos(2𝜋𝑓0𝑡 + 𝜋) = −√2𝐸_𝑇𝑠cos 2𝜋𝑓0𝑡 where 0 ≤ 𝑡 ≤ 𝑇

In this case, 𝑠₁(t) is mapped to 1 and 𝑠₂(t) is mapped to 0. T represents the duration of symbol in seconds and 𝐸_𝑠is the symbol energy, or bit energy in this case. This form of

modulation is called binary-phase shift keying, or BPSK. BPSK is defined as having our binary bits, 0 and 1, mapped to two signals which are phase shifted between 0 and 𝜋. Each

(18)

signal is transmitted every T seconds and transmits only 1 bit at a time over the channel1. When we increase the M-ary (𝑀 = 2𝑘) modulation scheme, i.e. if we transmit k bits instead of transmitting 1 bit at a time, we see that our symbol energy relates to bit energy as

𝐸_𝑠 = 𝑘𝐸_𝑏

where k is the number of bits transmitting per symbol.

Finally, we can also define the BPSK bit error probability as

𝑝 = 𝑄(√2𝐸𝑠⁄ )𝑁0

where 𝑄(𝑥) ≜ 1

√2𝜋∫ 𝑒

−𝑦2⁄2_𝑑𝑦 𝑖𝑛𝑓

𝑥 is the complementary error function, or the Q-function, of

Gaussian statistics.

2.2.1 Channel

If the transmitted signal is 𝑠(𝑡), then the received signal becomes

𝑟(𝑡) = 𝑠(𝑡) + 𝑛(𝑡)

where 𝑛(𝑡) is a Gaussian random process with one-sided power spectral density (PSD), 𝑁₀. The channel introduced a Gaussian random process to our signal which distorts our signal. By definition, a channel is a space where the signal is distorted by noise. The physical channel described as 𝑛(𝑡) is called an Additive White Gaussian Noise (AWGN) channel where its output is a Gaussian random variable with zero mean, 𝜇 = 0, and variance 𝜎2 =

𝑁₀⁄ 2. Therefore, our AWGN channel will have a probability density function of

𝑝(𝑥) = 1 √2𝜋𝜎2𝑒

−(x−μ) _2𝜎₂2

Aside from AWGN noise we could also consider other type of noise such as thermal noise, noise that comes from components, or multipath noise, called fading, and is defined as

1_{Note that only 1 bit at a time which means that our symbol energy is the same as that of our bit energy}_𝐸

(19)

noise composed of delayed versions of our original signal added together with our transmitted signal.

2.2.2 Demodulator

The demodulator must produce an output corresponding to the received signal. An optimum demodulator always includes a matched filter, or correlation detector, followed by a switch that samples the output once every T seconds. For BPSK modulation with coherent detection the sampled output is

𝑦 = ∫ 𝑟(𝑡)√2𝐸𝑠 𝑇

𝑇 0

cos 2𝜋𝑓₀𝑡 𝑑𝑡

The sequence of un-quantized demodulator outputs must be quantized so they can be passed on directly to the channel decoder for processing [2]. Therefore, decoding will occur by matching each output to a decision region which will map our output to our original point.

2.3 C

HANNEL

C

ODING

Channel coding is a way of introducing redundant bits, or parity bits, into a transmitted bit sequence in order to increase the transmission reliability in noisy

environments and improve the system’s error performance. Simple channel coding schemes allow the received data signal to detect errors while more advanced channel coding schemes provide the ability to correct channel errors as well [1]. Forward Error Correction (FEC) codes are used to enable the detection and correction of channel codes and are used in practice due to their ability to reduce bit error rate (BER) at a fixed power level or reduce power level at a fixed error rate at the cost of increasing bandwidth [2]. To describe channel coding, we look into two structurally different classes of coding methods: block codes and convolutional codes.

2.3.1 Linear Block Codes

A linear block code starts by dividing the information sequence into message blocks of k information bits, or symbols, each. A message block is represented by the binary k-tuple

𝒖 = (𝑢0, 𝑢1, … , 𝑢𝑘−1) called a message. There are a total of 2𝑘 different possible messages.

(20)

symbols, called a codeword. Therefore, corresponding to the 2𝑘 different possible messages, there are 2𝑘 different possible codewords at the encoder output. This set of 2𝑘 codewords length n is called an (n, k) block code. It is considered ‘linear’ if and only if the modulo-2

sum of two codewords is also a codeword [3]. While looking into the codeword and message bits it is of interest to look into the ratio 𝑅 = 𝑘 𝑛⁄ which we call code rate. The code rate is defined as the ratio between the number of information bits entering the encoder over the number of encoded bits outputting the channel encoder. In other words, it tells how many redundant parity bits we add per message bit.

For a binary code, each codeword c is also binary and when a codeword is assigned to a message u, then 𝑘 ≤ 𝑛. When 𝑘 ≤ 𝑛, 𝑛 − 𝑘 redundant bits are added to each message to form a codeword [2]. These redundant bits add protection to the code against any channel impairments. Figure 2.2 shows an example of a linear block code with 𝑘 = 4 and 𝑛 = 7.

Figure 2.2. Linear block code with k = 4 and n = 7.

(21)

Each message u can generate a codeword c through a matrix G of size 𝑘 × 𝑛called a

generator matrix. In other words, we can encode our message vector u by multiplying it with our generator matrix G which results in our codeword c.

𝒄 = 𝒖𝐺

Example 1. To develop the codewords from Figure 2.2, we would multiply each message by a 4 × 7 generator matrix. This is shown as:

[𝑐1𝑐2𝑐3𝑐4𝑐5𝑐6𝑐7] = [𝑢1𝑢2𝑢3𝑢4] | 1 1 0 1 0 0 0 0 1 1 0 1 0 0 1 1 1 0 0 1 0 1 0 1 0 0 0 1 |

It follows that an (n, k) linear code is completely specified by the k rows of a generator matrix G. A desired property from linear block codes is the systematic structure codewords can take, as shown in Figure 2.3. A codeword can be divided into two parts, the message bits and the parity bits. The message consists of all the k message bits sent into the encoder. The parity bit part consists of 𝑛 − 𝑘 parity check bits which are linear sums of the information bits. A linear code producing a systematic codeword is produced when our generator matrix is specified with an 𝑘 × 𝑘 identity matrix.

Figure 2.3. Systematic codeword structure.

For each 𝑘 × 𝑛 generator matrix G, there exists a (𝑛 − 𝑘) × 𝑛 matrix Hsuch that any vector of the row space of G is orthogonal to the rows of H and any vector that is orthogonal to the rows of H is in the row space of G. This means that a codeword c is a codeword if and only if is generated by generator matrix G and satisfies

(22)

this means that

𝐺𝐻𝑇 _{= 0}

Matrix H is called a parity-check matrix. Each row of H corresponds to a parity-check equation and each column of H corresponds to a bit in the codeword.

Example 2. Given the generator matrix from Example 1 which was a (7, 4) linear block code, our corresponding 3 × 7 parity-check matrix is

𝑯 = [1 0 0 1 0 1 10 1 0 1 1 1 0 0 0 1 0 1 1 1 ]

Note that a parity-check equation is used to detect whether each codeword is valid. In order to find if a codeword is valid, a modulo-2 sum of all the codeword bits need to be added and add up to zero. If the parity-check equation is not satisfied, then we can conclude that c is not a valid codeword.

Example 3. The parity-check equations for example 2 are

𝑝0 = 𝑐1+ 𝑐4 + 𝑐6+ 𝑐7 = 0

𝑝₁ = 𝑐₂+ 𝑐₄ + 𝑐₅+ 𝑐₆ = 0 𝑝₃ = 𝑐₃+ 𝑐₅+ 𝑐₆+ 𝑐₇ = 0

Error detection and correction for linear block codes can become quite simple with the use of a parity-check matrix H. Firstly, because we know that each codeword must satisfy

𝑐𝐻𝑇_{= 0 ,}_{we can detect errors in any received word by noting which words do not satisfy}

this equation. Note that after we transmit our codeword c through a noisy channel, we will have the received codeword

𝒓 = 𝒄 + 𝒆

Where e is the error vector, or error pattern, which adds the incorrect bits in our codeword. To determine if the received codeword has errors, we compute the vector

(23)

Which we call the syndrome of r. The syndrome indicates which parity-check constraints are not satisfied by r. If 𝒔 = 0, then we can determine r is a codeword, otherwise,𝒔 ≠ 0 and r is not a codeword. In general, a linear block code can only detect a set of bit error if the error pattern e is another codeword.

To measure the ability of a code to detect errors is through the minimum Hamming distance or simply minimum distance𝑑_𝑚𝑖𝑛. The Hamming distance of a code refers to the number of bits two codewords differ from each other. Minimum distance is technically defined as the smallest Hamming distance between any pair of codewords in a code [3]. A code’s 𝑑_𝑚𝑖𝑛 can tell how many errors t can be detected as long as

𝑡 < 𝑑_𝑚𝑖𝑛

Similarly, we can find how many error bits we can correct by noting that

𝑡 = ⌊𝑑𝑚𝑖𝑛− 1 2 ⌋

2.3.2 Convolutional Coding

A binary convolutional code is defined by three parameters: n, k and m. Like encoders in block codes, k refers to the number of bits from the incoming message u, or the number of input bits going into the encoder, and parameter n refers to the number of bits in codeword c, or the number of output bits from the encoder. The symbols u and c in this case refer to sequences of blocks not a single block like in linear codes. However, each encoder not only depends on the current input bit k message block but also on m previous message blocks. Parameter m refers to the memory registers available in the encoder. Therefore, to encode a message block sequence, a convolutional code takes into consideration the current and previous code bit to create its codeword sequence. Hence, this encoder contains memory and when implemented should be done using sequential logic.

Similar to linear block codes, the code rate R is defined as 𝑅 = 𝑘 𝑛⁄ . It tells us how many output bits do we receive per input bit inserted. In order to encode our input sequence, we connect the memory registers to adders in a particular bit order, which we call the

generator polynomial (g), for that particular bit. This generator polynomial selects the bits that need to be combined to create our output sequence. Figure 2.4 show these polynomials.

(24)

Figure 2.4. A rate 1/3 convolutional code.

A generator polynomial can be thought like a generator matrix in the sense that it “generates” our codeword c. Using Figure 4 as an example, generator polynomial

𝑔1 = (1,1,1), 𝑔2 = (0,1,1) and 𝑔3 = (1,0,1). The combination of these bits compose the

output sequence we plan to obtain. Each memory register can be described by using the terminology ‘D’. We can use a polynomial representation by utilizing D to describe the generator polynomials used. If using Figure 4, for example, our generator polynomials would be 𝑔₁ = 1 + 𝐷 + 𝐷2, 𝑔₂ = 𝐷 + 𝐷2 and 𝑔₃ = 1 + 𝐷.

Aside from memory m, convolutional codes are described through its constraint length L. The constraint length L of a code represents the number of bits in the encoder memory that affect the generation of the n output bits [3]. By definition 𝐿 = 𝑘(𝑚 − 1). When describing the encoding process of convolutional codes, it is best to think about it in terms of state diagrams. In other words, when output bits are calculated they are dependent on current and past inputs on the encoder, similar to that of state diagrams. Therefore, we can know which output bits will be created by noting the different combination that the encoder can create. The number of bit combinations a convolutional encoder can make is defined as:

(25)

This means that our constraint length gives us the amount of combinations we can obtain. Because our goal is to protect our message as much as we can against noise, the more output combinations we have the better the protection. Which means that to increase the number of bit combinations we have to increase our constraint length L and ultimately increase our memory m to have better error correcting [3].

One popular method to decode convolutional codes is using the Viterbi algorithm. This algorithm is used if we want to have a maximum likelihood path decoder which works by tracing a trellis structure. Depending on the received bit pattern, the decoder will decide on the most likely path by using either a minimum Hamming distance or Euclidian distance method. As many decoders, a Viterbi algorithm uses either a hard-decision or soft-decision decoding method. Although different measurements may be taken, the algorithm decision making is still the same. Figure 2.5 shows how a typical decoder trellis diagram looks.

Figure 2.5. Viterbi algorithm example.

2.4 H

ARD

-D

ECISION AND

S

OFT

-D

ECISION

D

ECODING

Recall that when BPSK modulation is used on an AWGN channel with optimum coherent detection and binary output quantization, our bit error probability (BER) for an uncoded BPSK signal becomes

(26)

If binary coding is used, the modulator has only binary inputs (𝑀 = 2). However, if the decoding process is said to be quantized for a binary input, then we would get binary outputs quantized to either a 0 or 1. This type of decoding is considered hard-decision decoding. Due to its implementation simplicity, linear block codes, and sometimes convolutional codes, tend to use hard decision decoding. However, this works well only when the quantization level Q is 𝑄 = 2, i.e. for every binary input we received a binary output. When 𝑄 > 2, or the output is left unquantized, we consider the demodulator to make

soft-decision decoding. This means that instead of having a 0 or 1 value at the output, we will get a continuous-value output. The input in this case is still a binary input but we are just deciding to leave the output values unquantized.

The benefits of having hard-decision over soft-decision and vice versa depends on the type of code implemented and the requirements needed to be met. If implementation

simplicity is required, hard-decision is used, however, if accurate decision making is needed in order to improve error performance then decision will be the target [3]. Although soft-decision decoding is more difficult to implement, its significant performance improvement compared to hard-decision is enough reason to use with codes. Soft-decision decoding is heavily used these days especially with codes which have iterative decoding such as Turbo codes and Low Density Parity Check codes.

2.4.1 Turbo Codes

Turbo codes were created and introduced by Berrou, Glavieux, and Thitimajshima in 1993. The codes are created by combining two or more component codes to different

interleaved version of the same information. Its decoding method involves not only using a hard-decision decoding but also using a soft-decision decoding within its iterative decoding algorithm. To best exploit the information learned from each decoder, the decoding algorithm must effect an exchange of soft decisions rather than hard decisions. For a system with two component codes, the concept behind turbo decoding is to pass soft decisions from the output of one decoder to the input of the other decoder, and to iterate this process several times so as to produce better decisions.

Turbo codes are also known as parallel concatenated convolutional codes (PCCC) because in their implementation two convolutional encoders are used in parallel. Since the

(27)

encoders are parallel, they act on the same information at the same time rather than one encoder processing information and then passing it on to the second encoder. Turbo decoders are based on a soft-in-soft-out (SISO) technique. At the decoder side the systematic input, and the two encoded data sequences from the two encoders are fed as inputs. At first, the decoder tries to decode the various inputs in an order. Then the data is fed back through the feedback path. The decoder iteratively decodes the inputs given to it; after a few iterations we can make a pretty good estimate of data bit that was transmitted, and, as a result of this feedback mechanism [4]. Figure 2.6 shows an example of a Turbo code encoder and decoder.

Figure 2.6. Turbo code encoder and decoder. Encoder on the top and decoder on the bottom.

2.4.2 Low Density Parity Check Codes

Low Density Parity Check codes were discovered in the early 60’s and were re-developed once research began to increase to find codes which could approach Shannon’s limit capacity. At the time when Turbo codes where gaining popularity, Low Density Parity Check codes, or LDPC for short, research began to increase as well. LDPC codes are

(28)

the next chapter we will discuss in detail about LDPC codes and show why they are a good alternative for soft-decision decoding.

(29)

CHAPTER 3 LOW-DENSITY PARITY CHECK CODES

As we discussed earlier, Low Density Parity Check codes, or LDPC codes, are capacity approaching codes that with the use of soft-decision decoding can achieve desirable performance levels. To begin understanding them, we first look into the definition for a regular LDPC code.

3.1 LDPC

D

EFINITION

A parity-check matrix H is defined as a (j, k)2regular LDPC code if the following properties are met:

Each row consists of ‘k’ ones Each column consists of ‘j’ ones

The number of ones in common between any two columns in H should not be any greater than one.

Both ‘j’ and ‘k’ are small compared to the code length, N, and the number of rows in our parity check matrix, H.

The first two points tell us that for our code to be considered regular each of our row weights must equal to ‘k’ and our column weights must equal to j. At the same time, we require that each row, and columns, must have constant non-zero weights to be regular. In other words, to be a regular LDPC code, each row must have exactly k ones and each column must have exactly j ones. If this condition is not met, then our code becomes an Irregular

LDPC code. The last point refers to the requirement that k and j should be smaller than the

2_{Note that we will define the non-zero values, or}_weights_{, of each row and column in our parity check}

(30)

rows and columns, respective, of our parity check matrix. When this property is met, it assures that our parity-check matrix is a ‘sparse’ matrix. This is the main characteristic about LDPC codes and hence where they get their name from, i.e. Low-Density3.

LDPC codes are simply linear block codes with sparse parity-check matrices or, in other words, a parity-check matrix that contains a very small number of non-zero weights, which equal to‘1’ in binary form. This small number of weights is what makes our parity-check matrix sparse and is necessary in order to have efficient soft-decision decoding which increases linearly, in terms of complexity, as code length increases. In Gallager’s paper [1], he showed the advantage of having a sparse H is that it can guarantee that minimum

distance, 𝑑𝑚𝑖𝑛 grows in a linear manner. This means that the sparser our code is, the better

our error detection and correction becomes hence why a classical block code will only work well with an iterative decoding algorithm as long as H can be represented by a sparse parity-check matrix.

3.1.1 Irregular Versus Regular

As defined previously, Irregular LDPC codes are defined when the number of non-zero column weights, j, or row weights, k, are not constant throughout the parity-check matrix. At times, it is convenient to use Irregular compared to Regular codes mainly because of performance conditions. It has been shown that long random Irregular LDPC codes can perform close to Shannon’s limit on very low noise channels. Although it is stated as random, random codes would provide complex hardware designs which would not be optimal for practical use. Therefore a pseudo-random irregular LDPC code construction is followed. In Table 3.1 we show a simple comparison between Irregular and Regular LDPC codes. Although this thesis will focus solely on regular LDPC codes, it is necessary to briefly mention irregular codes which a popular in code construction today.

3_{Because the matrix is sparse, there will be a low number of non-zeros, or ones, available making}_H_“Low

(31)

Table 3.1. Comparison Between Irregular and Regular LDPC Codes

Irregular Regular

Short Code Length

Irregular Codes may develop a small minimum distance 𝑑_𝑚𝑖𝑛 at these levels. Hence, small values

of N are not advised.

Regular and graph based parity-check matrices 𝐻 do well with small 𝑁 and outperform codes

with random structured 𝐻 because it can guarantee girth

and minimum distance properties difficult to achieve

for random codes. Long Code

lengths

They are superior to Regular LDPC codes. This is due to how

as N grows, we can have more sparseness in H therefore, a higher

𝑑𝑚𝑖𝑛 equating to better

performance.

Simulations have shown that Regular based H do not do as well with Long code lengths.

Performance Achieve performance close to channel capacity, however, these

LDPC codes equate poor word error rates and high error floors in

terms of BER. Hence these codes are not desirable in some

applications

Performance under iterative decoding is best when column

weight is restricted to 𝑗 ≥ 3. This condition allows the code

to reach good minimum distance, dmin. Hardware Due to longer lengths and

pseudo-random structures with Irregular Codes, we would require more

computing power making implementation complex.

Advantage in Hardware implementation due to how we

can simplify the iterative decoder.

3.1.2 LDPC Code Rate

For regular LDPC codes, we can make the following relationship:

𝑘𝑀 = 𝑗𝑁

Where M is the parity-check equations in our matrix H. If we assume that our parity-check matrix has full rank we find the code rate to be the following:

𝑅 = 1 − 𝑗 𝑘

This is because when the parity-check matrix is full rank:

(32)

In some cases, our parity-check matrix will not have full rank. Therefore, we can approximate our code rate to be close to our previous definition. When this is the case, we call R the design rate of the code [4].

3.1.3 LDPC Matrix and Graphical Representation

Often we can represent our LDPC code in either a matrix representation or a

graphical representation. If shown in matrix form, we can simply express it in a parity-check matrix form.

Example 1. If I had a code which has a length 6 codeword of

𝑐 = [𝑐₁𝑐₂𝑐₃𝑐₄𝑐₅𝑐₆]

And satisfies the following parity-check equations:

𝑐1 + 𝑐2+ 𝑐4 = 0

𝑐₂+ 𝑐₃+ 𝑐₅ = 0 𝑐₁ + 𝑐₂+ 𝑐₃+ 𝑐₆ = 0

Then we could represent a regular LDPC parity-check matrix in matrix form like:

𝐻 = [ 1 1 0 1 0 0 0 1 1 0 1 0 1 0 0 0 1 1 0 0 1 1 0 1 ]

This matrix has 𝑗 = 2, 𝑘 = 3 and 𝑟𝑎𝑛𝑘₂(𝐻) = 3. Note that each row of H matches the parity check equations that have been satisfied.

Even though a matrix representation seems straightforward, LDPC parity-check matrices are often represented in a graphical form by a Tanner Graph. A Tanner graph is a bipartite graph which is divided into two sets of nodes: Bit nodes and Check nodes. Bit nodes represent the codeword bits, N, in a check matrix and check nodes represent the parity-check equations, M found in H. An edge is defined as the line which connects the bit nodes to each check node corresponding to the bits which are found in the respective parity-check equation. If we look at the parity-check matrix H, the number of edges in a Tanner graph equals the number of 1’s in the parity-check matrix. A cycle in a Tanner graph is when a

(33)

sequence of connected nodes starts and ends in the same node in the graph and contains other nodes no more than once. The length of a cycle is the number of edges it contains, and the

girth of a graph is the size of its smallest cycle. Figure 3.1 shows the Tanner graph for the parity-check matrix shown in example 1 and highlights a 6 length cycle.

Figure 3.1. Tanner graph for a LDPC code. Note the squares represent check nodes and circles bit nodes.4

3.2 M

ESSAGE

-P

ASSING

I

TERATIVE

D

ECODING

Typically, error correction in linear block codes can happen if we apply direct comparison between the received codeword and the transmitted codeword. As explained previously, the syndrome of r, S, can tell us which parity-check constraints are not satisfied if

r does not meet the equation

𝑆 = 𝑟𝐻𝑇

However, this method only works successfully if the number of message bits, K, is relatively small. If this is not the case, error correction and detection can become tedious and complex. For LDPC codes, a class of algorithms were developed so they can perform better than maximum likelihood methods used previously and have reduced complexity. These

4_{Note that that the number of bit nodes in a Tanner graph equal}_N_{, or the number of columns of}_H_{, while}

(34)

types of decoding algorithms are called message-passing algorithms, and their operation can be explained by passing messages along the edges of Tanner Graph [4].

In message-passing algorithms, each node in the Tanner graph works in isolation, only having access to the information contained in the messages on the edges connected to it. These message-passing algorithms are also known as iterative decoding algorithms due to how messages are passed back and forth between bit nodes and check nodes iteratively. This process continues until the decoder converges to a result or until a maximum number

iterations is reached. To understand how this process works, we will look into two types of message passing decoding which are bit-flipping decoding and belief propagation decoding. Bit-flipping involves the use of binary messages passed back and forward between nodes and making a hard-decision after each iteration. The Belief Propagation algorithm, also known as

sum-product algorithm, uses log-likelihood ratios to describe the node messages using sum and product operations. Then, just like in bit-flipping, it will pass these log-likelihood ratios between nodes.

3.2.1 Bit-Flipping Decoding

A bit-flipping algorithm, is the name given to hard-decision message passing algorithms for LDPC codes. Typically, a binary, hard, decision is made by the detector and passed to the decoder. For bit-flipping, binary messages are passed along the Tanner graph edges and makes decision at both the bit and check nodes. The following process shows the process of how bit-flipping decoding works.

3.2.1.1 B

IT

-F

LIPPING

P

ROCESS

Assuming we receive a hard-decision binary channel output y from our channel, i.e. 0 or 1, we will take on the following steps:

Step 1. The first step involves checking if the syndrome vector, S, is zero. In other words, we check if

𝑆 = 𝑟𝐻𝑇

If 𝑆 = 0, then 𝑐 = 𝑟 and we can stop decoding.

If 𝑆 ≠ 0, then we have to consider all the non-zero components of S corresponding to the parity check equation that are not satisfied by the elements of y.

(35)

Step 2. Once step one has been completed, we check which elements in the particular parity-check equation are not satisfied. From this, we correlate them to those in y and update y by flipping those components in y which do not satisfy check equations. Step 3. After the update, we recalculate the syndrome. The whole process is repeated again for a fixed number of iterations or until the syndrome is 0.

3.2.2 Sum-Product Algorithm

As we saw in bit-flipping, we send binary messages to each of the nodes and make hard decisions accordingly to converge to the decoded codeword. Each message in each node is represented by a binary value of either 0 or 1. The sum-product algorithm works similar, however, this algorithm is a soft-decision based message-passing algorithm. This means that instead of binary valued messages, our messages our now represented as probabilities.

The sum-product algorithm accepts the probability of each received bit coming from the channel as input. Each input received, or channel, bit probabilities are called a priori

probabilities. A priori probabilities are the received bit probabilities which come out of the channel and which were known in advanced before the LDPC decoder was operated. Once the a priori probabilities are received, the extrinsic5 information is passed between nodes as probabilities instead as hard-decisions. Finally, the decoder outputs bit probabilities which are called a posteriori probability. The aim of sum-product algorithm is to accomplish two things.

1. To compute the a posteriori probability (APP) for each codeword bit. In other words,

𝑃𝑖 = 𝑃[𝑐𝑖 = 1|𝑠 = 0]

where 𝑠 = 0 refers to the event that all parity-check constraints are satisfied. 2. To select the decoded value for each bit as the value with the maximum APP

(MAP) probability6.

The extrinsic messages 𝐸_𝑗,𝑖and 𝑀_𝑗,𝑖 in a Tanner graph are defined as follows:

5

Extrinsic information refers to the probabilities that come from the check nodes to the bit nodes in a Tanner graph and that does not include the check node probability for the corresponding bit node.

(36)

𝐸𝑗,𝑖= Gives the probability messages sent from check node j to bit node i.

𝑀_𝑗,𝑖 = Gives the probability messages sent from the bit nodes i to the check nodes j

The extrinsic message 𝐸𝑗,𝑖 gives the probability that 𝑐𝑖 = 1 will cause parity-check

equation j to be satisfied. Note that 𝐸_𝑗,𝑖 is not defined if bit 𝑖is not included in check j since no extrinsic information is passed betweennodes i and j in this case. The values sent

represent the probability that a parity-check equation is satisfied if 𝑐_𝑖 = 1.This is represented as:

𝑃_𝑗,𝑖𝑒𝑥𝑡 = 1₂−1₂∏_𝑖′_∈𝐵_𝑗_{,𝑖′≠𝑖}(1 − 2𝑃_𝑗,𝑖𝑖𝑛𝑡′)

Where 𝑃_𝑗,𝑖′ is the current estimate available to check node j of the probability that 𝑐𝑖 = 1.

Note that the probability the parity-check equation is satisfied given 𝑐𝑖 = 0 is (1 − 𝑃𝑗,𝑖𝑒𝑥𝑡).

Although we can express the rest of the algorithm in terms of probabilities, it is conventionally represented in terms of Log likelihood ratios (LLR). Log Likelihood ratios are used to represent the metrics for a binary variable by a single value. Let’s define LLR’s as:

𝐿(𝑥) = log𝑝(𝑥 = 0) 𝑝(𝑥 = 1)

where the probability that our value equals 1, 𝑝(𝑥 = 1) is given by 𝑝(𝑥 = 1) = 1 −

𝑝(𝑥 = 0). The sign of L(x) gives a hard-decision on x and its magnitude shows how confident we are of our hard decision. With this, we can define that

𝐿(𝑥) > 0 The more confidence we have that 𝑝(𝑥) = 0

𝐿(𝑥) < 0 The more confidence we have that 𝑝(𝑥) = 1

The benefit of choosing to represent our probabilities in terms of log likelihood ratios is due to our probability computation. By using LLR representation we can make our decoder hardware implementation less complex by simply using adders instead of multipliers as it is required with single probabilities. This is one of the advantages LDPC codes has which makes it so attractive for use.

(37)

3.2.2.1 S

UM

-P

RODUCT

A

LGORITHM

R

EPRESENTATION

In terms of LLR ratios, our sum- product algorithm is expressed as follows:

1. We begin by defining our extrinsic messages,𝐸_𝑗,𝑖 to be the LLR of the probability that bit i causes parity check j to be satisfied. Therefore, if we want to express our extrinsic messages in terms of log-likelihood ratios we would obtain:

𝐸_𝑗,𝑖 = 𝐿𝐿𝑅(𝑃_𝑗,𝑖𝑒𝑥𝑡_{) = log}1 − 𝑃𝑗,𝑖𝑒𝑥𝑡

𝑃_𝑗,𝑖𝑒𝑥𝑡

2. If we substitute the initial probability equation (𝑃_𝑗,𝑖𝑒𝑥𝑡) into 𝐸𝑗,𝑖 we get

𝐸_𝑗,𝑖 = log 1 2 +12 ∏𝑖′_∈𝐵 (1 − 2𝑃𝑖𝑖𝑛𝑡′ ) 𝑗,𝑖′≠𝑖 1 2 −12 ∏𝑖′_∈𝐵_𝑗_,𝑖′_≠𝑖(1 − 2𝑃𝑖𝑖𝑛𝑡′ )

3. To simplify this previous expression, we will use the relationship

tan 1 2log 1 − 𝑝 𝑝 = 1 − 2𝑝 to make 𝐸𝑗,𝑖 become 𝐸_𝑗,𝑖 = log 1 2 +12 ∏𝑖′_∈𝐵 𝑡𝑎𝑛ℎ(𝑀𝑗,𝑖′/2) 𝑗,𝑖′≠𝑖 1 2 −12 ∏𝑖′_∈𝐵 𝑡𝑎𝑛ℎ(𝑀𝑗,𝑖′/2) 𝑗,𝑖′≠𝑖 where 𝑀_𝑗,𝑖′ = 𝐿𝐿𝑅(𝑃_𝑗,𝑖𝑖𝑛𝑡′) = log 1 − 𝑃_𝑗,𝑖𝑖𝑛𝑡′ 𝑃_𝑗,𝑖𝑖𝑛𝑡′ As an alternative, we could also use the relationship

2 tanh−1_{(𝑝) = 𝑙𝑜𝑔}1 + 𝑝

1 − 𝑝

which will make our extrinsic message be represented as:

𝐸𝑗,𝑖 = 2 tanh−1

∏

_𝑖′_∈𝐵

𝑗,𝑖′≠𝑖𝑡𝑎𝑛ℎ (

𝑀_𝑗,𝑖′

2 )

4. Each bit has access to the input a priori LLR, 𝑟_𝑖 and the LLR’s from every connected check node. The total LLR of the i-th bit is the sum of these LLR’s.

(38)

𝐿_𝑖 = 𝐿𝐿𝑅(𝑃_𝑖𝑖𝑛𝑡_{) = 𝑟}

𝑖+

∑

_𝑗∈𝐴_𝑖𝐸𝑗,𝑖

5. Aside from the extrinsic messages from check node to bit node, it is convenient we define the messages sent from the bit nodes to the check nodes, we will call this variable 𝑀_𝑗,𝑖. Note that 𝑀_𝑗,𝑖 is not the full LLR value for each bit. This is because it includes the check node information

3.2.2.2 S

UM

-P

RODUCT

A

LGORITHM

P

ROCESS

Let’s summarize how the message-passing algorithm process works.

Step 1. The decoder receives the channel output values which become initialized as the a priori LLR’s. These values are initially set to the bit nodes as

𝑀_𝑗,𝑖= 𝑟_𝑖

Step 2. Each of these bit node values is sent to their corresponding check node. This is done according to the location of the non-zero element in our parity-check matrix. (Ex: If there is a ‘1’ in code bit, or matrix column, 3 and parity-check equation, or

row, 2, then we will be sending the value at bit node 3 to check node 2)

Step 3. Once at the check node, it will compute the extrinsic message,𝐸_𝑗,𝑖 for each message received using its LLR equation. Note that, if we were to be computing the extrinsic message 𝐸_2,3 for example, the extrinsic probability from the 2nd check to the 3rd bit will not use the bit node 𝑣₃ to compute its value.

Step 4. After all the appropriate extrinsic probabilities are calculated, they will be sent back to the corresponding bit nodes. At this point we will test both the intrinsic, or the LLR value from the channel, and extrinsic messages and calculate the total LLR for bit i.

Step 5. Finally, we make a hard-decision on the received bits given by the sign of the LLR’s. This hard-decision gives us an estimated codeword 𝒛 which we check if it’s a valid codeword if its syndrome 𝑠 = 0.

Step 6. If 𝑠 = 0 then we can consider 𝒛 to be a valid codeword. If this is not the case, then we will use equation

𝑀𝑗,𝑖= ∑𝑗′_∈𝐴_𝑖_{,𝑗′≠𝑗}𝐸𝑗′,𝑖+ 𝑟𝑖

to set the bit node probabilities and repeat the process until we converge to the correct codeword or reach the maximum number of decoding iterations.

3.3 LDPC

P

ARITY

-C

HECK

M

ATRIX

C

ONSTRUCTION

When we are actually considering defining or modifying an LDPC code we are actually considering changing our parity-check equation. That is because the construction of

(39)

sparse parity-check matrices are what differentiates classical linear block codes to LDPC codes. Therefore, the construction of our parity-check matrix is critical to our LDPC code design. While thinking on how to construct a parity-check matrix, one needs to keep in mind the properties LDPC codes should meet. For example we may want to keep in consideration the code length N, avoiding 4 or 6 cycles in our Tanner graph, or just achieving a desired minimum distance. These properties should also match the design criteria designers want to accomplish such as near-capacity code performance, efficient encoding and decoding or lowering error-floor rates.

When considering practical LDPC codes we must consider long code construction. Normally, randomly constructing a parity-check matrix almost always produces a good code compared to structured parity-check matrices at higher code lengths. Although performance close to capacity can be accomplished, implementation of a random code is very complex for practical applications and shortening the code is required, therefore, H matrices are pseudo-randomly constructed instead. That is, these codes are constructed pseudo-randomly but certain undesirable configurations, such as 4-cycles, are either avoided during construction or removed afterwards.

The removal of cycles are important to consider especially when we use a message passing algorithm as a soft-decision decoder. When cycles are present, the probabilities between nodes become more correlated. This is even greater when the length of the cycle is small, i.e. 4 or 6 length. This correlation tends to lead in a negative impact in decoding performance, for when cycles are available, the probability correlations may not allow the decoding to converge to the original codeword. Hence why if we recall from the LDPC code definition, property 3 is stated such that we can avoid cycles of any form. Although this is our goal, it is almost impossible to avoid cycles in LDPC codes. Yet not always is it good to avoid cycles. It turns out that avoiding completely small cycles can also have an effect in the minimum distance of our code so a combination of code construction and flexible decoding methods needs to be achieved [4]. Therefore, as a rule of thumb it is always best to avoid small cycles, especially 4-cycles, and the best way to start with them is by considering different types of LDPC code constructions. The next two sections describe two methods of constructing LDPC codes.

(40)

3.3.1 Gallager Codes

The original LDPC codes presented by Gallager were regular and defined by a

banded structure in 𝐻 [5]. For a given choice of j and k, Gallager developed a construction of a class of linear codes specified by their parity-check matrices that are represented as

𝐻 = [ 𝐻₁ 𝐻2 ⋮ 𝐻_𝑑 ]

where 𝐻_𝑑 refers to a submatrix 𝜇 × 𝜇𝑘 where k refers to the row weight of each matrix and

𝜇 refers to an integer which is greater than 1. When 𝜇 > 1 H has a very small density and becomes a sparse matrix. Each row of a submatrix has k 1’s and each column of a submatrix

contains a single 1, or 𝑗 = 1. Therefore, each submatrix has a total of 𝜇𝑘 1′𝑠 . Note that the total number of ones in H equals 𝑘𝑗𝜇, and the total number of entries in H is 𝜇2𝑗𝑘, and the density of H becomes_𝑘1. The overall parity check matrix H is of size 𝜇𝑗 × 𝜇𝑘

In this representation, the rows of Gallager’s parity-check matrix are divided into j

sets with 𝑀/𝑗 rows in each set. The first set of rows contains k consecutive ones ordered from left to right across the columns. Every other set of rows is a randomly chosen column permutation of this first set. Consequently, every column of H has a ‘1’ entry once in every

one of the j sets [5].

Example 2. A length 12 (3, 4) –regular Gallager parity-check matrix is represented as

𝐻 = [ 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 0 1 0 0 1 0 0 0 1 0 0 0 1 0 0 0 0 1 1 0 0 0 1 0 0 0 1 1 0 0 0 1 0 1 0 1 0 0 1 0 0 1 0 0 1 0 0 0 1 0 0 0 1 0 1 0 0 1 0 0 0 1 0 1 0 0 0 1 0 0 1]

This example shows a few things. First, no two rows in a submatrix of H has any 1-component in common and no two columns of a submatrix H has more than one 1 in common. Second, this matrix shows that Gallager construction is of ‘regular’ form due to how each row and column has a constant amount of ones. Third, and most important, one

(41)

could notice that the following submatrices after 𝐻₁ are simply column permutations of the first submatrix. Therefore, to construct a Gallager code, one simply must define 𝐻₁ to define the overall parity-check matrix. However, as trivial as it may sound, Gallager did not provide a method for choosing column permutations of the submatrix 𝐻1 to form the other

submatrices such that the overall matrix H gives an LDPC code with good minimum distance and meets the structural properties required. Therefore, computer searches are needed to find good LDPC codes especially when there is a long code length and in terms of hardware implementation will have low-complexity encoders.

3.3.2 Repeat-Accumulate Codes

Another type of LDPC construction, which is the main focus of this thesis, is called repeat-accumulate (RA) codes. RA codes are constructions based on having our parity-check matrix be split in a systematic form in which one part of the matrix is constructed in a step pattern form with each column being weight-2.This part of H comprises of the last M

columns of the matrix [6]. The benefit of having these codes is that it makes for a systematic code and for encoding simplicity.

Example 3. We will show how a length 12 rate ¼ repeat-accumulate code as

𝐻 = [ 1 0 0 1 0 0 0 0 0 0 0 0 1 0 0 1 1 0 0 0 0 0 0 0 0 1 0 0 1 1 0 0 0 0 0 0 0 0 1 0 0 1 1 0 0 0 0 0 0 0 1 0 0 0 1 1 0 0 0 0 0 1 0 0 0 0 0 1 1 0 0 0 1 0 0 0 0 0 0 0 1 1 0 0 0 1 0 0 0 0 0 0 0 1 1 0 0 0 1 0 0 0 0 0 0 0 0 1]

This construction is that of a systematic RA code where the first three columns correspond to the message bits. The fourth column of H is considered to be the first parity-bit which can be used to encode our message. The next chapter goes in details about RA codes.

3.4 LDPC

E

NCODING

Typically in linear codes we find that the encoding process is a simple process to follow compared to the decoding process it needs to take on. However, when dealing with

(42)

LDPC codes this changes due to its sparsely parity-check matrix. This leads to tedious and complex versions of encoding LDPC codes. The next sections will give a brief example of how researchers have gone about encoding LDPC codes.

3.4.1 Simple Encoding

To begin encoding linear block codes we have to go back to the following equation

𝒄 = 𝒖𝐺

in which G is the generator matrix of the code. For a binary code with K message bits and length N codewords, a generator matrix is a 𝐾 × 𝑁 matrix. If my code is systematic, the generator matrix will include an 𝐾 × 𝐾 identity matrix, named 𝐼𝑁−𝐾, for its first K columns

of the matrix. Each generator matrix has a parity-check matrix H which is considered to be the null space of G. Knowing this we could relate a generator matrix to its parity-check matrix as

𝐺𝐻𝑇 _{= 𝟎}_{(mod 2)}

where 0 is an 𝐾 × 𝑚 all-zero matrix.

In general, a generator matrix for a code with parity-check matrix H can be found by applying Gauss-Jordan elimination to H. H will be in the form of

𝐻 = [𝐴 𝐼𝑁−𝐾]

where 𝐼_𝑁−𝐾 is a 𝐾 × 𝐾 identity matrix of order 𝑁 − 𝐾 and A is an (𝑁 − 𝐾) × 𝐾 binary matrix.

Based on this definition we could also define our generator matrix to be

𝐺 = [𝐼_𝐾 𝐴𝑇_]

therefore, by using the Gauss-Jordan elimination method, we could encode and decode our code simply by providing the code’s G and H matrices. However, this method has a few drawbacks which has led to look for various alternatives and which makes LDPC encoding a complex task to accomplish. First, by finding G with this method we cannot guarantee our generator matrix will be sparse. It is known that what makes these codes attractive is its ability to converge to accurate codewords by applying a soft-decision iterative decoding

(43)

method. However, for this to work we need to ensure that G and H be sparse. Due to this issue the matrix multiplication 𝒄 = 𝒖𝐺 will have a complexity of order 𝑁2 operations, where

N is the number of bits in a codeword. This leads to our second drawback which is implementation complexity. With an operation order of 𝑁2 we make our encoder

implementation very complex, especially for LDPC codes. As we know, LDPC codes work better with larger N lengths, which can range from hundreds to thousands. Therefore this method is not optimal to use. For arbitrary parity-check matrices it is a good approach to try and avoid constructing G altogether and instead use a back-substitution with H method to encode or pursue encoding LDPC codewords by using RA codes. In the next chapter we will go in detail on how to encode using systematic RA codes.

(44)

CHAPTER 4 REPEAT-ACCUMULATE CODES

Repeat-Accumulate codes, or RA codes, are serially concatenated codes in which the outer code is a rate 1/q repetition code and the inner code is a convolutional code with a rate 1/1+D. This 1/1+D convolutional code simply outputs the modulo-2 sum of the current input bit and previous output bit. In other words, it receives the input values and sums, or

accumulates, with all stored inputs. Due to its process, this block is called an accumulator

block. Just as Turbo codes, in between the two constituent codes there is an interleaver block which allows our RA code to produce a high-weight output thereby avoiding low-weight codewords. Figure 4.1 shows the basic structure of an RA code. When talking about an RA code, we are actually referring to the encoder design of the code, as Figure 4.1 shows. From the figure it is observed that an RA code has a similar characteristic to that of Turbo codes which is why many refer to RA codes as “Turbo-like” codes. With RA codes the only concern is the construction for its encoder block due to having the advantage of working extremely well with either a Turbo or LDPC decoder. However, to begin understanding RA codes, it will be beneficial to look into the systematic and non-systematic aspects of the codes as well as their parity-check matrix construction.

(45)

4.1 S

YSTEMATIC AND

N

ON

-S

YSTEMATIC

C

ODES

RA codes can transmit both the message bits and parity bits in its codeword creating a

systematic RA code. Typically, when dealing with systematic RA codes, our encoder block diagram includes an extra block we call a combiner. This block is placed in between the interleaver and the accumulator as shown in Figure 4.2. A rate-a combiner simply modulo-2 sums each set of a bits coming into it. When adding the combiner to the mix, we typically clump the combiner and the accumulator to be the inner code while our combiner is still is considered the outer code. Our interleaver in this case will not be affected and will still be in between the inner and outer codes. The purpose of having a combiner is that it allows us to represent and decode our RA codes as LDPC codes. In other words, by having a systematic codeword, we will be able to use a message passing algorithm as a decoding option.

Figure 4.2. Block diagram structure of a systematic RA code.

When we deal with non-systematic RA codes, our decoder is typically decoded as a Turbo code. However, the implementation of such a codeword would require the removal of our combiner block. It isn’t that a non-systematic RA code cannot be encoded with a

combiner, however, decoding would not be possible. This is because given only the parity bits from the channel, there is no way the decoder can determine the values of the a bits that are summed by the combiner to produce each parity bit [4]. This is why we will mostly focus on the systematic approach in this thesis. Figure 4.3 shows the block diagram for a non-systematic RA code.

A Study of Low Density Parity-Check Codes Using Systematic Repeat-Accumulate Codes

ABSTRACT OF THE THESIS

TABLE OF CONTENTS

LIST OF TABLES

LIST OF FIGURES

ACKNOWLEDGEMENTS

CHAPTER 1

INTRODUCTION

1.1

O

T

CHAPTER 2

BACKGROUND

2.1

D

C

2.2

M

/D

2.2.1 Channel

2.2.2 Demodulator

2.3

C

C

2.3.1 Linear Block Codes

2.3.2 Convolutional Coding

2.4

H

-D

S

-D

D

2.4.1 Turbo Codes

2.4.2 Low Density Parity Check Codes

CHAPTER 3

LOW-DENSITY PARITY CHECK CODES

3.1

LDPC

D

3.1.1 Irregular Versus Regular

3.1.2 LDPC Code Rate

3.1.3 LDPC Matrix and Graphical Representation

3.2

M

-P

I

D

3.2.1 Bit-Flipping Decoding

3.2.1.1

B

-F

P

3.2.2 Sum-Product Algorithm

3.2.2.1

S

-P

A

R

∏

∑

3.2.2.2

S

-P

A

P

3.3

LDPC

P

-C

M

C

3.3.1 Gallager Codes

3.3.2 Repeat-Accumulate Codes

3.4

LDPC

E

3.4.1 Simple Encoding

CHAPTER 4

REPEAT-ACCUMULATE CODES

4.1