Decoding Algorithms

For erasure channels, there exist essentially two different decoding algorithms, namely the greedy, yet suboptimal belief propagation (BP) decoding algorithm and the optimal, yet computationally more complex maximum likelihood (ML) decoding algorithm. Both will be briefly reviewed in the following.

Belief Propagation Decoding

The belief propagation (BP) decoding algorithm is also known as greedy decoding or, as is the case here, in conjunction with erasure channels it is known as peeling decoding. It is best explained by using the decoding graph, i.e. the bipartite graph that represents the relationship between the input symbols and the received output

2.1 Luby Transform (LT) Codes 19 symbols. A detailed example of BP decoding of an LT code over F₄ can be found in Figure 2.6 illustrating the individual steps of the BP algorithm listed below.

1. Find an output node of degree one. If none can be found, decoding fails and eventually more output nodes have to be collected to restart decoding.

2. Propagate the selected output node’s value to the connected input node by dividing the output node’s value by the weight of the connected edge.

3. Remove the used output node and its edge from the decoding graph.

4. Propagate the recovered input node’s value to all connected output nodes. This is done by adding to each output node’s value the input node’s value multiplied by the weight of the respective connecting edge.

5. Remove all used edges from the decoding graph. If all input nodes have been recovered, decoding ends successfully. If there still exist undecoded input nodes, continue with step 1.

Since each edge in the decoding graph is used at most once, the complexity of recovering an information word by BP decoding is proportional to the average row weight ¯d and the number of input nodes k, i.e. O( ¯dk). This low complexity makes BP decoding very interesting for practical applications.

BP decoding for erasure channels corresponds to solving a consistent system of linear equations solely by means of back substitution. Thus, it is clear that if the corresponding matrix G_R can be transformed into an upper triangular matrix just by column and row permutations, BP decoding succeeds and delivers the optimal ML solution. If on the other hand G_R can be upper triangulated only partially by column and row permutations, BP decoding can recover merely those input symbols that are associated with the upper triangular part of G_R and which do not depend on only optimally, i.e. ML decodable symbols. The other input symbols can, if at all, be solved solely by optimal decoding.

For properly designed codes and especially for large input sizes, the suboptimal BP algorithm works remarkably well. However, for small to medium input sizes the variance of the random process that creates degree one output nodes during decoding is too high. So decoding very often fails due to the lack of degree one output nodes and this leads to a too high overhead of additionally received symbols that is required for successful decoding, and thus in this domain of input sizes, optimal decoding is left as the only sensible choice.

. . . α2 0 1 1 α2 0 α α2 α α2 1 1 α α2 α α2 1

(a) Encoding of k = 3 input symbols and transmission. 1 0 α ? ? ? ? α α2 1 1 α α2 α (b) Reception of at least nR = k = 3

unerased output nodes.

1 0 α ? ? ? α α2 1 α α2 α

(c) Remove erased output nodes and their edges from the decoding graph. 1 1 0 α ? ? α α2 1 α α2 α

(d) Find an output node of degree one and propagate its value to the con- nected input node by dividing the value by the weight of the corre- sponding edge. Remove the used output node and its edge from the graph. 1 1 ? ? α2 α α2 1 α α2

(e) Propagate the value of the recov- ered input node to all connected output nodes by multiplying its value by the weights of the respec- tive edges and add these values to the values of the respective output nodes. Remove the used edges.

1 1 α2 ? ? α2 α α2 1 α α2 1

(f) Since no degree one output node has been created in the decoding process, additional output nodes have to be collected.

Figure 2.6: Exemplary belief propagation decoding of an LT code over F₄. The illus- tration is continued in Figures 2.6(g) – 2.6(j) on the next page. Ancillary addition and multiplication tables for F₄ are provided in Table 2.1.

2.1 Luby Transform (LT) Codes 21 1 1 ? ? α2 α α α2 1 α α2 1

(g) Upon reception of the new output node, propagate values of already re- covered input nodes to the new out- put node as done in Figure 2.6(e) and remove all used edges.

α2 1 1 ? α2 α α α2 1 α α2

(h) Since a new degree one output node has been created in the last step, up- date the connected input node as in Figure 2.6(d). α2 ? 1 0 0 α α2 1 α

(i) Perform an output node update as in Figure 2.6(e) and remove all used edges.

α2 0 1

0 0

α2 α

(j) Either of the two degree one out- put nodes can be used to perform the final input node update as de- scribed in Figure 2.6(d).

Figure 2.6: Exemplary belief propagation decoding of an LT code over F₄. Continuation from previous page.

Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.

Table 2.1: Addition and multiplication tables for F4, which is defined by the primitive

polynomial a(ξ) = 1 + ξ + ξ2 _{in F}

2. The primitive element is denoted α.

+ 0 1 α α2 0 0 1 α α2 1 1 0 α2 α α α α2 0 1 α2 α2 α 1 0 · 0 1 α α2 0 0 0 0 0 1 0 1 α α2 α 0 α α2 1 α2 0 α2 1 α

Maximum Likelihood Decoding

The maximum likelihood (ML) decoding algorithm for erasure channels coincides with the maximum a posteriori (MAP) decoding algorithm. It is the optimal decoding algorithm in terms of decoding failure probability. ML decoding of LT codes on erasure channels is equivalent to solving a system of nR consistent linear

equations in k unknowns over a finite field Fq. If the respective coefficient matrix G_R has full column rank, i.e. rank(G_R) = k, an arbitrary information word can be uniquely determined. If rank(G_R) < k, the solution of G_Rx = y_R spans a (k− rank(G_R))-dimensional vector space.

Such a system can be solved for instance by means of the well-known Gaussian elimination (GE) algorithm. Since GE has a relatively high computational com- plexity of O(k3) per information word, ML decoding is practically only applicable to codes with short to medium input sizes. However, as codes for short to medium input sizes are highly relevant for low-delay applications and have not been well investigated so far, this thesis concentrates on the finite length analysis of LT codes and Raptor codes under ML decoding. Moreover, LT ensembles are proposed that have a near-optimal erasure correction performance under ML decoding, given certain design constraints.

Efficient Maximum Likelihood Decoding Algorithms

Besides GE there exist several algorithms that achieve the ML erasure correc- tion performance but eventually exploit certain properties (cf. e.g. [Wie86, LO91, PS92, Cop93, RU01, Cop94, BM04, PNF04, SLK05, LMPC09, PLMC12]) such as the sparseness of the matrix G_R to reduce the number of operations and thus the com- putational complexity. The main difference to GE is the scheduling according to which operations are carried out. Nonetheless, the complexity of these algorithms is still upper bounded by O(k3).

A simple, yet effective method is the consecutive use of BP decoding followed by ML decoding by means of GE. If degree one output nodes exist, BP decoding can reduce the number of unknowns from k to ˜k with complexity O( ¯d(k− ˜k)) and the remaining ˜k unknowns can be solved by GE with complexity O(˜k3). Although this concatenation of BP and ML decoding may not be the fastest algorithm, it is the method of choice in this thesis. Despite its simplicity it contains already the ingredient for the main part of the speed-up, i.e. reducing the original num- ber of unknowns as far as possible by means of a low complexity algorithm and solve the rest by GE. Also it facilitates comparability with results from the liter- ature. Moreover, it is only used to quantify relative decoding speeds of given LT code ensembles, which should vary only slightly when applying other efficient ML decoding algorithms.

2.1 Luby Transform (LT) Codes 23 k nR nR − ˜ k ˜ k ˜ k k− ˜k 0 0 (a) k nR nR − ˜ k ˜ k ˜ k k− ˜k B C 0 0_(n R−˜k)טk I˜_kטk (b)

Figure 2.7: Decoding matrix G_R during inactivation decoding (cf. [SL11]).

So far, in practical systems such as the 3GPP Multimedia Broadcast/Multicast Service (MBMS) [3GP13], where Qualcomm’s Raptor10TM [LSWS07] and RaptorQTM [QUA10,LSW+11,SL11] codes are used, another efficient ML decoding algorithm named inactivation decoding [SLK05] is employed.

In contrast to the previously described simple method of first applying BP decoding and then GE, inactivation decoding consists of multiple steps, of which the first one is to rearrange the system of linear equations. This is accomplished by means of row and column permutations on the decoding matrix G_R to transform it into a matrix that is partitioned like in Figure 2.7(a) and to maximise the size ˜k of the lower triangular submatrix. The original decoding matrix G_R is supposed to have a low density. In the next step, row additions are performed in order to eliminate the entries on the left below the main diagonal of the decoding matrix. The result of this step is sketched in Figure 2.7(b). By row additions, the submatrices B and C usually become dense, which is indicated by a darker grey. The subsystem of linear equations defined by submatrix C is solved using GE. It can only be solved if C has full column rank. Then, the recovered unknowns can be used to solve the remaining unknowns using back substitution, i.e. BP decoding, which eliminates the entries in submatrix B. A more detailed description of inactivation decoding can be found in [SLK05, SL11].