Transactions Papers
Dynamics and Performance Analysis of Analog
Iterative Decoding for Low-Density
Parity-Check (LDPC) Codes
Saied Hemati, Member, IEEE, and Amir H. Banihashemi, Senior Member, IEEE
Abstract—Conventional iterative decoding with flooding or
par-allel schedule can be formulated as a fixed-point problem solved iteratively by a successive substitution (SS) method. In this paper, we investigate the dynamics of a continuous-time (asynchronous) analog implementation of iterative decoding, and show that it can be approximated as the application of the well-known successive relaxation (SR) method for solving the fixed-point problem. We observe that SR with the optimal relaxation factor can consider-ably improve the error-rate performance of iterative decoding for short low-density parity-check (LDPC) codes, compared with SS. Our simulation results for the application of SR to belief prop-agation (sum-product) and min-sum algorithms demonstrate im-provements of up to about 0.7 dB over the standard SS for ran-domly constructed LDPC codes. The improvement in performance increases with the maximum number of iterations, and by accord-ingly reducing the relaxation factor. The asymptotic result, corre-sponding to an infinite maximum number of iterations and infini-tesimal relaxation factor, represents the steady-state performance of analog iterative decoding. This means that under ideal circum-stances, continuous-time (asynchronous) analog decoders can out-perform their discrete-time (synchronous) digital counterparts by a large margin. Our results also indicate that with the assumption of a truncated Gaussian distribution for the random delays among computational modules, the error-rate performance of the analog decoder, particularly in steady state, is rather independent of the variance of the distribution. The proposed simple model for analog decoding, and the associated performance curves, can be used as an “ideal analog decoder” benchmark for performance evaluation of analog decoding circuits.
Index Terms—Analog decoding, asynchronous iterative methods, belief propagation (BP), dynamics of iterative de-coding, iterative dede-coding, low-density parity-check (LDPC) codes, min-sum (MS), successive relaxation (SR), successive sub-stitution (SS), sum-product.
Paper approved by C. Schlegel, the Editor for Coding Theory and Techniques of the IEEE Communications Society. Manuscript received November 22, 2004; revised May 3, 2005. This work was supported in part by Communications and Information Technology Ontario (CITO), and in part by the Natural Sciences and Engineering Research Council of Canada (NSERC). This paper was pre-sented in part at the 41st Annual Allerton Conference on Communication, Con-trol and Computing, Urbana, IL, October 2003, in part at the 2004 IEEE Inter-national Symposium on Information Theory, Chicago, IL, July 2004, and in part at IEEE Globecom, Dallas, TX, November 2004.
S. Hemati was with the Broadband Communications and Wireless Systems (BCWS) Centre, Department of Systems and Computer Engineering, Carleton University, Ottawa, ON K1S 5B6, Canada. He is now with the University of Ottawa, Ottawa, ON K1N 6N5, Canada.
A. H. Banihashemi is with the Broadband Communications and Wireless Sys-tems (BCWS) Centre, Department of SysSys-tems and Computer Engineering, Car-leton University, Ottawa, ON K1S 5B6, Canada (e-mail: [email protected]).
Digital Object Identifier 10.1109/TCOMM.2005.861668
I. INTRODUCTION
R
ECENTLY, iterative decoding methods have attracted much interest because they can be used for decoding the best-known coding schemes, such as turbo codes and low-den-sity parity-check (LDPC) codes. The need for floating-point computations and the iterative nature of these algorithms have motivated some very recent research on their analog electronic implementation [1]–[11]. The main motivation for performing iterative decoding in the analog domain instead of conventional digital circuits is to improve the power/speed ratio and to min-imize area consumption in very large scale integration (VLSI) chips. Also, analog circuits generate less noise compared with conventional digital circuits, and therefore, are attractive when one has to use a decoder in the proximity of certain noise-sen-sitive circuits, such as low-noise amplifiers. In this paper, we demonstrate yet another, less expected, advantage for analog decoding, i.e., an intrinsically better performance.Analog signal processing has been greatly influenced by Mead’s interesting idea of exploiting semiconductor physics and transistor characteristics for performing analog computa-tions in artificial neural networks [12]. While biological neural networks are essentially continuous-time and asynchronous, conventional iterative decoders are not [13]. In practice, these algorithms have been conventionally implemented by digital synchronous circuits, where a clock signal provides global timing among circuit elements and computational modules. In simulations also, the assumption of synchronization among different modules is always made, making the results only applicable to synchronous circuits. So far, to the best of our knowledge, no theoretical analysis for the dynamics of contin-uous-time asynchronous analog iterative decoding1 has been
provided.
Despite the missing theory for the dynamics of iterative coding on asynchronous machines, proof-of-concept analog de-coder chips for very short codes have already been designed and fabricated [1]–[11]. In [1], Hagenauer et al. designed analog de-coders for a tailbiting convolutional code, and turbo codes with (7,4) Hamming codes as component codes. They also solved the system of nonlinear differential equations corresponding to
1For the sake of brevity, in the rest of the paper, we may use the term “analog
decoding” and “digital decoding” to refer to “continuous-time asynchronous analog iterative decoding,” and “discrete-time synchronous iterative decoding,” respectively.
analog decoding circuits using the Runge–Kutta method, and reported bit-error rate (BER) curves that are slightly inferior to the conventional discrete-time simulation results. Solving the differential equations, however, appeared to be very time con-suming, and would limit the simulations to only simple codes [1], [2]. In [3] and [4], the fabrication of maximum a posteriori (MAP) decoders for short tailbiting convolutional codes were reported. In [5], the authors implemented one stage of the alpha metric computation of the Bahl–Cocke–Jelinek–Raviv (BCJR) algorithm for an eight-state convolutional code. In [2], [6], and [10], implementation of turbo decoders with interleaver lengths 16, 16, and 40 were reported, respectively. In [7] and [8], the au-thors propose analog circuits for implementation of the min-sum (MS) algorithm [13]. In [9], a MAP decoder for the (8,4) Ham-ming code was fabricated. The design of an analog decoder for convolutional duo-binary codes for the DVB-RCS standard was presented in [11].
While in [3] and [4], the authors report measured performance for their chips which closely follows the simulated BER, in [6], [9], and [10], a degradation is observed, compared with the sim-ulated BER. In all these references, simsim-ulated BER is obtained based on a discrete-time synchronous model. In [1] and [2], in addition to discrete-time simulations, the authors also pro-vide continuous-time simulation results for analog decoding cir-cuits which closely follow the measured and discrete-time sim-ulated BER curves. In [10], the authors propose a discrete-time model for analog turbo decoding. To perform this, they approx-imate the operation of each computational module by a linear first-order filter. They then include mismatch in this model and observe 0.3–0.5 dB degradation compared with the conventional synchronous decoding. This brings the mismatched curves in the close vicinity of the measured BER for analog decoding. For turbo codes, [2] and [11] provide examples where analog de-coding slightly (by about 0.1 dB) outperforms digital dede-coding with a finite number of iterations.
In this paper, we provide a framework for the analysis of analog iterative decoding of LDPC codes, and show that the dynamics of analog decoding is, in fact, different from that of digital decoding. For this, the dynamics of analog decoding is modeled by a system of first-order nonlinear differential equa-tions. This system can then be solved numerically. We show that, under certain conditions, by applying the forward Euler method to the system of differential equations, analog decoding can be approximated as the application of the well-known successive relaxation (SR) method [14]–[16] to the fixed-point problem of iterative decoding. The approximation error tends to zero as the relaxation factor approaches zero. The value of plays an important role in the performance and complexity of the SR method. The set of recursive equations resulting from the application of SR to the fixed-point problem of iterative de-coding provides a general framework, which embodies not only analog decoding ( ), but also digital decoding based on the flooding (parallel) schedule [13]. The latter corresponds to the application of the well-known successive substitution (SS) [16] to the fixed-point problem of iterative decoding and is a special case of SR when .
We demonstrate that the error-rate performance of SR im-proves by increasing the maximum number of iterations and
by correspondingly reducing . The optimal value of , corre-sponding to minimum BER, appears to be always less than one. In the limit, as and , the performance of SR asymptotically tends to the steady-state performance of analog decoding. Our simulation results indicate that this asymptotic performance is considerably better than the performance at
, which corresponds to conventional digital decoding. For the tested LDPC codes, up to about 0.7 dB improvement for belief propagation (BP) [13] is observed. This improvement in per-formance is due to a better convergence rate (larger percentage of convergence to correct codewords) for analog decoding. Our results also show that under the assumption of symmetric trun-cated Gaussian distribution for the delays between the computa-tional modules of an analog decoder, the error-rate performance, especially in steady state, is independent of the variance of the delay distribution.
Adopting alternative numerical methods to improve conver-gence in iterative decoding of turbo codes was proposed in [17], and was also studied in [18]. In [18], the authors identified turbo decoding as fixed-point iteration and examined alternative nu-merical methods, including SR, for solving the corresponding fixed-point problem. The final conclusion of [18], however, was that for turbo codes, the conventional SS is the superior choice in most situations. This is in clear contrast with our results for LDPC codes, where SR, in particular, provides considerable im-provement over SS for both the BP and MS algorithms. On the same theme of research, in the past few years and mainly in the context of short LDPC codes, there have been many attempts at improving the performance of iterative decoding algorithms [19]–[22]. We have noted that many of these proposed methods can be interpreted as the application of different iterative nu-merical methods for solving the fixed-point problem in hand. For instance, in [19]–[22], variants of the “damped substitution method” [18] have been used for improving the performance of the MS and BP algorithms.
In addition to showing that analog decoding can outperform digital decoding, one of the main contributions of this paper is to provide a new benchmark, as “ideal” analog decoding, for the performance of analog decoding circuits. This would replace the conventional benchmark of discrete-time synchronous decoding. Although in this paper, we are mainly concerned about the sta-tistical behavior of decoders, in general, and their error probabil-ities, in particular; the proposed model can also be used to an-alyze analog decoders for given instances of received values at the output of the channel. Such analysis includes transient and steady-state responses of the decoder, and can be used to effi-ciently study the behavior of an analog decoder once an estimate of the delay distribution between variable and check modules is available. Our simple model of analog decoding can also be used in designing decoders. As our proposed model is much simpler than the circuit-based model for analog decoding, it will signifi-cantly speed up any analysis or design process, and can be used even when the complexity of circuit simulation is prohibitive.
The SR method can also be implemented on a synchronous discrete-time platform. For smaller values of , the resulting im-provement over the conventional SS would be at the expense of a larger average number of iterations for convergence. It is im-portant to note that the application of SR to turbo codes (on a
synchronous discrete-time platform) was studied in [18], and unlike our results for LDPC codes, no considerable improve-ment over SS was observed.
The organization of this paper is as follows. In Section II, we discuss iterative decoding as a fixed-point problem, and explain how conventional iterative decoding is equivalent to the applica-tion of the so-called SS method to the fixed-point problem. Sec-tion III is on analog implementaSec-tion of iterative decoding. We propose our simple model for analog decoding and the corre-sponding set of first-order differential equations in this section. In Section IV, we use the model of Section III to analyze the dynamics of analog decoding by numerically solving the differ-ential equations. We demonstrate the accuracy of the model by comparing its results with those obtained by circuit simulations for a small decoder. We also explain how the application of for-ward Euler to the system of differential equations is equivalent to the application of the so-called SR method to the fixed-point problem of iterative decoding. In Section V, we present our sim-ulation results and conclude, among other things, that analog de-coding outperforms conventional digital dede-coding. Section VI is devoted to concluding remarks.
II. ITERATIVEDECODING
Iterative decoding can be considered as an iterative numerical method for solving the complex problem of decoding by finding the most probable transmitted codeword or bit, based on the observation of channel outputs, noise and channel characteris-tics, and the structure of the code. For the sake of simplicity, we consider binary linear block codes. The codes are repre-sented by “Tanner graphs” [23], which are constructed based on parity-check equations defining the code. There are two sets of nodes in a Tanner graph: 1) variable nodes (VAR) representing code bits; and 2) check nodes (CHK) representing parity-check equations. Suppose that the Tanner graph has edges, and that the iterative algorithm, which, for example, could be BP or MS, has converged to a solution. Also denote the incoming and out-going messages to and from CHK nodes by column vectors and , respectively. These vectors have length . We then have
VAR:
CHK: (1)
or equivalently
(2) where and represent variable node and check node opera-tions, respectively, and is the received vector and has length . Operations and are determined by the decoding algorithm, the type of messages, and channel characteristics [13]. Equation (2) is a fixed-point problem [16], as the function maps to itself.
Ideally, we would like the iterative algorithm to converge to a fixed-point , and the hard-decision assignment corresponding to to be the most likely transmitted codeword or to corre-spond to the bit-level MAP decoding. Although this is the case for cycle-free graphs, for graphs with cycles, which inevitably appear in representing good codes, this may not be the case [13]. In practice, in many cases, as soon as the hard-decision assign-ment at variable nodes is a codeword, the iterative algorithm is
stopped. This stopping criterion appears to result in very good performance for practical codes. One can think of iterative de-coding as finding a solution for (2) based on iterative numerical methods. In particular, for the commonly used flooding or par-allel schedule [13], iterative decoding can be expressed by the following iterative formula which is based on the application of the well-known SS2method [16] to (2):
(3) A flooding schedule can be easily implemented on a syn-chronous discrete-time machine where the global timing for message passing is fully controlled by a clock signal. In this case, messages are passed from VAR nodes to CHK nodes, and subsequently from CHK nodes to VAR nodes in a synchronized iterative fashion. There are, however, many advantages in imple-menting iterative algorithms on asynchronous machines using continuous-time analog VLSI [1]–[11]. In the following section, we discuss the dynamics of continuous-time analog iterative de-coding.
III. ANALOGIMPLEMENTATION OFITERATIVEDECODING
The dynamics of iterative decoding on continuous-time (asynchronous) machines is different from that of conventional discrete-time (synchronous) ones. In particular, it is fair to say that an analog decoder, even if it is designed based on a flooding-schedule framework [2], [6]–[8], [10], may no longer support the (synchronized) flooding schedule. This is because different computational modules may have different processing delays. Another reason is that the output of each module (before settling down to its steady-state value) is immediately propa-gated through the edges of the graph, and is applied to the input of the next module in the absence of global synchronization. The latter is prone to further deviate the decoding process from synchronous decoding due to different propagation delays among different pairs of VAR and CHK modules. This effect is particularly profound for large circuits with a large range of delays among different modules. Even if we assume that the processing and propagation delays are the same, our analysis indicates that continuous-time analog decoding will still have a different behavior, compared with conventional discrete-time decoding.
Assuming that we use analog circuits to implement iterative decoding in a parallel-scheduling framework, we can approxi-mate the dynamics of the system by the following differential equations:
(4) where and are diagonal matrices which contain different time constants, representing different propagation delays affecting the corresponding and variables, re-spectively. This model is derived based on Fig. 1, where the propagation delay due to interconnections between the nodes, corresponding to the edges of the Tanner graph, as well as the resistive and capacitive loads of the nodes, are simplified and modeled by lumped capacitors and resistors based on Elmore’s
2SS is also referred to in the literature as “successive approximation,”
Fig. 1. Model for continuous-time analog iterative decoding.
delay approximation [24]. Processing delays or dead-time of the nodes, which indicate the lagging time of the outputs with respect to the inputs, have also been taken into account
and represented by column vectors and
. Vectors and
in (4) are defined as and
, respectively. In general, pro-cessing delay can be different for nodes of different types (CHK or VAR) or nodes with the same type but different degrees.
Equation (4) expresses the dynamics of continuous-time analog iterative decoding by a pair of interrelated first-order nonlinear differential equations. If the analog decoder con-verges to a fixed point, then the left-hand side of these equations
becomes zero, and vectors and converge
to fixed vectors and , which are independent of , , and . In this case, equations in (4) reduce to (1), and thus to (2). Equation (4), in general, is very difficult to solve. To sim-plify the analysis, we ignore processing delays compared with propagation delays ( ). This may be justified for practical analog decoders, which are comprised of many processing modules and long interconnections between these modules. The dominance of the propagation delay would be even more profound for subthreshold complementary metal–oxide–semiconductor (CMOS) decoders, which are in-herently slower than standard CMOS or BiCMOS decoders. To further simplify the problem, we replace the pair of equations in (4) with the following first-order differential equation, which is based on the application of Elmore’s delay approximation to represent the total round-trip delay in one iteration (Fig. 2):
(5) We will demonstrate in the next section through an example that this significantly simplified model still provides a close ap-proximation for the behavior of analog decoding.
In this paper, we consider BP in the likelihood ratio (LR) domain and MS in the log-likelihood ratio (LLR) domain. BP [1]–[6], [9]–[11] and MS [7], [8] are the only iterative algo-rithms that have been implemented in analog electronics so far. For these algorithms, function in (5) corresponds to the fol-lowing VAR and CHK operations, respectively (the operations are shown for only two inputs. For a larger number of inputs, a cascade of two-input blocks can produce the output).
Fig. 2. Simplified model for continuous-time analog iterative decoding.
A. BP in the LR Domain
General form of the messages: .
VAR: ( is the initial message from
the channel).
CHK: .
B. MS in the LLR Domain
General form of the messages: .
VAR: ( is the initial
mes-sage from the channel).
CHK: .
In the above equations, and are the probabilities of a bit being 0 or 1, respectively, and , for , and , otherwise. Furthermore, with a slight abuse of notations, we have used and to also denote the variable and check node operations for nodes with two inputs, in addition to the notation in (1).
IV. ANALOGIMPLEMENTATION OFITERATIVEDECODING
For a given nonlinear function , it is usually not possible to solve the initial-value problem (5) analytically.3Numerical
methods should then be used instead. There are several numer-ical methods that can be used to solve (5), each with different stability, accuracy, and complexity. In the following, we use for-ward and backfor-ward Euler methods [15], [25]. These methods are, respectively, described by the following equations:
(6) (7)
where and are the estimates of at time
instants and (for a small value of ), respectively.
3In this paper, we assume that before applying the initial values corresponding
to the received vectorR to the decoder, the decoder is in the steady state corre-sponding to uniform a priori probabilities, i.e.,X(0 ) = 0 and X(0 ) = 1, in the LLR and LR domains, respectively, where 0 and 1 are the all-zero and all-one vectors, respectively. We also assume that at timet = 0, the initial values corresponding to the received vectorR are applied to the decoder. These initial values immediately appear on the edges at the output of VAR nodes, and correspond toX(0 ).
The method of forward Euler (6) is an explicit integration method and is particularly attractive for its simple implemen-tation. However, forward Euler, as well as any other conven-tional explicit integration method, is prone to stability problems (Dahlquist’s first barrier theorem) [26], and requires a suffi-ciently small time step ( ) in order to remain stable. Backward Euler, on the other hand, is one of the linear implicit integration methods that are absolutely stable (Dahlquist’s second barrier theorem) [26]. The downside of backward Euler, however, is its high complexity. For example, in (7), for finding an estimate of
, a nonlinear equation needs to be solved.
To demonstrate that Fig. 2 provides a good model for analog it-erative decoding, we use it to find the time response of the (analog) MS algorithm applied to the Tanner graph of the (7,4) Hamming code. This Tanner graph was also used in [8] to demonstrate the functionality of the designed circuits for VAR and CHK modules in that paper. Using the exact same input vector as the one used in
[8] ( ), we apply Euler’s backward
method of (7) to solve the differential equation (5), while function is replaced by the combination of VAR and CHK operations for the MS algorithm in the LLR domain, given in the previous section.4In our simulations, we have assumed that all the edges
have the same time constant, which is estimated to be equal to ns. This estimate is based on the circuits of [27], which are much faster than those reported in [8]. In Fig. 3(a) and (b), we have shown the LLR values corresponding to the seven VAR nodes of the Tanner graph. For each node, this is the sum of all the incoming LLRs and the initial value of the node. These outputs are denoted by “O” and shown by thicker lines. For comparison, in Fig. 3(a) and (b), we have also given similar curves obtained by circuit simulation using Cadence Spectre tools and based on the circuits in [27]. As one can see, the two sets of curves are close, in general, especially for steady-state responses. We also obtained the outputs of the MS decoder for the (7,4) Hamming code and by starting from the same initial conditions as those used in [8] (note that in [8], ). Again, although the circuits of [8] are very different than those of [27], the obtained curves are very close to those obtained by circuit simulation and presented in [8, Figs. 7–9].This is an evidence for the universality of the model introduced in Fig. 2 and (5), i.e., for an analog decoder, as long as the processing function , initial values , and matrix of time constants are known, one can use (5) to obtain an ap-proximation for the output signals of the decoder independent of the circuitry. Our examples show that for the tested MS analog decoders, these approximations are close to circuit simulations. It should be noted that the circuit simulation curves in Fig. 3(a) and (b) and also the curves in [8] are obtained by performing simulation on a circuit with a few thousand short-channel metal–oxide–semiconductor field-effect transis-tors (MOSFETs). This is while, using our model, the simulation is greatly simplified and is reduced to solving only a first-order differential equation with 12 variables ( ). Clearly, our simple model, which is mostly circuit-independent, is not capable of incorporating the precise characteristics of dif-ferent circuit elements, and is thus less accurate in predicting
4We have also applied higher order linear multistep numerical integration
methods to this problem, and the results are very close to those obtained by backward Euler.
Fig. 3. Outputs of MS analog decoder based on circuit simulation (o) and the simple model of Fig. 2 (O). (a) Outputs 1–3. (b) Outputs 4–7.
the actual behavior of an analog decoder integrated circuit. Nonetheless, our model can serve as a simple tool for analyzing ‘‘ideal” continuous-time analog decoding, and can be used to provide an estimate of the behavior of practical analog decoders, including important characteristics such as speed of convergence.
We also applied the forward Euler method of (6) to the same example, and observed that as long as we choose small enough such that , the results are almost the same as those obtained by backward Euler.5The application of
for-ward Euler to (5) is interesting not only because it is arguably the simplest numerical method that can be used to solve (5), but also because in the discrete-time domain, it reduces to the fol-lowing recursive equation:
(8)
5While there does not seem to exist any general result for the stability of
forward Euler when applied to nonlinear differential equations, our experiments with this algorithm for solving (5) indicates that choosing1t= 0:1 results in time responses very close to those obtained by1t= = 0:1.
which, as we will see shortly, will, in turn, reduce to a well-known iterative method for solving the fixed-point problem of (2). In (8), and of (6) are replaced by and , respectively. As , and thus the diagonal elements of
, go to zero, (8) provides a better estimate of the solution to (5). In particular, as goes to zero and as the number of iterations goes to infinity, we obtain the steady-state response of an analog decoder.
Here, we would like to emphasize that (8) can also be used as an iterative algorithm on a synchronous discrete-time plat-form. For this scenario, we only consider the case where all the diagonal elements of matrix are equal.6We use the
no-tation to denote this common value. Equation (8) can then be rewritten as
(9) which is equivalent to a well-known iterative method for solving the fixed-point problem of (2). This method is referred to as SR [14]–[16], and is an alternate to the SS method, described by (3). Parameter is called the “relaxation factor.” For , SR reduces to SS.
In the literature, SR is often used to iteratively solve systems of linear equations [14]–[16]. Parameter is then optimized to max-imize the speed of convergence. The optimal value of , which is often difficult to determine, appears to be between 1 and 2 [15]. In fact, in some literature, the term “overrelaxation” is used for , and is referred to as “underrelaxation” [14]. While overrelaxation helps with the speed of convergence for the cases where convergence is guaranteed, underrelaxation is often used for nonlinear equations to help with increasing the chance of convergence [14]. Thus, in (9), as we tend to zero, we not only obtain a better estimate of analog decoding, but also expect to improve the chance of convergence for SR.
One should note that in this paper, we are mainly interested in the convergence properties of SR from a statistical point of view, i.e., when vector changes randomly. To be more precise, we are interested in the statistics on how often SR can find the transmitted codeword or the transmitted bit correctly; or equiv-alently, what the average probability of error for SR is.
Finally, we would like to make the remark that when the prop-agation delay is the dominant delay factor in an analog iterative decoder, i.e., , the same approach taken to derive (8) and (9) from (5) can also be used to derive the SR version of the more complex model formulated in (4). This can be simply verified by rewriting (4) as follows:
where
6Note that as we will see in the next section, the assumption of equal time
constants has no effect on the statistical steady-state response of (8).
and and are zero and identity matrices, respec-tively. While in this case, the number of differential equations is twice that of (5), we have not observed noticeable change, com-pared with the results obtained by the simpler model of (5).
V. SIMULATIONRESULTS
We perform simulations on an optimized irregular (1268, 456) LDPC code [28] and a regular (504, 252) code [29] over an additive white Gaussian noise channel with binary phase-shift keying (BPSK) modulation. In our simulations, we stop itera-tions as soon as a codeword is detected or when a maximum number of iterations is reached. We have investigated the per-formance of each code for both BP and MS algorithms. BP and MS simulations are performed in the LR and LLR domains, respectively. For all the reported results, we have either 200 codeword errors per simulation point, or codewords have been simulated. In the following, we analyze the performance of the SR algorithm and analog decoding by first simulating (8) for different delay distributions, and then by focusing more on the constant delay scenario.
A. Effect of Delay Distribution
To study the effect of the delay distribution on the error-cor-recting performance of the decoder, we use (8) for updating the messages. For delay distributions of the edges, we assume symmetric truncated Gaussian distribution in the interval
, where and . The
first constraint is enforced to make sure of a good accuracy for the forward Euler algorithm, while the second constraint is derived based on the lengths of wires in a practical layout for an LDPC decoder [30] (note that the delay is proportional to the square of the wire length [24]). We consider Gaussian distributions with four different variances. The normalized values of standard deviations are selected as 0, 19.8, 198, and 1980. The first value ( ) corresponds to fixed delay for all the edges, while the last value results in an almost uniform distribution of delays in the interval.
Fig. 4 shows the BER curves of BP applied to the (504, 252) code for the four delay distributions and three dif-ferent values of , the maximum number of iterations, i.e.,
. The curves for have also
been obtained, but are close to those for , and thus are not reported. The word-error rate (WER) curves follow the same trends as the BER curves do. These results indicate that the performance of the analog decoder improves with increasing , which is equivalent to increasing the processing time of the decoder. This performance improvement, however, saturates and reaches a steady state when the processing time is large enough. Our results show that when enough time is given to the decoder to settle down in its steady-state performance, this performance is almost independent of the delay distribu-tion. As can be seen in Fig. 4, for , the four curves corresponding to different values of practically coincide. Even for smaller values of , the effect of on performance is rather small. Similar observations are made for the application
Fig. 4. Effect of different delay distributions on BP error rate of regular (504, 252) LDPC code.N is the maximum number of iterations. =1t = 0: “__”, =1t = 19:8: “_ _}”, =1t = 198: “_. ”, and =1t = 1980: “__3”.
of BP to the (1268, 456) code and also for the MS algorithm applied to both codes.
It is worth mentioning that one should not interpret the above results as the independence of the response of analog decoding from delay distribution. First, it is easy to show that the response of the decoder (even the steady-state response) to particular in-stances of input vector would change, depending on the delay distribution. So the above results are valid in a statistical con-text. Second, the results are obtained based on the assumption of truncated Gaussian distribution for delays. One may be able to find other, probably less realistic, delay distributions where the statistical behavior of the decoder (including its steady-state behavior) would be different from the one described above. B. Comparison Between SR and SS Methods
In this part, we consider SR based on (9) for updating the mes-sages. To compare the performance of iterative decoding based on SR (which represents the performance of analog decoding when is small) with that of the conventional SS method (which reflects the performance of digital decoding), in Fig. 5(a), we show the BER curves of the (1268, 456) code decoded by re-laxed BP (SR applied to BP) versus the relaxation factor for different values. There are two sets of curves in Fig. 5(a), corresponding to maximum number of iterations equal to 200 and 10 000. It can be seen that for a large range of , SR provides improvement over SS ( ), and that the improve-ment, in general, is increased by reducing and increasing .
The performance for and can be
considered as a good approximation for the steady-state perfor-mance of the continuous-time analog implementation of BP de-coder. The BER curve for this case is presented in Fig. 6, along with the curve for SS (discrete-time synchronous BP decoding based on flooding). The maximum number of iterations for SS is also chosen to be . As can be seen, SR outper-forms SS by up to 0.5 dB. Similar improvement is observed for WERs. From Fig. 5(a), one can see that for SR, the performance
gap between and is very small over the
Fig. 5. BER curves of (1268, 456) LDPC code versus the relaxation factor forN = 200 (_._) and 10 000 (___) and different E =N values, decoded by a) Relaxed BP, b) Relaxed MS.
range of , which includes (SS). Over this
range, one can then obtain close to the full potential of SR by selecting . Fig. 7 shows the average number of iter-ations versus for relaxed BP with at different values. As can be seen, the average number of iterations remains almost unchanged over the range of , and increases by decreasing below 0.3. This indicates that there is a complexity cost associated with the better performance of SR on a synchronous discrete-time platform only for smaller values of . For continuous-time decoding, iterations disappear and are replaced by (or are translated to) the processing time of the de-coder.
From Fig. 5(a), one can see that for , the perfor-mance begins to degrade when is reduced below 0.1. This can be explained based on the results on the convergence rate of the algorithm. As can be seen in Fig. 7, by decreasing below a certain range, the algorithm requires, on average, more itera-tions to converge to a solution. Limiting the maximum number
Fig. 6. BER curves for the (1268, 456) code, decoded by relaxed BP (N = 10 000, = 0:05, }), conventional BP with flooding schedule (N = 10 000, r), relaxed MS (N = 10 000 and = 0:3, o), and conventional MS with flooding schedule (N = 10 000, ).
Fig. 7. Average number of iterations versus at different E =N values for the (1268, 456) LDPC code (N = 10 000), decoded by relaxed BP (___) and relaxed MS (-.-).
of iterations to a small value, such as 200, would then reduce the convergence instances for the algorithm, and hence, increase the error probability.
At this point, to better position the improvement obtained by SR, we compare it with the result of [31] (BP + order 1 with maximum of 50 iterations), which, for this code, improves over conventional BP with flooding by about 0.3 dB at BER
[32]. This is about 0.2 dB less than the improvement that SR provides over conventional BP. Note that SR is much less com-plex than the algorithm of [31].
Results of relaxed BP for the (504, 252) code follow the same trends as those for the (1268, 456) code. The BER curve for
relaxed BP with and is given in Fig. 8,
along with the curve for conventional BP with .
Fig. 8. BER curves for (504, 252) LDPC code. Relaxed BP (N = 10 000, = 0:05, }), conventional BP with flooding schedule (N = 10 000, r), relaxed MS (N = 10 000, = 0:4, o), and conventional MS with flooding schedule (N = 10 000, ).
At low error rates, relaxed BP outperforms conventional BP by about 0.7 dB.
Trends similar to those observed for the application of SR to BP are also observed for MS with SR. The BER results of the (1268, 456) code for relaxed MS with 200 and 10 000 versus are given in Fig. 5(b) for different values of . As can be seen, the performance difference between 200 and 10 000 is much larger than the corresponding difference for BP in Fig. 5(a). This is consistent with the fact that MS usually requires a larger number of iterations compared with BP for convergence (compare the curves for relaxed MS with those for relaxed BP in Fig. 7). It is only over the range of that the performance gap between and is rather small, and thus, close to the full potential of the algorithm can be achieved by only selecting . As Fig. 7 shows, over this range, the average number of iterations is, in fact, reduced by decreasing . Fig. 5(b) also shows that the optimal value of , which minimizes BER, is a decreasing function of . Similar to BP, the trend seems to indicate that as tends to infinity, optimal tends to zero. This asymptotic solution corresponds to the steady-state performance of continuous-time analog decoding. The same trends are observed for the application of relaxed MS to the (504, 252) code. Figs. 6 and 8 show the BER curves of both codes for relaxed MS with optimal when . For comparison, the curves for conventional MS with flooding and are also given. As can be seen, improvements of about 0.5 dB are achievable at large values. Similar improvements are observed for WERs. This improvement in decoding performance comes with the added benefit of a decreased average number of iterations for convergence on a synchronous discrete-time platform.
It is important to note that in our simulations, all the errors were detectable, and the performance improvement of SR over SS at a given is due to the higher rate of convergence to the right codewords.
VI. CONCLUSIONS
In this paper, we model continuous-time asynchronous analog iterative decoding by a set of first-order differential equations. Using this model, we approximate analog decoding as the numerical method of SR applied to the fixed-point problem of iterative decoding. We then show that SR provides considerable performance improvement over the conventional numerical approach of SS (associated with conventional iterative decoding on a discrete-time synchronous machine). For the tested LDPC codes, improvements up to 0.7 dB in for a given BER are observed. The improvement increases, and the approximation error for the estimation of the performance of analog decoding decreases, as the maximum number of iterations tends to infinity and as the relaxation factor goes to zero. This means that analog decoding can not only increase the ratio of speed to power consumption, but also provide a better performance, compared with conventional digital decoding.
Our results show that the performance of analog decoding, particularly in steady state, is rather independent of the distribu-tion of delays among the computadistribu-tional modules. This is based on the assumption of truncated Gaussian distribution for delays. The effect of other delay distributions on performance is to be investigated.
In our discussions, we assumed that the analog circuits are ideal, in the sense that they have no mismatch and enjoy un-limited accuracy and dynamic range. One important contribu-tion of this paper is thus to provide an ‘‘ideal” analog decoding benchmark. One can then compare the measured performance of analog decoders against this benchmark, instead of the com-monly used synchronous discrete-time benchmark.
In addition to the analysis of analog decoding, our work also suggests a general framework for improving iterative decoding algorithms on graphs with cycles. On such graphs, SS does not necessarily converge to the optimal MAP or maximum-likelihood solution, and the application of SR with on a synchronous discrete-time platform can improve the performance. Our results of SR for the tested codes indicate that for a given maximum number of iterations, choices of exist that improve the performance of conventional iterative decoding ( ) with smaller or almost the same average number of iterations. As an example, for the tested (1268, 456) code and BP in the LR domain with , by selecting , about 0.4 dB improvement in at a BER of about is obtained with almost the same average number of iterations. For the same code with MS in the LLR domain and , the choice of
results in about 0.35 dB improvement in at a BER of about , with the average number of iterations reduced by a factor of about 1.2.
ACKNOWLEDGMENT
The authors wish to thank the anonymous reviewers for their helpful comments which improved the presentation of this paper. They also wish to thank the Associate Editor for handling this paper and his constructive comments.
REFERENCES
[1] J. Hagenauer, E. Offer, C. Méasson, and M. Mörz, “Decoding and equal-ization with analog nonlinear networks,” Eur. Trans. Telecommun., vol. 10, pp. 659–680, Nov./Dec. 1999.
[2] J. Hagenauer, M. Moerz, and E. Offer, “Analog turbo networks in VLSI: The next step in turbo decoding and equalization,” in Proc. Int. Symp.
Turbo Codes, ENST, Bretagne, France, 2000, pp. 209–218.
[3] M. Moerz, T. Gabara, R. Yan, and J. Hagenauer, “An analog 0.25-m BiCMOS tailbiting MAP decoder,” in Proc. IEEE Solid-State Circuits
Conf., San Francisco, CA, Feb. 2000, pp. 356–357.
[4] H.-A. Loeliger, F. Lustenberger, M. Helfenstein, and F. Tarköy, “Prob-ability propagation and decoding in analog VLSI,” IEEE Trans. Inf.
Theory, vol. 47, no. 2, pp. 837–843, Feb. 2001.
[5] A. F. Mondragón-Torres, E. Sánchez-Sinencio, and K. R. Narayanan, “Floating-gate analog implementation of the additive input soft-output decoding algorithm,” IEEE Trans. Circuits Syst. I, vol. 50, no. 10, pp. 1256–1269, Oct. 2003.
[6] V. C. Gaudet and P. G. Gulak, “A 13.3-Mb/s 0.35-m CMOS analog turbo decoder IC with a configurable interleaver,” IEEE J. Solid-State
Circuits, vol. 38, no. 11, pp. 2010–2015, Nov. 2003.
[7] S. Hemati and A. H. Banihashemi, “Full CMOS min-sum analog itera-tive decoder,” in Proc. IEEE Int. Symp. Inf. Theory, 2003, p. 347. [8] , “Iterative decoding in analog CMOS,” in Proc. 13th ACM Great
Lakes Symp. VLSI, Washington, DC, 2003, pp. 15–20.
[9] C. Winstead, J. Dai, S. Yu, C. Myers, R. R. Harrison, and C. Schlegel, “CMOS analog MAP decoder for (8,4) Hamming code,” IEEE J.
Solid-State Circuits, vol. 39, no. 1, pp. 122–131, Jan. 2004.
[10] D. Vogrig, A. Gerosa, A. Neviani, A. G. i. Amat, G. Montorsi, and S. Benedetto, “A 0.35-m CMOS analog turbo decoder for the 40-bit rate 1/3 UMTS channel code,” IEEE J. Solid-State Circuits, vol. 40, no. 3, pp. 753–762, Mar. 2005.
[11] M. Arzel, C. Lahuec, M. Jézéquel, and F. Seguin, “Analogue decoding of duo-binary codes,” in Proc. Int. Symp. Inf. Theory Appl., Parma, Italy, Oct. 2004, pp. 10–13.
[12] C. Mead, Analog VLSI and Neural Systems. Reading, MA: Addison-Wesley, 1989.
[13] N. Wiberg, Codes and decoding on general graphs. Linköping, Sweden: Linköping Univ., 1996.
[14] J. M. Ortega and W. C. Rheinboldt, Iterative Solution of Nonlinear
Equa-tions in Several Variables. New York: Academic, 1970.
[15] D. M. Young and R. T. Gregory, A Survey of Numerical
Mathe-matics. Reading, MA: Addison-Wesley, 1973, Series in Mathematics. [16] C. T. Kelley, Iterative Methods for Linear and Nonlinear
Equa-tions. Philadelphia, PA: SIAM, 1995.
[17] T. Richardson, “The geometry of turbo-decoding dynamics,” IEEE
Trans. Inf. Theory, vol. 46, no. 1, pp. 9–23, Jan. 2000.
[18] P. Moqvist and T. M. Aulin, “Turbo-decoding as a numerical analysis problem,” in Proc. IEEE Int. Symp. Inf. Theory, 2000, p. 485. [19] J. Chen and M. P. C. Fossorier, “Near-optimum universal
belief-propa-gation-based decoding of low-density parity-check codes,” IEEE Trans.
Commun., vol. 50, no. 3, pp. 406–414, Mar. 2002.
[20] , “Density evolution for two improved BP-based decoding algo-rithms of LDPC codes,” IEEE Commun. Lett., vol. 6, no. 5, pp. 208–210, May 2002.
[21] M. R. Yazdani, S. Hemati, and A. H. Banihashemi, “Improving belief propagation on graphs with cycles,” IEEE Commun. Lett., vol. 8, no. 1, pp. 57–59, Jan. 2004.
[22] J. Zhao, F. Zarkeshvari, and A. H. Banihashemi, “On implementation of min-sum algorithm and its modifications for decoding LDPC codes,”
IEEE Trans. Commun., vol. 53, no. 4, pp. 549–554, Apr. 2005.
[23] R. M. Tanner, “A recursive approach to low-complexity codes,” IEEE
Trans. Inf. Theory, vol. IT-27, no. 9, pp. 533–547, Sep. 1981.
[24] J. M. Rabaey, A. Chandrakasan, and B. Nikolic, Digital Integrated
Cir-cuits, A Design Perspective, 2nd ed. Englewood Cliffs, NJ: Prentice-Hall, 2003.
[25] J. D. Lambert, Numerical Methods for Ordinary Differential Systems:
The Initial Value Problem. New York: Wiley, 1973.
[26] G. Dahlquist, “A special stability problem for linear multistep methods,”
J. BIT Numer. Math., vol. 3, pp. 27–43, 1963.
[27] S. Hemati and A. H. Banihashemi, “Full CMOS Min-Sum Analog Iter-ative Decoder,” U.S. Patent (Pending), Application 10/832806, Apr. 27, 2004.
[28] Y. Mao and A. H. Banihashemi, “A heuristic search for good low-den-sity parity-check codes at short block lengths,” in Proc. IEEE Int. Conf.
[29] D. J. MacKay. Encyclopedia of Sparse Graph Codes. [Online]. Avail-able: http://www.inference.phy.cam.ac.uk/mackay/codes/data.html. [30] A. J. Blanksby and C. J. Howland, “A 690-mW 1 Gb/s 1024-b rate-1/2
low-density parity-check code decoder,” IEEE J. Solid-State Circuits, vol. 37, no. 3, pp. 404–412, Mar. 2002.
[31] M. P. C. Fossorier, “Iterative reliability-based decoding of low-density parity-check code,” IEEE J. Sel. Areas Commun., vol. 19, no. 5, pp. 908–917, May 2001.
[32] , Private Communication, Univ. Hawaii at Manoa, Dept. Elect. Eng., Honolulu, HI, May 2003.
Saied Hemati (S’01–M’06) was born in Isfahan,
Iran, in 1970. He received the B.Sc. and the M.Sc. degrees in electronics engineering from the Isfahan University of Technology, Isfahan, Iran, in 1993 and 1997, respectively, and the Ph.D. degree in electrical engineering from the Department of Systems and Computer Engineering, Carleton University, Ottawa, ON, Canada, in 2005.
From 1997 to 1999, he was with the Isfahan Op-tics Centre, Integrated Electronics Industry (IEI), and from 1999 to 2001, he was with the Isfahan Telecom-munication Company, both located in Isfahan, Iran. Currently, he is a Postdoc-toral Fellow at the University of Ottawa, Ottawa, ON, Canada. His research interests include the theory of operation and the design and implementation of continuous-time analog and optical iterative error-correcting decoders.
Dr. Hemati is the recipient of several awards and scholarships, including the Senate medal for outstanding academic achievement at Carleton University, a Natural Sciences and Engineering Research Council of Canada (NSERC) post-doctoral fellowship, a Communications and Information Technology Ontario (CITO) research excellence award, an Ontario Graduate Scholarship, and an Ontario Graduate Scholarship in Science and Technology.
Amir H. Banihashemi (S’90–A’98–M’03–SM’04)
was born in Isfahan, Iran. He received the B.A.Sc. degree in electrical engineering from the Isfahan University of Technology (IUT), Isfahan, Iran, in 1988, and the M.A.Sc. degree in communication engineering from Tehran University, Tehran, Iran, in 1991, with the highest academic rank in both classes. He received the Ph.D. degree in 1997 from the De-partment of Electrical and Computer Engineering, University of Waterloo, Waterloo, ON, Canada.
From 1991 to 1994, he was with the Electrical En-gineering Research Center and the Department of Electrical and Computer Engi-neering, IUT. In 1997, he joined the Department of Electrical and Computer En-gineering, University of Toronto, Toronto, ON, Canada, where he was a Natural Sciences and Engineering Research Council of Canada (NSERC) Postdoctoral Fellow. He joined the Faculty of Engineering, Carleton University, Ottawa, ON, Canada, in 1998, where at present he is an Associate Professor in the Department of Systems and Computer Engineering and the Director of the Broadband Com-munications and Wireless Systems (BCWS) Centre. His research interests are in the general area of digital and wireless communications and include coding, information theory, and theory and implementation of iterative coding schemes. Dr. Banihashemi held two Ontario Graduate Scholarships for international students between 1994 and 1997, while working toward the Ph.D. degree. He has served as an Editor for the IEEE TRANSACTIONS ONCOMMUNICATIONSsince May 2003. He is a member of the Board of Directors for the Canadian Society of Information Theory.
