Suppose that a message m(x) can be expressed in the exponential form
m(x) =e− f (x), where f(x) has first and second derivatives. We can expand f(x)in a second-order Taylor’s series at a given point x∗,
f(x)≈ f(x∗) + f0(x∗)(x−x∗) + f ”(x∗)(x−x∗)2. (3.33)
Using this result, a Gaussian message approximating m(x) can be obtained
as
mG(x)∝ exp{−f0(x∗)(x−x∗)−f ”(x∗)(x−x∗)2} ≈m(x). (3.34) This method is one of the main contributions of Paper F.
6
Other Approximate Message Passing Algorithms
In our work we have focused on the message passing methods and the approximation techniques discussed in the previous sections of this chap- ter. There are, however, other alternative methods that can be used for the design of iterative signal processing algorithms. We close this chapter by shortly mentioning two methods that have attracted considerable interest from the research community in recent years: approximate message passing (AMP) [32], and its more general form, generalized AMP (GAMP) [33].
The AMP algorithm was proposed as a computationally efficient and ac- curate tool to perform inference on signal models of the form
y= Ax+w (3.35)
where the entries of a unknown vector x are observed through a known mixing matrix A of large dimensions, with some additive white Gaussian
noise (AWGN) w. The mixing matrix is typically a dense matrix, so that the factor graph representation of the model is densely connected graph. Keeping this factor graph representation in mind, the AMP algorithm can be interpreted as EP message passing [34] on the densely connected graph, in which the large dimensions of A are used to simplify the algorithm using large system argumentations. GAMP is derived based on similar principles, but overcoming the assumption of AWGN channel. Both techniques have been extensively applied since they were proposed [14, 15, 35], especially within the context of compressed sensing and sparse estimation problems.
We have used GAMP to benchmark some of our proposed receiver de- signs, as e.g. in Paper C. We also embedded it into the BP-MF framework to design a low-complexity OFDM receiver in Paper E.
Chapter 4
Design of Wireless Receivers:
State-of-the-Art and Thesis
Contributions
With the principles of variational inference and message passing algorithms laid out in Chapters 2 and 3, we proceed in this chapter to describe their application to the design of receiver algorithms in digital communications systems. We start with a short description of the problem of receiver design for digital communications. After this generic discussion, we review in more depth the specific problems that have been approached in this thesis, detail- ing the state-of-art and our contributions to solve them. These are collected in Papers A-F, and constitute the main contribution of this thesis.
Digital communications deal with the transmission of binary information from a transmitter to a receiver. The transmitter converts the binary infor- mation to an analogue waveform and sends this waveform over a physical medium, such as a wire or open space, which we call the channel. At the receiver, the received signal appears distorted by channel and interference and is further corrupted due to thermal noise. Not only has the receiver to recover the original binary information, but it must also deal with channel and interference effects, thermal noise, and synchronization issues. In ad- dition, channel coding is often used to mitigate the effects of intersymbol interference (ISI), so the particular code structure needs to be accounted for as well. In such a context, iterative signal processing based on message pass- ing techniques emerges as a feasible tool for receiver design, as it allows to perform approximate inference in large probabilistic systems including both continuous and discrete random variables.
al. [36] proposed an encoding structure concatenating two simple encoders (turbo-codes) in parallel, associated with a feasible decoding procedure (turbo- decoding) that exchanges soft information between two soft-in-soft-out (SISO) decoders operating in parallel. The drastic performance improvement achieved with this turbo-code and turbo-decoder has lead to a paradigm shift in the coding community. It has also sparkled feverish activities worldwide [37–41] aiming at generalizing the turbo-principle to incorporate –in addition to the decoding procedure– other functionalities of receivers, such as channel esti- mation, channel equalization, synchronization, multi-user interference can- cellation, etc. Well-investigated applications today are turbo-equalization as well as joint channel estimation and decoding in multi-user systems. It has been recently recognized that turbo-algorithms are particular instances of the variational Bayesian method. For instance, Berrou’s turbo-decoding is known to be an instance of BP algorithm [42]. Other principles, such as VMP or EP, have also been widely applied to design iterative receiver structures.
In this thesis, we focus on the design of iterative receivers for different communication systems integrating tasks such as channel estimation, phase noise estimation, equalization, and decoding. With this aim, we rely on es- tablished message passing frameworks, namely BP, the MF approximation, and EP. For each of the studied problems, a probabilistic model of the sys- tem is developed. Subsequently, the message passing tool (or combination of tools) most suitable to perform variational inference in such model is cho- sen, applied, and the performance of the resulting algorithm is evaluated against previously proposed solutions. In the following, we describe each of the problems that have been studied in the thesis, including a short review of their state-of-art, and specify the contributions made for each of the areas.
1
Turbo Equalization
Iterative equalization and decoding of ISI channels has been thoroughly stud- ied over the last two decades, and many currently well known iterative equal- izer structures have been derived, e.g. decision feedback equalizers [43] and turbo-equalizers [39, 41]. More recently, the popularization of message pass- ing algorithms and factor graphs [17] has inspired the proposal of various message passing solutions. For instance, a BP-based turbo equalizer has been proposed in [44]. BP-based equalizers suffer, however, from a large compu- tational complexity that scales exponentially with the modulation order and the amount of channel taps L.
Other message passing algorithms, alternative to BP, have been proposed to circumvent the aforementioned complexity problem. In [45], the residual interference plus noise component is approximated as a Gaussian variable, similarly to [11], in an approach that can be interpreted as direct approxima-
1. Turbo Equalization
tion of BP messages by Gaussian messages. While this Gaussian approxima-
tion successfully reduces the complexity of the receiver toO(L2)per symbol,
this is achieved at the expense of a significant degradation in bit-error-rate (BER). The trade-off between complexity and BER can be adjusted by resort- ing to algorithms which apply the same Gaussian approximation as [11], but only on a subset of the interfering symbols. This approach, presented in [30], is coined partial Gaussian approximation (PGA). The PGA algorithm for a generic channel matrix was developed in [30]. Another alternative to obtain Gaussian messages and, consequently, low-complexity equalization is to use Gaussian EP, as proposed in [25] and Paper A. As shown in Paper A, ap- plying EP yields a significant improvement in the equalizer’s performance, compared to directly approximating BP messages as Gaussian.
The computational complexity (O(L2) per symbol [11, 24, 39, 41, 45])
of the linear MMSE SISO equalizer in time domain is still too high when the length of channel taps L is long. For instance, in broadband wireless and underwater acoustic communications, L may often be dozens or hun- dreds [46, 47]. This has motivated researchers to pursue alternative equaliza- tion algorithms and, in particular, algorithms that perform in the frequency domain, rather than the time domain. Along these lines, single carrier fre- quency domain equalization (SC-FDE) technique is an attractive technology for wireless communications due to its ability to cope with the temporal dispersion introduced by multipath channels. It preserves the performance, efficiency and low complexity benefits of its OFDM counterpart, while being less sensitive to power amplifier nonlinearities and carrier frequency offsets, in addition to exhibiting a lower peak-to-average transmitted power ratio [46]. For these reasons, SC-FDE has been selected as the access scheme for the up- link of the 3GPP long term evolution (LTE) and LTE advanced standards [48]. Recently, the linear MMSE equalizer has been implemented in the fre- quency domain (with or without the assistance of cyclic prefixing), reducing the equalizer’s complexity to logarithmic level [49–52]. Although its low com- plexity makes it attractive, the frequency domain linear MMSE (FD-LMMSE) equalizer may suffer from significant performance loss when the transmitted signal is severely distorted by the ISI channel. Guo et. al. developed a turbo frequency domain equalization (FDE) algorithm [53] based on GAMP [33], which can achieve significant performance gain with slight complexity in- crease compared to FD-LMMSE equalization. This turbo FDE algorithm can be regarded as the state-of-the-art solution for equalization in frequency do- main.
In this thesis, we apply the message passing frameworks to design re- ceivers performing iterative equalization and decoding in time domain (see
Papers Aand B ) and frequency domain (see Paper C).
Paper A: Iterative Receiver Design for ISI Channels Using Combined Belief-
imate inference method combining BP and EP and apply it to design a turbo equalization and decoding receiver for ISI channels. The proposed receiver, using Gaussian message passing approximated by EP in channel equalization part, avoids the exponential complexity problem of BP-based turbo equaliz- ers. The numerical assessment of our proposed receiver illustrates the advan- tages of applying the combined BP-EP framework over receivers using solely BP and using BP with direct Gaussian approximation of its messages.
Paper B: Turbo-Equalization Using Partial Gaussian Approximation We
further develop the BP-EP receiver in Paper A, by applying the PGA principle proposed in [30] to modify the output messages from equalization. Since PGA allows the receiver to tune the number of symbols that are considered as strong interferers, the proposed receiver enables a flexible performance- complexity tradeoff. The simulation results illustrate the merits of the new turbo equalization receiver compared to the receiver we proposed in Paper A and other benchmarks.
Paper C: Message-Passing Receivers for Single Carrier Systems with Fre-
quency Domain Equalization In this contribution, we design a turbo equal-
ization receiver based on the BP-MF framework for SC-FDE systems. Two re- ceiver algorithms with, respectively, parallel and sequential message passing schedules are proposed in the MF part for channel equalization. Monte Carlo simulations show that our proposed design outperforms a similar structure derived using GAMP, and performs very closely to the matched filter bound.
2
Sparse Channel Estimation
Turbo equalizers in time or frequency domain are designed under the as- sumption that exact channel state information (CSI) is known. However, it is typically unknown and should be estimated in communication receivers. It is well-known that the accuracy of channel estimation is a crucial factor deter- mining the overall performance in wireless communication systems and net- works, in terms of BER and throughput. In addition, as the communication bandwidth increases, multipath propagation channels are usually dominated by a small number of significant paths, resulting in most of the channel co- efficients being either zero or nearly zero, and therefore compressive sensing and sparse signal reconstruction become very powerful tools for the design of channel estimators. Consequently, we also investigate sparse channel es- timation which can provide CSI for detection and equalization in OFDM or SC-FDE receivers.
Various Bayesian and non-Bayesian approaches have been proposed for sparse signal reconstruction in the literature. Greedy constructive algorithms such as orthogonal matching pursuit (OMP) [54] and compressive sampling MP (CoSaMP) [55] share the common characteristic that a proper iteration
2. Sparse Channel Estimation
number or a predetermined sparsity is required to stop the iteration. More- over, these greedy algorithms are non-Bayesian, and are therefore not very suitable to be embedded in receiver structures derived using Bayesian for- malisms. There are many convex optimization based methods, such as the
very popular LASSO regression [56, 57] using l1 norm or Laplace prior, FO-
CUSS (focal undetermined system solver) algorithm [58] using l2 norm or
Gaussian prior and smooth l0 (SL0) algorithms [59, 60] using smoothly ap-
proximate of the l0pseudo norm. These convex optimization based methods
can usually be recast into a Bayesian MAP estimation form, by designing appropriate prior pdfs for the entries of the sparse vector that play the role of the different regularization terms in the convex optimization formulation. Sparse Bayesian learning (SBL) [61–64] has also been proposed for sparse sig- nal reconstruction methods. Instead of working directly with a prior, SBL approaches often model a two-layer (2-L) or three-layer (3-L) hierarchical structure using random hyper-parameters.
Given that direct computation of a Bayesian estimate is often intractable for most choices of a prior pdf, the hierarchical modeling approach allows for the use of iterative estimation approaches instead. For instance, the gener- alized expectation-maximization (EM) algorithms has been a popular choice for this task, along with variational inference approaches. Most recently, SBL has been efficiently implemented using BP [65, 66] and AMP [35, 67]. How- ever, these methods assume that the power of noise is known, which may not be true in many applications.
Using 2-L and 3-L hierarchical prior models, Pedersen et al. proposed a MF-based algorithm [68] to approximate a sparse Bayesian estimate of radio channels in OFDM systems. Since the MF-based iterative algorithm updates the estimates of all channel taps with length L at once –i.e. it jointly updates the full vector of channel taps– its computational complexity is as high as
O(L3) per iteration. Such large complexity stems from the inversion of an
L×L matrix required at each iteration. A low complexity MF-based SBL
algorithm, which decomposes the inversion of said matrix into a set of inver- sions of matrices with smaller dimension, was later developed to reduce the high complexity for OFDM systems [69]. The tradeoff between complexity and performance of channel estimation can be adjusted by selecting the size of such matrix inversions, implying that the reduction of complexity is at the cost of performance loss. Recently, a low complexity MF-based SBL algorithm which completely avoids matrix inversions has been applied to estimate both gains and delays of channel propagation paths [16].
In our work, we have sought to find low-complexity and accurate SBL algorithms that, in addition, include the estimation of the noise variance. Our main contribution is described next.
Paper D: Low Complexity Sparse Bayesian Learning Using Combined BP
ity SBL algorithm is proposed based on the BP-MF framework for in under- determined linear systems. We also simplify the BP messages passed on the densely connected subgraph by approximating some BP messages to further reduce the computational complexity, yielding an approximate BP-MF SBL algorithm. The proposed SBL algorithms present better MSE performance and lower complexity than the original algorithm derived using solely MF.
3
Receivers Performing Iterative Estimation, Equal-
ization and Decoding
Many diverse iterative receiver architectures performing joint estimation (in- cluding channel response, noise power and/or phase noise), equalization and decoding (JEED) for a multiplicity of communication systems have been pro- posed by signal processing researchers over the last 20 years. While many of the early designs [38, 51, 52, 70–73] were based on ad-hoc extensions of the turbo equalization principle to include estimation, which was designed individually as described in Section 2 of this chapter, holistic design method- ologies [12–16, 74–77] have dominated the scene in the recent past. Among the latter, the use of message passing frameworks in which a global objective function is iteratively optimized has been one of the main approaches. We shortly review some of them next.
As we discussed in Chapter 3, BP often presents good performance and therefore has been widely applied for designing receivers iteratively per- forming joint estimation, equalization and decoding in communication sys-
tems [14, 74, 75, 77]. However, such modern systems usually involve a
complicated signal model with a densely connected factor graph in which both discrete and continuous random variables coexist, along with non-linear functions, cumbersome distributions, such as large mixtures of distributions. Those lead to the computations of messages being significantly complex and even intractable when BP is applied directly. Due to this fact, researchers have sought to exploit approximate message techniques, often involving Gaussian approximations, to achieve feasible solutions.
Considering the respective advantages and disadvantages of BP, MF and EP, the combinations of two or more of them can be exploited to design receivers in modern communication systems in complicated scenarios. The BP-MF framework has been applied to JEED receivers for OFDM systems [12, 16, 76] and MIMO-OFDM systems [13]. Paper [15] uses combined BP-EP to design JEED receiver for OFDM systems and also investigates methods to re- duce the computational complexity by approximating some of the messages. In this thesis, we propose designs for JEED receivers in two different com- munication scenarios. These contributions are detailed next.
3. Receivers Performing Iterative Estimation, Equalization and Decoding
MF Message Passing In this contribution, we apply the approximate BP-
MF algorithm proposed in Paper D to carry out channel estimation in time- frequency domain for a JEED OFDM receiver. Our numerical assessment demonstrates the advantages the proposed receiver in terms of BER perfor- mance, complexity and convergence rate.
Paper F: A BP-MF-EP Based Iterative Receiver for Joint Phase Noise Es-
timation, Equalization and Decoding This contribution is the extension
of BP-EP-based turbo equalization proposed in Paper A to the scenario of unknown phase noise. We exploit the BP-MF-EP framework to design an it- erative receiver for joint phase noise estimation, equalization and decoding. In addition, a second-order Taylor expansion is chosen to approximate some MF messages, so that it provides a Gaussian prior for the phase noise estima- tion subgraph. Simulation results confirm that remarkable performance gain can be achieved with the proposed approach.
Chapter 5
Conclusions and Outlook
In this thesis, we exploit the formulation of signal processing problems in digital communications as approximate inference problems in probabilistic models. This, in turn, allows for the design of iterative signal processing so- lutions via message passing algorithms in (probabilistic) factor graphs. This general methodology has the advantage of providing large flexibility in the design of the signal processing algorithms. This flexibility is due to two main reasons:
• On the one hand, there is flexibility in the modeling of each particular problem, i.e. the definition of the probabilistic system that represents the problem. In turn, this results in flexibility regarding the definition of the factor graph representation of the system.
• On the other hand, there is as well flexibility with respect to the approx- imate inference methods, i.e. the particular message passing technique used to perform estimation in the probabilistic system and associated factor graph.
By finding appropriate combinations of the two aspects mentioned above, receiver algorithms that attain remarkable performance can be derived.
While the advantages of using message passing techniques to design dig- ital receivers are nowadays well established in the signal processing com- munity, most instances of receivers are derived using only a single message passing technique. For some specific systems, such designs may yield solu- tions with intractably high computational complexity or unsatisfactory per-