We conclude this section by making a number of important observations. To theoreti- cally explain and also understand the near-ML decoding performance achieved by turbo decoding, the main task of this chapter is to link iterative turbo decoding to the ML de- coding problem. For accomplishing this task, we have followed the approach introduced in [93,94]. Namely, we have constructed a continuous optimization problem, whose global optimal solution has a deterministic relation with the ML solution. Moreover, we have demonstrated that iterative turbo decoding as an approximative iterative solution to that continuous optimization problem. However, during the derivations, we have observed two important issues: 1) the global optimal solution of the continuous optimization problem is always a boundary point and 2) iterative turbo decoding can only converge to an interior local extremum. These two observations indicate that turbo decoding can never find the global optimal solution that is proved to reflect the ML solution. As such, the connection between the fixed-point of turbo decoding and the ML solution remains unclear. Further investigations on the connection will be done in the next chapter.
Approximate Algorithms for
ML Decoding
We continue the work of linking turbo decoding to the ML decoding problem in this chap- ter. Particularly, this chapter contributes to the derivation of sufficient conditions on the existence of a fixed-point of turbo decoding that reflects the ML solution. Different to the approach in [93,94], we try to accomplish this task by connecting the ML decoding problem to a constrained Bethe free energy minimization problem, which is commonly studied by physicists. The motivation of doing so relies on the following two results in the literature:
• Turbo decoding has been identified as an instance of the BP algorithm in [66]. • Consider a system with a factorizable Boltzmann distribution (cf. Section 2.3). At
unit temperature, fixed-points of BP operating on the factor graph of the Boltzmann distribution correspond to stationary points of the constrained Bethe free energy of the underlying system [126].
Using a constrained Bethe free energy minimization problem as the relay (see Fig. 4.1), turbo decoding can be linked to the ML decoding problem by illustrating the following two connections:
1. The connection between the ML solution and the global minimizer of the constrained Bethe free energy.
2. The connection between turbo decoding and the constrained Bethe free energy min- imization problem.
Note that the above-mentioned two results in [66, 126] are inadequate to demonstrate the second connection. The reason is simple. In general, the global optimal solution of an optimization problem can be a boundary point rather than a stationary point. Attempting to establish the two connections with mathematical proofs and theoretical derivations, we take the following procedure in this chapter:
• For the first connection, we start from formulating a constrained Bethe free energy based on the ML decoding criterion and then derive the relation between the ML
Figure 4.1: The constrained Bethe free energy minimization problem is used as a relay to connect the turbo decoding algorithm and the ML decoding problem.
solution and the global minimizer of the constrained Bethe free energy at zero tem- perature. By proving the continuity property of the global minimizer over the tem- perature, we eventually establish the connection between the ML decoding prob- lem and the constrained Bethe free energy minimization problem for general non- negative temperatures. Regalia and Walsh tried to establish such a connection at a specific temperature, i.e., unit temperature. However, their results obtained in [84] have relied on several mathematical unorthodox arguments1. Our approach is totally
different to theirs and the results are valid for general temperatures.
• In the previous chapter, we have interpreted turbo decoding as the block Gauss- Seidel-type fixed-point iteration to solve a set of stationary point equations. For the second connection in Fig. 4.1, we prove that one solution to these stationary point equations can yield the global minimizer of the constrained Bethe free energy at unit temperature.
After establishing the two connections in Fig. 4.1, we proceed to investigate the follow- ing two aspects related to turbo decoding:
1. Evidently, turbo decoding is not the only way to solve the constrained Bethe free energy minimization problem at unit temperature. In the field of statistical physics, the so-called double-loop iterative algorithm [42] is a popular algorithm, as it can successively reduce a constrained Bethe free energy and is guaranteed to converge. However, the application of the double-loop iterative algorithm for approximate ML decoding requires high complexity and latency, making it unpractical for real-time implementation. Therefore, we propose a low complexity approximation to it. By comparing turbo decoding with the low complexity alternative, the discovered con- nection helps us gain an enhanced understanding on the convergence behavior of turbo decoding.
2. The connection between the ML decoding problem and the constrained Bethe free energy minimization problem exists not only for temperature one, but also for other 1On the one hand, they constructed the so called pseudo-dual problem from the primal constrained Bethe
free energy minimization problem, but without demonstrating their relation. On the other hand, they con- structed a constrained maximization problem equivalent to the ML decoding problem and also formalized the Lagrange function associated to that constrained maximization problem. With the argument that solving the pseudo-dual problem is equivalent to maximizing the Lagrange function after fixing the values of Lagrange multipliers at−1, they considered that the ML decoding problem is connected to the constrained Bethe free energy minimization problem.
positive temperatures. Therefore, we include the temperature parameter into the conventional turbo decoding algorithm, and then study the temperature effect in its decoding performance. By means of simulations, we notice the optimal temperature for turbo decoding to achieve the minimal FER is case-specific.
4.1
ML Decoding Problem
Let us recall the PCCC- and SCCC-coded system depicted in Fig. 3.1. In terms of the in- formation bit sequence m, the criterion for ML decoding of the PCCC in Fig. 3.1(a) can be written as ˆ m= arg max m∈{0,1}Nmln p (y|h, m) = arg min m∈{0,1}Nm[− ln p (y1|h1, m)− ln p (y2|h2, m)] (4.1)
where the last equality holds because the two observation vectors, i.e., y1 and y2, are
mutually independent after conditioning on the channel coefficients and the information bit sequence. For ML decoding of the SCCC shown in Fig. 3.1(b), we express the criterion in terms of cc1, which is the codeword generated by the outer CC and further encoded by
the inner CC, see Fig. 3.1(b). Formally, we have ˆcc1 = arg max
cc1∈G[1]ln p (y|h, cc1)
= arg min
cc1∈{0,1}Nm/rc1
[− ln IG[1](cc1)− ln p (y|h, cc1)] (4.2)
where the last equality holds because the logarithm of the indicator function IG[1](cc1)
equals zero if and only if cc1 ∈ G
[1]; otherwise, it equals−∞. Since the encoding process
of the outer CC is a bijective function, we can uniquely determine ˆmbased on ˆcc1.
The common feature of (4.1) and (4.2) is that their objective functions consist of two component functions. To obtain a unified formulation for (4.1) and (4.2), we introduce a bit sequence e∈ {0, 1}Ne with lengthN
eand two factor functionsΛα(e) and Λβ(e). They
are configured differently in the PCCC- and SCCC-coded system:
PCCC The bit sequence e represents the information bit sequence m. The factor func- tionΛα(e) and Λβ(e) represent the likelihood function p (y1|h1, m) and p (y2|h2, m), re-
spectively, i.e., e∈ {0, 1}Ne∆=Nm Λα(e) ∆ = p (y1|h1, m = e) ∀e ∈ {0, 1}Ne Λβ(e) ∆ = p (y2|h2, m = e) ∀e ∈ {0, 1}Ne . (4.3)
SCCC The bit sequence e represents the codeword cc1attained at the output of the outer
CC. The factor functionΛα(e) and Λβ(e) represent the indicator function IG[1](cc1) and the
likelihood functionp (y|h, cc1), respectively, i.e.,
e∈ {0, 1}Ne∆=Nm/rc1 Λα(e) ∆ = IG[1](cc1 = e) ∀e ∈ {0, 1}Ne Λβ(e) ∆ = p (y|h, cc1 = e) ∀e ∈ {0, 1} Ne . (4.4)
In the above, we do not make the dependence of the function Λα(e) and Λβ(e) on the
observations {yk}, channel coefficients {hk} and the knowledge of coding scheme, e.g.,
G[1] explicit for simplifying the notation. Furthermore, as we consider a noisy Gaussian
channel, the functionΛβ(e) is strictly positive in both cases. On the contrary, the function
Λα(e) is positive in the PCCC-coded system and becomes non-negative in the SCCC-coded
system. Throughout this chapter, we generally takeΛα(e) as a non-negative function and
takeΛβ(e) as a positive function. In particular, the non-negative function Λα(e) has the
following property
∀i ∈ {1, 2, . . . , Ne} ∀e ∈ {0, 1} {e′|e′ ∈ {0, 1}Ne, Λα(e′) > 0, e′i = e} 6= ∅ (4.5)
based on the explanation for (3.72) given at the end of Section 3.3.3. Using e, Ne, Λα(e)
andΛβ(e), the unified formalism of the ML decoding problem in both types of turbo-coded
systems is given as
ˆ
e= arg min
e∈{0,1}Ne[− ln Λα(e)− ln Λβ(e)] (4.6)
where ˆeis equivalent to ˆmin the PCCC-coded system and becomes identical to ˆcc1 in the SCCC-coded system. Throughout this chapter, we refer to the optimization problem in (4.6) as the target ML decoding problem.
In the previous chapter, we have introduced an alternative representation of the en- coding process respectively associated to PCCCs and SCCCs, see Fig. 3.4 and Fig. 3.6. In accordance with them, the ML decoding problem in (4.6) can be alternatively expressed as
{ ˆα, ˆβ} = arg min
α,β∈{0,1}Ne[− ln Λα(α)− ln Λβ(β)] (4.7)
subject to α= β. (4.8)
In words,{ ˆα, ˆβ} are chosen to minimize − ln Λα(α)− ln Λβ(β) under the equality con-
straint α= β.