Scalar Quantization

The scalar quantizer (as any quantizer) can be explicitly separated into two parts, an encoder and a decoder. The encoder maps E : R → I, where R are the real numbers and I is the index set for partition cells. The decoder is the mapping D : I → R, so the quantizer can be written asQ(x) = D(E(x)). Note that we restrict ourselves to real scalar quantizers, since the scalar quantization of a complex number is in fact the vector quantization of two real–valued quantities.

The output set or codebook C = {y1, y₂, y₃, . . . , y_M} with yi ∈ R for the decoder process satisfies thaty₁ ≤ y2 ≤ . . . yM, with the codebook size |C| = M. Therefore,

4.3 Quantization Error 87

.. . ..

. y

S1

S2

SM

y1

y2

yM

Q(y)

Figure 4.5: Structure of a Scalar Quantizer.

the number of bits needed to specify the quantized value is given by log₂M , the so–

called resolution. The resolution indicates the accuracy with which the original analog amplitude is described.

The encoder operation can be modeled by means of a selector function S_i(y). The selector functionS_i(y) is 1 if y ∈ Ri, i.e. thei–th partition region, and 0 elsewhere, and the quantization task can be expressed as

Q(y) = XM

i=1

yiSi(y). (4.17)

Note that for any given input valuex, only one term of the sum is nonzero. In the sequel, we will use this notation for scalar quantization. Fig. 4.5 depicts the structure of a scalar quantizer according to this notation.

The main goal of the quantizer design is to select the representants and the partition regions or cells to provide the minimum possible average distortionE[d(•, •)] for a fixed number of levels M . In general, this problem has no explicit, closed-form solution but some efficient algorithms can be used.

By assuming that one part (the encoder or the decoder) is fixed, it becomes easy to specify a condition for optimality of the other part. Specifically, the encoder part of an optimal quantizer must be optimal for the given decoder while the decoder must be optimal for the given encoder [108]. The two conditions are necessary for a quantizer to be optimal.

The best encoder for a given codebook satisfies the nearest neighbor condition. This

requires that thei–th region of the partition consists of all input values closer to y_ithan to any other output level, i.e. the partition cells satisfy

Ri ⊆ {x ∈ R : d(x, yi)≤ d(x, yj) ∀j}

that is,

Q(x) = yi only if d(x, yi)≤ d(x, yj) ∀j.

Thus, given the decoder, the encoder minimizes the distortion:

d(x, Q(x)) = min

yi∈Cd(x, yi). (4.18) This result holds in general if the goal of the quantizer is the minimization of the average distortion. The most convenient and widely used measure of distortion between an inputx and its quantized valueQ(x) is the squared error or squared Euclidean distance between two scalar values, defined as

d(x, Q(x)) = |x − Q(x)|². (4.19) We now examine the second necessary condition for optimality which is obtained by fixing the encoder (partition) and optimizing the decoder (codebook). The centroid condition is found based on this optimization provided that the squared error distortion measure is used. The centroid condition is simply the condition that the optimal output level,yi, for thei–th cell of the partition is the centroid, or center of mass, of that part of the input PDF that lies in the regionRi, i.e.

yi = E[x|x ∈ Rⁱ] = Z

R

xf_x,x∈Ri(x, x∈ Ri)

Pr[x∈ Rⁱ] dx = E [xSi(x)]

E [Si(x)] . (4.20) When the quantizer satisfies the centroid condition, the following properties are fulfilled

E [Q (x)] = E [x]

E [x Q (x)] = E

Q (x)² E

Q (x)²

= E x²

− E

(x− Q (x))²

. (4.21)

Uniform Quantizer

The most common of all scalar quantizers is the uniform quantizer whose principle is rather simple (see [108]). A uniform quantizer is a quantizer where the boundary points x1, . . . , x_{M −1} defining the partition cells Ri are equally spaced and the representants are the midpoints of the quantization interval. The first condition implies that with step size ∆, x_i − xi−1 = ∆, for i = 2, 3, . . . , M − 1, whilst the second implies that

4.3 Quantization Error 89

C ^m+1

Centroid

C ^m

Computation Partitioning

Neighbor

Figure 4.6: Lloyd Iteration for Codebook Improvement.

yi = (x_i−1+ xi)/2, for i = 2, 3, . . . , M − 1. Consider the case of a uniform quantizer where the input is bounded with values lying in the range(a, b). When the input PDF is uniformly distributed over the region, the quantizer error ǫ = Q(x)− x has a uniform PDF on[−∆/2, ∆/2], where ∆ = (b − a)/M is the step size. Given the cell, the average distortion is simply the variance of a random variable that is uniformly distributed on an interval of width ∆, that is ^∆₁₂². Obviously, the mean of the quantizer error is zero.

Although the input distribution might not be uniform, the uniform quantizer gives a reasonably good performance for a wide variety of input signals. In fact, due to this reason and also for simplicity, the uniform quantizer is widely used in A/D conversion.

In the context of this work we have the coefficients of the rank-reduced representation h˜T,k[q] in Eq. (4.13). As we will see, although the input PDF is not uniform but Gaussian, we make the simplifying assumption that the input is bounded, i.e. we assume that the real and imaginary parts of every entry of ˜hT,k[q] lie in some interval (a, b) (see Chapter 6).

The overload region has a very low probability (less than 5 %) of containing an input sample. Thus, we choose representants betweena and b to construct an initial codebook that is stored at both transmitter and receiver. After transmission, every receiver performs a search to find for each component of the coefficients (real and imaginary part) the element in the corresponding codebook that is closest. Then, the codebook index is fed back to the transmitter. Finally, the transmitter simply looks into its codebook and reconstructs the estimated channel from the selected codeword [103]. This estimate will be used to obtain the precoder filters.

Non-uniform Quantizer based on Gaussian Inputs and Lloyd Algorithm

When considering non-uniform quantizers, we have to select the quantizer step sizes to fit the input distribution (Gaussian in our case) employing non–uniformly spaced levels. We employ the Lloyd algorithm based not on a training sequence, but rather on the known exact input distribution. Note that it is crucial to avoid a new training sequence from the point of view of efficiency and performance. Given a codebookCm, the Lloyd algorithm

Initial Codebook

Test Distortion

Compute Cm

Stop Cm+1 Lloyd iteration

Figure 4.7: Lloyd Algorithm for Quantizer Design.

finds the optimal partition by means of using the nearest neighbor condition. The partition cells are thus defined as

Ri ⊂ {x ∈ R : d(x, yi)≤ d(x, yj) ∀j 6= i}.

Basically, the procedure is as follows. We begin with an initial codebook C1(m = 1), for example, the uniform codebook. Then, in the next step, given the codebookC^m, we perform the Lloyd algorithm to generate the improved codebook C^m+1. After that, we compute the average distortion for C^m+1. If it is less than a fixed threshold, we stop.

Otherwise, we set m + 1 → m and repeat the previous step. In this way, we get a locally optimal codebook for the Gaussian input with the minimum average distortion (see Figs. 4.6 and 4.7). ²

This quantization scheme is modified when considering the final approach shown in

2The nearest neighbor condition and the centroid condition are only necessary but not sufficient for global optimality. Therefore, only local optimality is ensured by the Lloyd algorithm.

4.3 Quantization Error 91

Chapter 7, where the codebook entries are not the channel coefficients but the employed precoders. Therefore, the index fed back by each user corresponds to a set of precoders and the intersection of the sets gives us the best precoder (see Chapter 7).