Bounds on Distortion Bit-Cost Function for First Order Σ∆
Analog-to-Digital Converter with Input Noise
Hossein Kakavand and Abbas El Gamal
Department of Electrical Engineering Stanford University
Stanford, CA, 94305, USA
E-mail: [email protected]; [email protected]
Abstract
We establish upper and lower bounds on the distortion bit-cost function for incremental first-order Σ∆ ADC with constant input signal U that is uniformly distributed on (0, 1) and additive white Gaussian (AWG) input noise. It is shown, under mild conditions, that the mean squared-error distortion D scales with the number of index bits m roughly as 1/m. This is in sharp contrast to the 1/m3
scaling for the case of an ideal Σ∆, i.e., when there is no input noise, with optimal decoding. The upper bound is proved assuming the averaging decoder is used, while the lower bound is proved using the Cramer-Rao bound. We also show that the minimum distortion bit-cost function (over all encoder-decoder pairs) is lower bounded by (c/m) (1 − O (1/m)) for some constant c. Thus the first order Σ∆ with averaging decoder performs close to optimally in the presence of input noise.
1
Introduction
Σ∆ architectures are among the most popular analog-to-digital converters (ADCs) due to their large dynamic range and resilience to circuit component noise and device mismatches [1]. In this paper we seek to quantify this resilience for non-idealities that can be modeled as additive input noise.
To begin the discussion, we view an ADC as a quantization system consisting of an encoder e that maps an analog signal U into an index sequence Ym ∈ {0, 1}m, m = 1, 2, . . ., and a decoder d that
maps Yminto an estimate of the input signal ˆU (Ym) (see Figure 1). We assume throughout that U is
uniformly distributed over (0, 1).
Encoder Decoder
Figure 1: Encoder decoder pair.
Different criteria have been used to measure the performance of an ADC as a function of the number of index bits m, that is, bit-cost. For example in [2], the worst case error distortion criterion is assumed. In this paper we assume the expected squared error distortion
D = EdU, ˆU(Ym)= EU
− ˆU (Ym)2.
A distortion bit-cost pair (D, m) is said to be achievable if there exists a decoder d, such that for bit-cost m
EU− ˆU(Ym)2
≤ D.
The distortion bit-cost function D(e, m) for an encoder e is the infimum of distortions D such that (D, m) is achievable, i.e., D(e, m) = inf d D : EU − ˆU (Ym)2≤ D .
The optimal decoder for a given encoder is a decoder that achieves D(e, m) for all m≥ 1. The optimal decoder and D(e, m) are known for some ADC encoders, such as flash, incremental first and second order
Σ∆ encoders [3] and Beta expansion [4]. These results, however, assume ADCs with ideal components. In practice the encoder part of the ADC is built using mostly analog components, which are intrinsically imperfect suffering from device mismatches, nonlinearities as well as thermal and 1/f noise [5].
Finding the distortion bit-cost function D(e, m) for an ADC with imperfect components is in general quite difficult because (i) modeling the imperfections in a tractable manner is not always possible, and (ii) finding the optimal decoder in the presence of imperfections in the encoder is computationally intractable. Recently some progress has been made on this problem. In [2], Daubechies et al. studied the effect of an imperfect comparator on the worst case error distortion for the first order Σ∆ architecture as a function of bit cost. In [6], we studied the same problem assuming the expected squared error distortion criterion.
In this paper we investigate the distortion bit-cost function for the first order Σ∆ encoder with additive white Gaussian (AWG) noise at the input. The input noise represents the “input-referred” circuit noise, for example, due to the integrator. Since computing the optimal decoder in this case is quite difficult, we establish upper and lower bounds on D(Σ∆, m). In Section 2, we introduce the model of the first order Σ∆ encoder with input noise. In Sections 3 and 4, we show that1D(Σ∆, m) =
cσ2/m + o
P(1/m) for some 1 ≤ c ≤ 2. The upper bound is established using the averaging decoder
while the lower bound is established using the Cramer-Rao lower bound. Thus in the presence of input noise, the simple averaging encoder is close to optimal. This is in sharp contrast to the ideal encoder case, where D(Σ∆, m) scales as 1/m3 and is achieved by a highly nonlinear decoder.
So, how good is the performance of Σ∆ in the presence of input noise. To answer this question, we compare our lower bound on D(Σ∆, m) to the minimum distortion bit-cost function D(m) defined as the infimum of D(e, m) over the set of all encoders. First, it is trivial to show that with ideal components, D(m) = 2−2m/12 and is achieved by the binary expansion encoder (i.e., successive approximation
encoder) and dyadic decoder. This, however, gives a very poor lower bound on D(Σ∆, m) in the presence of input noise. As finding D(m) in the presence of input noise appears to be quite difficult, in Section 5, we derive the lower bound2D(m)≥ σ2/m (1 − O (1/m)). Thus quite surprisingly, the first-order Σ∆
encoder with averaging decoder performs close to optimally in the presence of input noise.
2
First Order
Σ∆
Consider the discrete-time 1-bit first-order Σ∆ encoder with additive white Gaussian (AWGN) input noise depicted in Figure 2. We assume that the encoder is reset before conversion commences, which is commonly referred to as incremental Σ∆ [3]. The discrete-time dynamics of this encoder are described by
U0= 0,
Yi= Q(Ui), i = 1,· · · , m,
Ui+1= Ui− Yi+ U + Zi, i = 0,· · · , m − 1.
where{Zi} is WGN with power σ2, i.e., Zi∼ N (0, σ2) for all i, and Q(.) is a 1-bit quantizer, that is,
Q(z) = (
0, z < 1, 1, z≥ 1. Note that the threshold is set to 1 because the input U ∈ (0, 1).
In a practical implementation, the input to the Σ∆ encoder is continuously integrated, sampled, compared to the threshold and then fed back to the input. Assuming that the additive input noise Z(t) is a WGN process, then Zi = R
i+1
i Z(t)dt for i = 1, 2, . . .. Clearly the Zis are i.i.d. Gaussian.
Considering the continuous-time dynamics of the encoder, observe that between each two samples that generate consecutive ones, say j and j + k, the Ui values are samples of a Brownian motion process with
initial value (Uj− 1) and positive drift U, thus
U (t) = (Uj− 1) + Ut + σB(t), for t ∈ (j, j + k],
1
We use the following notation throughout. For a sequence of random variables Wm: (i) Wm = oP(1) means that
Wm→ 0 in probability, i.e., ∀ > 0, limm→∞P(|Wm| > ) = 0. (ii) Wm= OP(1) means that ∀ > 0, ∃M < ∞ such that
P(|Wm| > M ) < . 2
Figure 2: First-order Σ∆ encoder with input noise.
where B(t) is the standard Brownian motion process. Based on this observation and employing the equivalent rampanalogy introduced in [3], the continuous-time dynamics of the first order Σ∆ encoder can be alternatively described by
Y0= 0,
˜
U (t) = U t + σB(t),
Yi+1= Yi+ Q( ˜U(i + 1)− Yi), i = 0,· · · , m. (2.1)
In this representation, the ˜Ui values are samples of the ˜U (t) process where instead of subtracting a
one from the ˜Ui values the encoder increases the threshold by one (see Figure 3). This alternative
representation simplifies the derivation of the upper and lower bounds on D(Σ∆, m). To make the derivations of the bounds tractable we assume that σ satisfies the relation
σ √
2πexp −1/2σ
2
≤ δ. (2.2)
As explained in [7], this assumption guarantees that the expected total absolute difference between the Uisequence with and without the assumption that{|Ui+1− Ui| ≤ 2, ∀i ≥ 0} is less than δ. We assume
that σ≤ 0.1, which results in δ = 10−23.
Figure 3: Sample path of the alternative (equivalent ramp) description of the continuous-time first order Σ∆ encoder dynamics. The solid dots represent ˜Ui values.
3
Upper Bound
To establish an upper bound on D(Σ∆, m), we assume the averaging decoder ˆ U = 1 m m X i=1 Yi ! .
Theorem 1. The distortion bit-cost functionD(Σ∆, m) for the first order Σ∆ encoder with AWG input noise and averaging decoder, whereσ satisfies Equation 2.2, is upper bounded by
D(Σ∆, m)≤ 2σ
2
m + O
m−3/2.
Proof. Since the input U ∈ (0, 1), we assume that U ∈ [σ, 1), for some arbitrarily small > 0. Further, we divide the [σ, 1) interval into two disjoint intervals [σ, τ σ) and [τ σ, 1), where τ 1 is an arbitrary constant such that τ σ < 1. This is possible since σ satisfies Equation 2.2. We provide upper bounds for each interval separately. The idea is that we can treat the case where U σ and the case where U and σ are comparable, separately.
First, it is easy to show that
D = EU− ˆU2= 1 1− σ Z τ σ σ Eu− ˆU2du + Z 1 τ σ Eu− ˆU2du . (3.1)
Referring to Fig. 3, it follows that
m X i=1 Yi (a) = max k∈{1,··· ,m} ku + k X i=1 Zi = mu− (m − r)u + r X i=1 Zi− δr,
where (a) follows, sincePm
i=1Yi is the number of ones in the Yi sequence, which is equal to the number
of thresholds the Ui sequence exceeds, r = argkmax
ku +Pk
i=1Zi
and δr =ru + Pri=1Zi , where
· denotes the fractional part. Thus Eu− ˆU2= E u−m1 " mu− (m − r)u + r X i=1 Zi− δr #!2 = m X l=1 P (r = l) E 1 m2 " (m− l) u − l X i=1 Zi+ δl #2 r = l (a) ≤ m12 m X l=1 P (r = l) " (m− l)2u2+ lσ2+ 1 + 2 (m− l) u + 2 r 2lσ2 π # , (3.2)
where (a) follows from upper bounding each term. Next, we upper bound P (r = l) for the two intervals. For u∈ [τσ, 1) we upper bound P (r = l) for l = 1, 2, · · · , m − 1 and for l = m separately (since we expect the maximum to occur at r = m).
Let Vj = ju +Pji=1Zi for j = 1,· · · , m, then
P (r = m) = P (Vm≥ Vm−1, Vm≥ Vm−2,· · · , Vm≥ V1) ≤ P (Vm≥ Vm−1) = P mu + m X i=1 Zi≥ (m − 1) u + m−1 X i=1 Zi ! = P (Zm≥ −u) (a) ≤ 1 − √1 2π u/σ 1 + u2/σ2exp −u 2/2σ2 ,
where (a) follows from Z ∞
s
exp −x2/2 dx ≥ 1 + ss 2exp −s 2/2
for all s > 0. Now consider the case of j = m− 1,
P (r = m− 1) = P (Vm−1> Vm, Vm−1≥ Vm−2,· · · , Vm−1≥ V1)
= P (Zm<−u) (a)
≤ σu√1
2πexp −u
2/2σ2 ,
where (a) follows from Z ∞ s exp −x2/2 dx ≤ 1 sexp −s 2/2 for all s > 0. Similarly, for j = m− 2, P (r = m− 2) = P (Vm−2> Vm, Vm−2≥ Vm−1,· · · , Vm−2≥ V1) ≤ P (Vm−2> Vm) = P (Zm+ Zm−1<−2u) ≤√σ 2u 1 √ 2πexp −2u 2/2σ2 .
The remaining bounds can be proved similarly. Hence, P (r = m)≤ 1 − √1 2π u/σ 1 + u2/σ2exp −u 2/2σ2 , (3.3) P (r = m− l) ≤ σ u√2πlexp −lu 2/2σ2 , l = 1, · · · , m − 1. (3.4)
Combining Equations 3.3 and 3.4 with Equation 3.2 results in Eu− ˆU2≤ 1 m2 " mσ2+ 1 + 2 r 2mσ2 π # 1−√1 2π u/σ 1 + u2/σ2exp −u 2/2σ2 + 1 m2 m−1 X l=1 " (m− l)2u2+ lσ2+ 1 + 2 (m− l) u + 2 r 2lσ2 π # σ u√2πlexp −lu 2/2σ2 . Carrying out the summations results in
Eu− ˆU 2 ≤ σ 2 m " 1−√1 2π σ 1 + σ2exp −1/2σ 2 + 1 τ√2π exp −τ2 1− exp (−τ2/2) # + O(m−3/2). Further, it can be shown that for u∈ [τσ, 1),
Eu− ˆU2≤ 2σ
2
m + O(m
−3/2). (3.5)
We now consider the case when u∈ [σ, στ). Note that the bounds given in Equations 3.4 are rather loose and so we bound the corresponding probabilities using the Chernoff bound to obtain
P (r = m)≤ 1 − √1 2π u/σ 1 + u2/σ2exp −u 2/2σ2 , (3.6) P (r = m− l) ≤ exp −lu2/2σ2 , l = 1, · · · , m − 1. (3.7)
Combining Equations 3.6 and 3.7 with Equation 3.2 results in Eu− ˆU2≤m12 " mσ2+ 1 + 2 r 2mσ2 π # 1−√1 2π u/σ 1 + u2/σ2exp −u 2/2σ2 + 1 m2 m−1 X l=1 " (m− l)2u2+ lσ2+ 1 + 2 (m− l) u + 2 r 2lσ2 π # exp −lu2/2σ2 .
Carrying out the summations results in Eu− ˆU2≤ σ 2 m " 1−√1 2π 1 + 2exp − 2/2 + exp −2/2 1− exp (−2/2) # + O(m−3/2). It can further be shown that
Eu− ˆU2≤ 2σ
2
m + O(m
−3/2). (3.8)
4
Lower Bound
In this section we derive a lower bound on D(Σ∆, m). This is achieved by devising, for each u∈ (0, 1), a sequence of i.i.d. random variables from which we can deterministically find the{Yi}mi=1sequence. We
then use the Cramer-Rao bound [8] to provide a lower bound on the variance of any estimator of u given that sequence. Clearly this bound is also a lower bound on the variance of any estimator for u given {Yi}mi=1. Based on this observation, we obtain the following lower bound.
Theorem 2. The distortion bit-cost functionD(Σ∆, m) for the first order Σ∆ encoder with AWG input noise is lower bounded by
D(Σ∆, m)≥σ 2 m " 1− OP log log m m 1/2!# .
Proof. Given U = u, let
Xi= inf{s ≥ 0 : su + σB(s) = i}, i = 1, 2, · · · ,
where B(s) is the standard Brownian motion process and X0 = 0. Define the sequence T0 = 0, Ti =
Xi− Xi−1, i = 1, 2, . . .. Given the {Ti}ni=1 sequence we can reconstruct the corresponding {Yi}mi=1
sequence simply by setting Yi = 1 if i∈ {dPkj=1Tje, k = 1, · · · , n} and Yi= 0, otherwise, as depicted
in Fig. 4. Hence, an achievable lower bound on the minimum variance of any estimator of u based on the{Ti}ni=1 sequence is a lower bound on an achievable lower bound on the minimum variance of any
estimator of u based on the{Yi}mi=1 sequence.
Figure 4: The Xi and Ti sequences corresponding to a sample path of ˜U (t), where n = 4 and m = 14.
The solid dots indicate the ˜U (i) which result in ones in the corresponding Yisequence.
The{Ti}ni=1 is a sequence of i.i.d., inverse Gaussian distributed [9] random variables, i.e.,
fT1(t) = 1 √ 2πσ2 1 t3/2exp −(1− ut) 2 2σ2t
Note that by making the change of variables λ = 1/σ2 and µ = 1/u, this distribution can be written as
a canonical member of the exponential family of distributions [8] fT1(t) = r λ 2πexp λ µ exp −2µλt2 1 t3/2exp −2tλ ,
where σ is known, µ is unknown, and t is the observation. Since Eµ(t) = µ, the Fisher information
contained in the observation t about the parameter µ is given by If(µ; t) =
1 var(t) =
λ µ3,
which can be written as If 1 u; t = u 3 σ2. (4.1)
For a differentiable function of u, such as 1/u, we have the following relation between the Fisher infor-mation contained in the observation t about the parameters u and 1/u
If(u; t) = If 1 u; t d (1/u) du 2 . (4.2)
Combining Equations 4.1 and 4.2, we have
If(u; t) =
1
uσ2. (4.3)
It follows from the Cramer-Rao bound and Equation 4.3 that, given the observation t, the variance of any estimator δ(t) of the parameter u is lower bounded by
var(δ(t))≥ 1 If(u; t)
= uσ2. (4.4)
Since the{Ti}ni=1 is an i.i.d. sequence, it follows that
If(u;{ti}ni=1) = nIf(u; t1) = n
uσ2,
and thus given the observation (t1,· · · , tn), the variance of any estimator δ(t1,· · · , tn) of the parameter
u is lower bounded by var(δ(t1,· · · , tn))≥ 1 If(u;{ti}ni=1) =uσ 2 n . (4.5)
From Lemma 2 in the Appendix, the number of thresholds, n, the process defined by Equations 2.1 “hits” before m is related to m by
u n = 1 m " 1− OP log log m m 1/2!# . (4.6)
Combining Equations 4.5 and 4.6, we obtain the bound var(δ(t1,· · · , tn))≥ uσ2 n = σ2 m " 1− OP log log m m 1/2!# , (4.7)
which is also a lower bound on the variance of any estimator of the parameter u given{Yi}mi=1. This
completes the proof.
Remark: Note that since fT1(t) belongs to the exponential family, the variance uσ
2in 4.4 is achievable,
i.e., there exists an estimator based on t with variance uσ2(see [10] for details). Since
{Ti}ni=1is i.i.d., its
distribution is also a member of the exponential family, and thus the lower bound 4.7 is again achievable. This suggests that our bound is reasonably good.
5
Lower Bound on D
(m)
To evaluate the performance of the first order Σ∆ encoder in the presence of input noise, define the minimum distortion bit-cost function D(m) as the infimum of D(e, m) over the set of all encoders, i.e.,
D(m) = inf e infd D : EU− ˆU (Ym)2≤ D .
The following provides a lower bound on D(m) in the presence of AWG input noise.
Lemma 1. The minimum distortion bit-cost function with AWG input noise is lower bounded as D(m)≥ σ 2 m 1− O 1 m .
Proof. Assume that the sequence {U + Zi}mi=1 is provided to the decoder. Clearly, the minimum
mean-squared error (MSE) of estimating U based on this sequence is a lower bound on the minimum MSE based on any binary sequence generated using this sequence. Using standard estimation techniques, the minimum MSE of any estimator ˆU of U is given by σ2/ 12σ2+ m. This serves as a lower bound on
the distortion bit-cost function for any encoder-decoder pair. Thus D(m)≥ σ 2 m 1− O 1m .
Hence, the Σ∆ encoder has a distortion bit-cost function that is close to the optimal distortion bit-cost function.
6
Conclusion
The paper established upper and lower bounds on the distortion bit-cost function for incremental first-order Σ∆ ADC with constant input signal U in the presence of input additive white Gaussian noise. It was shown that the mean squared-error distortion D scales roughly as 1/m with the bit-cost m. This is in sharp contrast to the 1/m3scaling for the case of an ideal Σ∆, i.e., with no input noise and with
optimal decoding. We also showed that the minimum distortion bit-cost function, over all encoder-decoder pairs is lower bounded by O(1/m). Thus the first order Σ∆ with averaging encoder-decoder performs close to optimally in the presence of input noise.
Future directions include characterizing the distortion bit-cost function in the presence of input noise for other ADC architectures, such as higher order Σ∆ and Beta expansion. Also, studying the distortion bit-cost function for non-constant inputs is of great importance, since the results of such a study would enhance our understanding of the theoretical aspects of analog-to-digital conversion and signal representation in general, as well as guide the design and performance analysis of ADCs.
7
Appendix
Lemma 2. For a givenu, the relation between the number of bits generated by the Σ∆ encoder, m, and the number of thresholds n that the process defined by Equations 2.1 exceeds before m is given by
u n = 1 m " 1− OP log log m m 1/2!# . (7.1)
Proof. Refereing to Fig. 4, it follows that
n =bmu + σB(m)c,
which results in mu + σB(m)− 1 ≤ n < mu + σB(m). We can write Equation 4.5 as uσ2 n > uσ2 mu + σB(m) = σ 2/m 1 + σB(m)/mu =σ 2 m " 1−σB(m)mu + σB(m) mu 2 +· · · # , which results in uσ2 n (a) ≥ σ 2 m 1−σB(m)mu (1− oP(1)) (b) = σ 2 m " 1− OP log log m m 1/2!# , (7.2)
where (a) follows from B(m)/m = oP(1) and (b) follows from lim supm|B(m)| =√2m log log m (see [11]). Also, uσ2 n ≤ uσ2 mu + σB(m)− 1 = σ 2/m 1 + (σB(m)− 1)/mu = σ 2 m " 1−σB(m)− 1 mu + σB(m) − 1 mu 2 +· · · # (a) = σ 2 m 1−σB(m) mu (1− oP(1)) = σ 2 m " 1− OP log log m m 1/2!# , (7.3)
where, (a) follows from (σB(m)− 1)/mu = oP(1) and 1/B(m) = oP(1) (see [11]). Combining
Equa-tions 7.2 and 7.3 completes the proof.
References
[1] R. Schreier, C. G. Temes, “Understanding Delta-Sigma Data Converters,” Wiley-IEEE Press, November 2004.
[2] I. Daubechies, R. DeVore, C. S. Gunturk, and V. A. Vaishampayan, “A/D Conversion with Im-perfect Quantizers”, IEEE Transactions on Information Theory, Volume 52, Issue 3, pp. 874-885, March 2006.
[3] S. Kavusi, H. Kakavand, and A. El Gamal, “On Incremental Sigma-Delta Modulation with Optimal Filtering”, TCAS-I, pp. 1004-1015, Volume 53, Issue 5, May 2006.
[4] I. Daubechies, R. DeVore, C. S. Gunturk, and V. A. Vaishampayan, “Beta Expansions: A New Approach to Digitally Correct A/D Conversion”, Proceedings of the IEEE International Symposium on Circuits and Systems, Phoenix, May 2002.
[5] B. Razavi, “Principles of Data Conversion System Design,” IEEE Press, 1995.
[6] H. Kakavand, A. El Gamal, “On Energy-Reliability Tradeoff in Analog-to-Digital Convertors with Imperfect Comparators”, Proceedings of the 40’th CISS, Princeton University, March 2006. [7] R. M. Gray, P. W. Wong, “Sigma-delta Modulation with i.i.d. Gaussian Inputs”, IEEE Transactions
on Information Theory, Volume 36, Issue 4, pp. 784-798, July 1990.
[8] E. L. Lehmann, and G. Casella “Theory of Point Estimation”, Second Edition, Springer, New York, 1999.
[9] V. Seshadri, “The Inverse Gaussian Distribution: A Case Study in Exponential Families”, Oxford University Press, Oxford, 1993.
[10] H. V. Poor, “An Introduction to Signal Detection and Estimation”, Second Edition, Springer-Verlag, 1994.
[11] I. Karatzas and, S. E. Shreve, “Brownian Motion and Stochastic Calculus”, Springer-Verlag, New York, 1991.