Denote by zi(t) the unquantized message of node i at iteration t, and by ˜zi(t) its quantized value. We consider two main strategies which are represented in Fig. 2.1.2. For each of these two strategies, we consider the two possible com-munication scenarios in Fig. 2.1.2 called the peer to peer and the broadcast cases,
) of the sender (sensor i) a) Scheme 1: Predictive Coding
b) Scheme 2: Wyner Ziv Coding
States of the receiver (sensor j) Estimates of
previous state values of the sender (sensor i)
(a) Encoding/Decoding strategies for Consensus Problem.
Encoder m
(b) Peer to Peer vs Broadcast.
Figure 2.1: Charts for different quantization schemes.
respectively. As seen in Fig. 2.1.2 a) the peer to peer schemes utilize a different encoder for each particular destination, while the broadcast schemes in Fig. 2.1.2 b) require only one encoder for all receivers of a given sender. The reason of this distinction is that the peer to peer methods generally outperform the broad-cast methods, but at the price of a more complex encoder structure and forcing to send a different message to each neighbor. While sending a different mes-sage over each link is acceptable in a wired network, it is wasteful in a wireless medium, where communications are naturally broadcast and each transmission reaches all neighbors. The peer to peer methods are proposed for wired networks and the broadcast methods are proposed for wireless networks, though our main focus will be broadcast methods.
We consider an undirected graphG(V, E) with the set of vertices V and the set of edgesE. We assume that there are N nodes in the network, i.e., |V| = N and N is finite. The set of edges E consists of doubles (i, j), i, j ∈ V, and if (i, j)∈ E, it means that nodes i and j are neighbors and they can communicate with each other. We note that since the network of interest is assumed to be undirected, (i, j) ∈ E if and only if (j, i) ∈ E. We define the adjacency matrix of the graphG as A [67]:
[A]ij =
aij > 1, if there is an edge between node i and j
aij = 0, if i = j or i ̸= j and there is no edge . (2.1) Laplacian matrix associated with graphG is defined as:
L = D− A, (2.2)
where D = diag(A1) is the degree matrix, diag(.) is the diagonal matrix of its arguments and 1 is all ones column vector. In this study, we consider the
dis-tributed average consensus algorithm which follows the update rule:
xi(t + 1) = xi(t)−
∑N j=1,j̸=i
ϵlij(
xj(t)− xi(t))
(2.3)
for i∈ V and t = 0, 1, . . .. We can rewrite (2.3) in vector form as:
x(t + 1) = (I− ϵL)x(t) = W x(t) (2.4)
where x(t) = [x1(t) x2(t) . . . . xN(t)]T and [L]ij = lij. It was shown by Xiao and Boyd that above system converges to the average of any initial vector x(0) ∈ RN if and only if W is balanced1, and ∥ W − N111T ∥< 1 where the norm is maximum singular value norm [92]. In other words, convergence is satisfied if 1 is an eigenvalue of W and it is also the only eigenvalue with the greatest magnitude. Similar results for continuous time iterations are in [82]. For A = AT, by constraining 0 < ϵ < 1/ max(A1), we guarantee that (2.4) asymptotically converges to average as in [92].
If we decompose (2.3) as follows:
x(t + 1) = (I− ϵD)x(t) + ϵAx(t) (2.5)
we see that there are two different parts in the update rule: computing (I − ϵD)x(t) requires only local values and ϵAx(t) uses the neighbors’ values. We can therefore define:
z(t) = ϵx(t) (2.6)
as the vector of variables which needs to be exchanged over the links available on G at each iteration t. The entries of the vector z(t), quantized with finite precision, are received by the neighbor as ˜z(t) = z(t) + w(t), where w(t) is the quantization error.
11TW =(
W 1)T = 1T
While there exists a substantial body of work on average consensus proto-cols under infinite precision and noiseless peer to peer communications, little research has been done introducing distortions in the message exchange, such as the noisy update assumption made in [93]. Specifically, Xiao and Boyd con-sider the following extension of (2.3):
xi(t + 1) = xi(t)− ϵ
∑N j=1,j̸=i
lij
(xj(t)− xi(t))
+ wi(t) (2.7)
where wi(t), i ∈ V, t ≥ 0 are independent zero mean fixed variance identically distributed random variables. The authors show that under these assumptions, the system converges to the initial average only in mean (i.e. we do not have mean squared convergence) and that the mean squared error (MSE) deviation from the actual average, increases beyond a certain iteration. The authors also show that the node values do not converge to a common value as the number of iterations increases.
We propose, under a detailed internode communication model, to charac-terize wi(t) as an additive quantization noise in (2.7). By utilizing the increas-ing correlation among the node state values, we show that the variance of the quantization noise diminishes even with zero asymptotic rate. Furthermore, we show that the node values converge to a consensus as t → ∞. We derive an expression for the mean squared deviation of the consensus value for both practical scalar quantizer and infinite length vector quantizer schemes. We also show that the mean squared deviation is bounded even under vanishing quan-tization rate regimes.
In the rest of the chapter, we will be using the following assumptions:
(A1) The entries of x(0) are random variables with zero mean2 and finite vari-ance (not necessarily independent).
(A2) The nodes are strongly connected3, and W satisfies convergence condi-tions satisfied by unquantized average consensus protocol [92].
(A3) The quantization noise samples at each step and sensor are uncorrelated with the messages and are also spatially and temporally uncorrelated, zero mean random variables4.
(A4) Knowledge of the initial node statistics and topology (W matrix) is avail-able at each node.
Remark 1 With regard to (A3), in [86] Synder proved that a sufficient condition for quantization noise to be modeled as uniformly distributed and uncorrelated with the input message in the uniform scalar quantizater model is that characteristic function (CF) of the input message is bandlimited where 2π∆ is the upper bound on the bandwidth and ∆ is the quantization bin width. In practice, characteristic functions are not exactly band-limited and the quantization theorems apply only approximately. However, as our adaptive consensus algorithm iterates ∆ converges to 0 as discussed in Sections 2.4 and 2.5. Therefore above assumption will hold closely for larger iterations. Even in the initial iterations, under sufficiently large quantization rates such an assumption closely reflects the actual system behavior. In [86], Synder also showed that similar rules apply for spatial and temporal uncorrelation, i.e. the joint CF has to be band-limited.
In Section 2.6, we numerically show that the theoretical behavior of the system matches with the simulated behavior and conclude that such assumptions hold closely
2In the case of unknown mean, due to the fact that the predictive coding scheme transmits the difference between the current state and its Linear Minimum Mean Squared Estimate, the mean of the difference can be approximated as 0. Hence, the performance is expected to be similar.
3A graph is strongly connected if there is a path from each vertex to each of the others.
4E[wi(t)wj(l)] = 0unless t = l and i = j.
even under lower rate regimes. In the case of infinite length vector coding, Zamir et.al.
showed in [105] that the quantization noise approaches to a white Gaussian process.
Remark 2 The knowledge of the topology mentioned in (A4) is not practical in a decen-tralized setting. While this assumption is used to keep track of the network statistics at each node, any predictive strategy or coding with side information strategy which does not make explicit use of the statistics can still be analyzed with our methods. Moreover, in [99], we have showed that as the node density increases or in homogeneously dis-tributed networks, the encoder-decoder coefficients become independent on the network size and specific location of a node. Therefore, (A4) can be relaxed even for relatively small network sizes (i.e.. N = 64 as discussed in [99]).