Traditional control theory usually assumes that the various components of the system (sensors, controllers and actuators) can exchange signals reliably and with an arbitrarily high precision. His- torically this assumption has made sense since data was typically transmitted over small geographical distances using dedicated communication links. Thus, effects introduced by the communication me- dia were minimal and could be ignored. In networked multi-agent systems, components are typically situated far away from each other and may communicate over wireless links or communication net- works that are also used for transmitting other unrelated data. Thus, the effect of such imperfect communication links needs to be considered and accounted for. Communication links introduce
many phenomena that are potentially detrimental to the estimation / control performance, such as quantization error, random delays, data corruption and packet drops to name a few. In extreme cases, poor network performance can even destabilize a nominally stable control loop. Since such effects are not well-understood even for systems that comprise of a single dynamical process that needs to be estimated or controlled, we begin our investigations by focusing on such systems in this chapter.
We begin by considering a dynamic process that is being observed by a single sensor. The sensor communicates to a controller over a communication link. Recently, much attention has been directed towards such systems (see, e.g., [3, 119] and the references therein). As an example, quantization effects have been analyzed with increasing regularity since the seminal paper of Delchamps [46]. The problem of stabilization with finite communication bandwidth was considered by Wong and Brockett [204, 205]. Baillieul [6] also reported a tight bound on the data rate requirement for stabilizing a scalar system. Nair et al. [149, 150] considered the stabilization of stochastic linear systems and Markov jump linear systems with finite data rates. Tatikonda [185] studied stabilization of finite-dimensional discrete-time noiseless linear processes and also presented results about the optimal LQG control of linear systems across noisy feedback links (see also [23]). Elia and Mitter [54] considered the question of the optimal quantizer for stabilization. Various quantization and coding schemes for stabilization have been proposed in the literature, (see, e.g., [25, 57, 98, 104, 129, 160]). Similarly, the effects of delayed packet delivery have also been considered in many works, such as Nilsson [152], Blair and Sworder [19], Luck and Ray [135], Gupta et al. [87], Tsai and Ray [188], and Zhang et al. [208] to name a few, using various models for the network delay.
In this dissertation, we will mostly be interested in systems communicating over links that can be modeled as dropping packets randomly. The nominal system is shown in Figure 3.1, where the channels randomly drop packets being communicated from the plant to the controller and back. Pre- liminary work in this area has largely focused on the case of a single sensor transmitting information over a single channel and studied the stability of systems utilizing lossy packet-based communica- tion, as in [94, 174, 208]. Performance of such systems as a function of packet loss rate was analyzed by Seiler in [174] and by Ling and Lemmon in [131] assuming certain statistical dropout models. Various approaches to compensate for the lost data have also been proposed. Nilsson [152] proposed two approaches for compensation for data loss in the link by the controller, namely keeping the old control or generating a new control by estimating the lost data, and presented an analysis of the stability and performance of these approaches. Hadjicostis and Touri [90] analyzed the performance when lost data is replaced by zeros. Ling and Lemmon [131, 133] proposed compensators for specific statistical data loss models in the case of single input single output (SISO) systems. In particular, in [131] they posed the problem of optimal compensator design for the case when data loss is inde- pendent and identically distributed (i.i.d.) as a nonlinear optimization. Azimi-Sadjadi [5] took an
Figure 3.1: The architecture of a packet-based control loop. The links are unreliable and unpre- dictably drops packets. Most of the works in the literature look at the case of a single sensor transmitting information over a single channel.
alternative approach and proposed a sub-optimal estimator and regulator to minimize a quadratic cost. Sinopoli et al. [173] and Imer et al. [103] extended this approach further to obtain optimal controllers when the packet drops were i.i.d. The related problem of optimal estimation across a packet-dropping link was considered by Sinopoli et. al in [178] and extended by Gupta et al. in [77]. Most of the designs proposed in these references aim at designing a packet-loss compensator, as shown in Figure 3.2. The compensator accepts those packets that the link successfully transmits and comes up with an estimate for the time steps when data is lost. This estimate is then used by the controller. Our work takes a more general approach by seeking the LQG optimal control for this packet-based problem. In particular, our architecture is as shown in Figure 3.3. We will jointly design the controller, the encoder and the decoder to solve the optimal LQG problem. Even though the terminology reminds one of information theoretic designs, the encoder and the decoder can not be designed in our problem using information theoretic algorithms since the system has real-time constraints. The controller needs to generate a control input at every time step and thus, e.g., block coding based coding strategies cannot be used. We need to identify the optimal coding strategy for the purpose of estimation and control.
Based on a separation principle that we prove, the control problem is separated into one of de- signing a state-feedback optimal controller and another of transmitting information across unreliable links. This allows us to identify the information that needs to be made available to the controller for optimal performance. We then propose a simple recursive algorithm that ensures that this in- formation is available to the controller for the case of a single sensor transmitting information over
Figure 3.2: A common design for control over packet-based links. The compensator aims at miti- gating the effects of packet losses. In most works, the controller-actuator channel is assumed to be absent.
a single link. Even though the algorithm requires a constant amount of memory, transmission and processing at the sensor node, it is optimal for any packet drop pattern and has many additional desirable properties that we will illustrate later.
We then extend the optimal coding algorithm to the case of a single sensor transmitting infor- mation to the controller over a network of communication links that drop packets stochastically. Transmission of data over networks for the purpose of estimation and control is largely an open problem. In [186], Tatikonda studied some issues related to the quantization rates required for sta- bility when data was being transmitted over a network of digital memoryless channels. Also relevant is the work of Robinson and Kumar [166], who consider the problem of optimal placement of the controller when the sensor and the actuator are connected via a series of communication links. They ignore the issue of delays over paths of different lengths (consisting of different number of links) and under aLong Packet Assumptioncome up with the optimal controller structure. There are two main reasons why the problem of encoding data for transmission is much more complicated in the case of transmission over a network:
1. If the intermediate nodes are allowed to process data, the network cannot be replaced by an erasure channel with the equivalent drop probability as the reliability of the network. Processing by intermediate nodes leads to an element ofmemory.
2. There are potentially multiple paths from the source to any node. These paths may offer data with varying amounts of delay and the processing algorithm needs to take care of this fact. We again solve for the optimal encoder and decoder structures that are recursive in nature and hence require only a constant amount of memory, transmission and processing at every node. The analysis of the algorithm identifies a property of the network called the max-cut probability that is relevant for the purpose of stability of the control loop (or equivalently, that of the estimate error). We also provide a framework to analyze the performance of our algorithm. Our viewpoint allows us to view the intermediate nodes as repeaters in a digital communication channel that fight the degradation introduced by the channel.
Having solved the problem for the case of a single sensor, we move on to the case when multiple sensors are present. We start with the simplest case when only one of the sensors transmits data over a link that drops packets. This problem is also largely open. We encounter this case in our work on the multi-vehicle wireless testbed [40, 198]. In the testbed, each vehicle is equipped with an on-board gyro. In addition, each vehicle also obtains measurements from an overhead camera. While the gyro-controller link is hard-wired and hence does not drop packets, the camera communicates to the controller over a wireless link that drops packets randomly. Our solution to this problem again adopts the philosophy of processing information at the sensor end before transmission to combat the effects of the channels. Our architecture is as shown in Figure 3.4. We again provide recursive
yet optimal designs of the encoders, the decoder and the controller.
Figure 3.4: The structure of our optimal LQG control solution for the multiple-sensor case. As an intermediate step, we also solve the following problem. Suppose, as shown in Figure 3.5, two sensors are estimating a process jointly while communicating over links that drop packets stochas- tically. What information should the sensors exchange so to obtain a good estimate? Related work to this problem has dealt with fusion of data from multiple sensors and track-to-track fusion. The classical Kalman filter is a centralized filter that assumes all observations coming to a central com- puting facility. The usual starting point is to come up with techniques to decentralize the filter computations. An early contribution was [200], where information obtained from the local sensors is combined to generate the global estimate. However, it required that data about the global estimate be sent from the fusion node to the local sensors. A similar requirement was imposed in the ‘suc- cessive orthogonalization of measurement subspaces’ algorithm proposed in [93]. This difficulty was first overcome in [30, 183], in which each local node sends its own local estimate based on its own data and communicates this estimate and the error covariance data to the fusion center. Similar results for continuous time systems were presented by Willsky et al. in [201]. These results were further extended by Hashemipour et al. in [92], where both the measurement and time update steps of the Kalman filter were decentralized. An alternative approach for data fusion from many nodes using the Federated filter was proposed by Carlson in [27]. A Bayesian method was used and some algorithms presented in [32, 101] which are optimal when there is no process noise. A scattering framework [128] and algorithms based on decomposition of the information form of the Kalman filter [15, 163] have also been proposed for data fusion. A scheme based on the concept of dynamic consensus was proposed in [182], but it assumes multiple communication rounds per time step of
Figure 3.5: Structure of the joint estimation problem.
the system evolution. For some other approaches proposed in the literature (e.g. those based on tracklets [49]), see [31, 145].
However, these approaches assume a fixed communication topology among the nodes with a link, if present, being perfect. In our case, packets of information from one node to another will be dropped randomly by the communication channel present between them. This random loss of information reintroduces the problem of correlation between the estimation errors of various nodes [9] and renders the approaches proposed in the literature as sub-optimal. An approach to solve this problem was proposed by Bar-Shalom and Campo in [10] in the context of track-to-track fusion through exchange of state estimates based on each sensor’s own local measurements, but the specific scheme that was used was not proven to be optimal. Moreover, as was found in [167], the algorithm for fusing the local state estimates that was proposed is not optimal in the mean square sense. It was subsequently proven in [28, 145] that the technique was based on an assumption that was not met in general and, in fact, calculation of global estimate using just the local estimates is possible only in very specific cases. We wish to address this problem of finding the optimal global estimate for each node in the case when there are communication channels present between the nodes and packets of information are being randomly dropped. Once again, we disallow approaches such as transmitting all the measurements taken by each node every time communication is possible because that can potentially entail transmitting arbitrarily large amounts of data. Instead, we will propose a recursive yet optimal strategy.
Our algorithm can also be extended to the case when there are multiple sensors that share a channel. Thus, at any time step, either all the sensors drop packets or all the transmissions are
successful. For the case of multiple sensors transmitting over multiple channels that can drop packets independently of each other, the problem is still open. In Chapter 5, we will discuss some partial results for the optimal algorithm and analyze some strategies using tools that we develop in the next chapter.
In the first appendix, we look at another effect that communication channels may introduce: quantization. As we saw above, many works have looked at the presence of digital memoryless channels in control loops. However, most of the work reported so far has focused on the effect of quantization on stability. It is worthwhile to also consider the question of performance of the system in the presence of quantization. This problem is much less well-studied. The performance of a scalar statically quantized system with delays was considered in [191]. Lemmon and Ling [126] presented an upper bound for the quantization noise for the case when dynamic uniform quantization is done over a channel that drops packets. They defined the performance in terms of signal to quantization ratio and presented some interesting trade-offs between the number of bits, locations of the system poles and the performance.
We study the effect of quantization on the LQR performance of the system. We consider a linear time-invariant scalar system with a control law in place and see how the performance degrades as less and less data is allowed to pass from the process to the controller. We come up with some interesting bounds for specific quantizers and some entropy-based general bounds on general centroid-based quantization and encoding schemes. We also consider extensions to dynamic quantization schemes and packet-dropping channels.
In the second appendix, we consider a system in which a channel between the sensor and the con- troller can exist in one of many states. These states can, e.g., correspond to different random delays applied or different noise powers that corrupt any signal transmitted over the channel. The channel transitions between the states according to a Markov chain. Thus, the system can be modeled as a jump linear Markov system. Such systems have been studied and analyzed extensively. As an example, Ji and Chizeck [108] studied the problem in detail and defined concepts like stability and controllability. Discrete-time versions of the jump-linear quadratic (JLQ) optimal control problem were solved for finite-time horizons in Blair and Sworder [19]. Nilsson and Bernhardsson [153] gen- eralized the results of Ji and Chizeck to the case where the Markov chain determines the probability density function of the variables rather than the values of the variables themselves.
However, all the above approaches assumed the Markov state to be known. In the context of the work presented in this dissertation, this assumption means, e.g., that the receiver knows whether the link has dropped a packet or the delay it has introduced and so on. In the second appendix, we discuss the case when this assumption does not hold and a Markov state estimation algorithm is used to estimate the state of the channel as well. We analyze the case where the state estimate update depends only on the latest observation value. In particular, we consider a suboptimal version of the
causal Viterbi algorithm and show that a separation property need not hold between the control law and the state estimation algorithm.