Planning Architectures

When planning for autonomous multi-agent networked teams, there are several planning architectures that can be considered depending on the mission scenario, available resources, operating environment, and communication infrastructure [105, 171, 201, 204]. This section formalizes definitions for three planning architectures that can be employed given different computational resources and communication environments: centralized, distributed, and decentralized. The differences between these architectures are highlighted with regards to how agents can handle consensus and cooperation in the assignment space, whether the underlying algorithms require synchronization or not, and how these choices affect the

performance of algorithms within these respective environments1_.

1_{For a more detailed description of centralized, distributed, and decentralized architectures, and the}

different issues associated with underlying planning algorithms within these environments, the reader is referred to [105, 171].

Centralized planning architectures refer to multi-agent algorithms that can run on a sin- gle machine, and where information exchange between different modules of the algorithm occurs through shared memory. Fast centralized algorithms are often implemented using parallelized computation structures that can take advantage of the large number of cores available in modern computer systems. Since all modules in the algorithm have instant access to the current global shared memory state, communication cost between modules is effectively negligible (communication cost includes both the additional resources required to facilitate communication and the associated delays introduced by explicit message pass- ing). As a result, centralized planning architectures are often very good choices, even when planning for multi-agent teams. Figure 3-1(a) illustrates an example of planning for a net- worked team of UAVs using a centralized planning architecture. In some environments,

however, enabling agents with more autonomy may be more suitable. For example, if

planning parameters are changing rapidly (e.g. agent state, task parameters, environment variables), and large amounts of data have to be transmitted to the centralized solver’s location, a centralized architecture may not be ideal. Remote processing of data to extract useful information prior to communicating with the base station may be more appropriate (referred to as distributed processing [204]), or the agents could converge locally on new assignments given the updated information without ever communicating with a centralized location (referred to as distributed problem solving [204]). Furthermore, the solution speed and quality of centralized solvers is limited by the accuracy of the communication and the rate at which information is received. Given slow, unreliable, or expensive communication channels, passing large amounts of (possibly irrelevant) data through the network may not be justifiable. If most of the information needed by each agent to create its cooperative assignments is obtained locally, much faster response times might be obtained by keeping all relevant planning computations local as well, and sharing only the results of these local computations with other agents.

Distributed planning architectures consist of algorithms that run as separate modules, where each of these distributed algorithmic pieces uses its own memory partition to store data associated with the planning process, and where relevant information is shared be- tween the modules through reliable communication channels. This communication aspect introduces an additional layer of complexity over centralized algorithms, since there are communication costs and delays typically associated with sharing information. Distributed

algorithms typically rely on a strong communication infrastructure, meaning that each dis- tributed node has knowledge of the existence of all other nodes it is able to communicate with, and assumes that messages between nodes will be sent and received reliably and with low latency. The main trade-off to consider when moving from a centralized to a distributed architecture is that the additional time and resources associated with message communi- cation must be offset by the additional computation available by using multiple machines. For example, if the computational load of an optimization problem is too immense for a single (possibly multi-core) machine, distributed algorithms could be used to split the problem into modules over multiple machines communicating through reliable channels. For the multi-agent applications considered in this thesis, distributed algorithms are better suited than centralized algorithms, since they perform better when information is being acquired remotely (see Figure 3-1(b)). For example, in cooperative planning environments, this could involve an agent observing a local event, changing its own plan based on this newly observed information, and then reliably communicating the results to the rest of the distributed modules, as opposed to communicating the raw data associated with the local event. This local processing tends to dramatically reduce the overall communication load on the system while using a similar amount of computation as a centralized architecture. As mentioned before, distributed algorithms rely on stable communication channels and information sharing. If communication links are not sufficiently reliable, performance may degrade significantly, motivating the use of decentralized algorithms.

A decentralized planning architecture consists of independent agents planning autonomously for themselves in environments with unreliable and sporadic communication infrastructures, where there are no rigid constraints and guarantees placed on message delays, network con- nectivity, program execution rates, or message arrival reliability. In these types of envi- ronments, algorithms cannot rely on information being shared appropriately between the modules, thereby limiting the amount of coordination and cooperation achievable. As a result, these algorithms may be conservative and have lower performance than distributed architectures when communication conditions are actually favorable, but they are also more robust to drastic changes in the communication environment. Fully decentralized algorithms enable the sparsely connected agents with a high degree of autonomy, since they do not place rigid rules on how agents must interact. In addition, in weak communication environments, decentralized algorithms can allow large teams to interact efficiently, without bogging down

(a) Centralized Planning (b) Distributed and Decentralized Planning

Figure 3-1: Illustration of centralized and distributed/decentralized planning for a net- worked team of UAVs performing target search and track tasks at different locations. In centralized planning, agents communicate with a ground station to receive plans. In dis- tributed/decentralized architectures, agents can plan individually and perform consensus through local communication.

the entire network with restrictive message passing requirements.

Given the three planning architectures described above, choices can be made about how to coordinate the parallelized computation and communication amongst the different mod- ules. In particular, consideration needs to be given about whether these interactions will happen synchronously or asynchronously and how that impacts the convergence and perfor- mance of the algorithms. Highly structured synchronous algorithms enforce constraints on when computation and communication can take place, whereas at the other extreme asyn- chronous algorithms provide extreme flexibility for each module to execute these at their own pace. Synchronization is typically used by most iterative algorithms, and involves re- stricting computations to occur only after certain event driven triggers, to ensure that the algorithm has a predictable state during program execution. During typical synchronous operation, the individual modules perform parallel computations, share state variables, and then wait until a certain trigger (e.g. heartbeat, scheduled time) before computing the next

iteration of the algorithm. The main benefit of synchronous planning is that guarantees can be made about the state of each agent, as well as allowing the algorithms to make assumptions about the information states of the other modules. This information can be leveraged to increase planning performance and rate of convergence. The drawback, how- ever, is that in some environments it may take significant effort to enforce synchronous behavior (referred to as the synchronization penalty [29]). In distributed and decentralized algorithms, the computation triggers are often not local and must therefore be inferred through inter-module communication. In centralized approaches, the mechanisms required to enforce synchronization are fairly lightweight, but when information is being shared across a network, and modules must spend time waiting for messages to arrive from phys- ically separated machines, this synchronization penalty can become severe. On the other hand, asynchronous computation can be used when the algorithmic modules are executed relatively independently of one another, such as in decentralized algorithms, where informa- tion is incorporated into the planning process whenever it is available instead of on a rigid schedule. There is fundamental trade-off, therefore, between enforcing synchronicity which allows the algorithm to make assumptions about algorithmic states, versus adopting a more flexible asynchronous architecture while taking a performance hit for losing information assumptions and state guarantees. Typically, in centralized and distributed environments, the synchronization penalty is weak enough to encourage the use of synchronous compu- tation, while in decentralized approaches the synchronization penalty is much higher and asynchronous computation is often preferable.