A Class of The Multi-Agent Problems

The motion planning problem for single agent in presence of uncertainty in the form of process uncertainty, and stochastic maps, has been solved using the proposed generalized sampling based motion planners, i.e. GPRM and GRRT. Once the single agent problem has been solved some of the important extensions are to : dynamic maps, multi-agent systems and problems with sensing uncertainty. We shall not solve

problem with sensing uncertainty in this dissertation. We would like to generalize our solution methodology to multi-agent scenarios in the presence of process uncertainty and stochastic maps. The particular multi-agent problem that we would like to solve in this dissertation will be discussed in subsection 5.2.1.

Researchers solve the coordination problem to solve the MMDP posed by the multi-agent motion planning problem (refer subsection 5.1.4). All the successful approaches [74,78] were top-down in solution methodology, i.e. the coordination problem was solved in the MMDP framework.

In this work, we intend to present an approximate approach of solving a partic- ular MMDP for the start locations and a particular cost structure as explained in subsection 5.2.1. In Section 3 and 4 we successfully solved the single agent motion planning problem in presence of uncertainty posed as MDP which was converted to SMDP and solved thereafter. Our approach to solving the MMDP posed by multi- agents is bottom-up where we intend solve multiple MDPs for single agents and use the cost of transition and transition probabilities generated by the solution to solve the coordination problem and hence solve the overall MMDP posed by the systems of agents. This coordination problem is equivalent to the routing problem solved by the TSP research community as discussed in subsection 5.1.5. Hence we intend to use the already existing literature related to solution of MTSPs to solve the routing problem (or the coordination problem in MMDPs). Details of the approach to solve the MMDP will be discussed in further sections.

5.2.1 A Class of Multi-Agent Problems in Presence of Uncertainty We want to solve the motion planning problem, under given scenario : • m agents, with m initial configurations, i.e. q_I = {qI1, qI2, . . . , qIm},

• n goal locations, i.e. q_G= {qG1, qG2, . . . , qGn},

• Environment given by stochastic maps, i.e. static obstacle probabilities. Given the above multi-agent scenario, additional considerations need to be made: 1. Number of agents and number of goal locations might not be same, i.e. m 6= n (general case). If, the number of agents and number of goal locations are same, then solving m single agents problems can be straightforward. But as the goal locations are different in number, some agents will have to go to more than one goal and some might not have to go to any goal. Hence with the given scenario one has to solve a routing problem (or the coordination problem as discusses in subsection 5.1.4) for the multi-agent system.

2. Due to presence of other agents in the given static stochastic map, in addi- tion to collision with obstacles, we need to address collision with other agents. Furthermore, this has to be a real-time solution.

3. Typically in a multi-agent system, one can assume presence of heterogeneous agents, i.e. agents having different capabilities. Thus, there is a need to discuss homogeneous as well as heterogeneous agents scenario.

The routing problem, in item 1 above, has been solved extensively in the travel- ing salesman problem (TSP) research community, primarily in deterministic frame- work [79]. The generalized multi-agent routing problem has been posed as a multiple traveling salesman problem (MTSP) [80,31] and there have been multiple approaches to solve it. We aim to use the already existing routing problem solution techniques for multi-agent systems developed in [31], and in synergistic manner apply it along with GPRM to the multi-agent systems in presence of process uncertainty and stochastic maps. We expect this generalized technique - multi-agent adaptive sampling gen- eralized probabilistic roadmaps(MAGPRM), will help us solve the feedback motion planning problems in high dimensional state spaces under uncertainty in a multiple agent scenario, with GPRM as the underlying framework, which successfully solved the single agent scenario.

The section 5.3 will discuss the solution approach for the multi-agent motion planning problem under uncertainty.

5.3 Solution Approach : Problem 2

In order to solve the motion planning problem under uncertainty involving multi- agent systems, stated in Problem 2 (refersubsection 1.2.2), a solution methodology is proposed using the multiple traveling salesman problem (MTSPs) [31].

We have solved the motion planning problem under uncertainty for a single agent case (Problem 1). We propose to address the multi-agent motion planning problem under uncertainty by solving the following sub-problems using proposed methodolo- gies along with GPRM as follows:

Routing Problem : This is the problem of identifying which agents will go to which goal locations. Hence given m agents and their initial configurations, q_I = {qI(i)}, i = 1, . . . , m, and n target final configurations, qG= {qG(i)}, i = 1, . . . , n, and given m 6= n (general case), how to determine which set of goals any given agent will go to. Due to the condition of m 6= n, some agents may go to multiple goal configurations and/ or other agents may not go to any goal configuration.

Solving the “routing problem” amounts to solving a passive/offline co-ordination problem, i.e., co-ordination between agents before starting the execution of the planning. MTSP is the tool through which we plan to solve this routing prob- lem. We will solve the original problem of multi-agent motion planning under uncertainty using GPRM in conjunction with MTSP, in a hierarchical fashion. The solution of GPRM will generate transition costs and probabilities between any pair of goal locations3 for any given agent. Using these cost and transition probabilities, the MTSP algorithm will be used to solve the “routing problem”

and hence, solve the multi-agent motion planning problem under uncertainty. Inter-Agent Collision Avoidance : Inclusion of multi-agents in the domain cre-

ates a need to address the inter-agent collision avoidance through an updated collision detection module which, in general, cannot be assured by the routing problem. To address this problem, we shall use collective partial observability (ref. subsection 5.1.1), and develop schemes with guaranteed collision avoid- ance.

Homogeneous and Heterogeneous Agents : In case of all agents being homo- geneous in dynamics and capabilities, a single graph based solution can be provided using GPRM along with MTSP. For heterogeneous agents, solving the motion planning problem will involve constructing topological graphs in GPRM for every type of agent present in the system, i.e. given m agents consisting of t type of agents will need t number of graphs and a number of GPRMs solutions.