Advanced Artificial Intelligence (Multi-Agent Systems) Dr. Muhammad Adnan Hashmi

Advanced Artificial Intelligence

(Multi-Agent Systems)

3

Multi-Agent Planning

■

Planning for single agents

Computing plan from Initial State to Goal State

■

Coordination of plans

Removing conflicts (negative interactions)

Utilizing help relations (positive interactions)

a b c A1 A2 d e q I G

Related Work (Plans Coordination)

■

Coordination before planning

Social Laws [Shoham 1995], [Buzzing 2006]

■

Coordination during planning

Partial Global Planning [Durfee and Lesser 1987]

Incremental Plan Merging [Alami et al. 1994]

Recursive Petri Nets [El Fallah Seghrouchni and Haddad 1996]

■

Coordination after planning

Temporal Plan Merging [Tsamardinos et al. 2000]

Merging Hierarchical Plans [von Martial 1992]

5

Objectives (Plans Coordination)

■

Propose plans coordination mechanisms for the plans

having different priorities

■

Two different scenarios

Coordination during planning

■ Coordinated Planning Problem (CPP)

Coordination after planning

Assumptions

■

Two agents

α

and

β

sharing the same environment

Agent

α

having higher priority (reactive) goals

Agent

β

having normal priority (proactive) goals

■

Two possible conflicts among plans

Causal link threat

7

Two Possible Conflicts

■

Causal Link

(A1, A2, p)

Action A1 adds an effect p

Action A2 needs this effect

No action between A1 and A2 adding p

■

Causal Link Threat

If an action A deletes p and lies between A1 and A2, then A

threatens the causal link (A1, A2, p)

A1 p A2

A ¬p

Two Possible Conflicts

■

Causal Link

(A1, A2, p)

Action A1 adds an effect p

Action A2 needs this effect

No action between A1 and A2 adding p

■

Parallel Actions Interference

Actions A1 and A2 lie in parallel

Either one of them deletes the preconditions or add effects of the other A1 A2 ¬p p A1 A2 p ¬p

Coordinated Planning Problem

■

Prerequisite:

Plan

P

of Agent

α

■

Our Aim:

Compute a Plan

P

for Agent

β

■

Has no conflict with

P

■

Avails the cooperative opportunities offered by

P

■

Solution:

Non Temporal Domains µ-SATPLAN

Temporal Domains Coordinated-Sapa

SATPLAN

(Kautz and Selman 2006)

A Classical Planner that Finds Optimal

Plans in Non-Temporal Domains

SATPLAN

Increment plan length If unsatisfiable Planning Problem • Init State • Goal • Actions CNF Simplifier (polynomial inference) Solver (SAT engine/s) Decoder CNF

Propositional formula in conjunctive normal form (CNF)

We Use GraphPlan Encoding

13

Constructing the planning graph

Level

P

1

: All literals from the initial state

Add an action in level

A

i

if all its preconditions are present in

level

P

■

Add a proposition in level

P

i

if it is the effect of some action in

level

A

■

Maintain a set of exclusion relations to eliminate incompatible

GraphPlan Encoding

Fact ⇒ Act1 ∨ Act2

Act1 ⇒ Pre1 ∧ Pre2

¬Act1 ∨ ¬Act2

■

Can create such constraints for every node in the

planning graph

µ-SATPLAN

An Extension of SATPLAN that

Computes Coordinated Plans in

Handling Causal Link Threat

■

While constructing the planning graph for Agent

β

,

don’t add

an action

O

at level

A

if

It has an effect

¬p

, and

There is a causal link

(Aj, Ak, p)

in plan

P

, and

j

≤

i

≤

k

Action O threatens Causal Link (Aj, Ak, p)

17

Handling Positive Interactions & Parallel

Actions Interference

■

For each time step

i

in the plan of Agent

α

, create an

action NoName(

i

)

Add(NoName(i)) All the effects added by P_α at time step i

Del(NoName(i)) All the effects deleted by P_α at time step i

Pre(NoName(i)) All the preconditions of actions of P_α at time step i

■

Explicitly add all the NoName actions in the planning

Handling Positive Interactions

■ Pα = {α1(0), α2(0), α3(1)}

■ Eff(α1) = a0, Eff(α2) = a1, Eff(α3) = a2

■ Eff( NoName(0) ) = {a0,a1}, Eff( NoName(1) ) = {a2}

a0 a1 a2 a4 a5 a7 a8

Level 0 Level 1 Level 2

a4 NoName(0) β1 a4 a0 a1 NoName(1) a2 a0 a1 β2 a5 β3 a5 a4 a7 β4

19

Handling Positive Interactions

■

Partial CNF Sentence

■

Add NoName actions as

unary clauses to CNF

Level 0 Level 1 Level 2

a4 a6 NoName(0) β1 a6 a4 a0 a1 NoName(1) a2 a0 a1 β2 a5 β3 a6 a5 a4 a7 β4

Example Plan Generated (Logistics)

Time Action Performed

0 DRIVE-TRUCK (TRUCK1, LOC1-1, LOC1-2, CITY1) 1 LOAD-TRUCK (PACKAGE1, TRUCK1, LOC1-2) 1 DRIVE-TRUCK (TRUCK2, LOC2-2, LOC2-1, CITY2) 1 LOAD-PLANE (PACKAGE2, PLANE1, LOC1-1) 2 DRIVE-TRUCK (TRUCK1, LOC1-2, LOC1-1, CITY1) 2 FLY-PLANE (PLANE1, LOC1-1, LOC2-1)

3 UNLOAD-PLANE (PACKAGE2, PLANE1, LOC2-1) 3 UNLOAD-TRUCK (PACKAGE1, TRUCK1, LOC1-1) 4 LOAD-PLANE (PACKAGE1, PLANE2, LOC1-1) 4 LOAD-TRUCK (PACKAGE2, TRUCK2, LOC2-1) 5 FLY-PLANE (PLANE2, LOC1-1, LOC2-1)

5 DRIVE-TRUCK (TRUCK2, LOC2-1, LOC2-2, CITY2) 6 UNLOAD-PLANE (PACKAGE1, PLANE2, LOC2-1) 6 UNLOAD-TRUCK (PACKAGE2, TRUCK2, LOC2-2)

Time Action Performed

0 LOAD-TRUCK (PACKAGE4, TRUCK2, LOC2-2) 1

2

3 LOAD-PLANE (PACKAGE3, PLANE1, LOC2-1) 4 FLY-PLANE (PLANE1, LOC2-1, LOC1-1)

4 UNLOAD-TRUCK (PACKAGE4, TRUCK2, LOC2-1) 5 UNLOAD-PLANE (PACKAGE3, PLANE1, LOC1-1) 6 LOAD-PLANE (PACKAGE4, PLANE2, LOC2-1) 6 LOAD-TRUCK (PACKAGE3, TRUCK1, LOC1-1) 7 FLY-PLANE (PLANE2, LOC2-1, LOC1-1)

7 DRIVE-TRUCK (TRUCK1, LOC1-1, LOC1-2, CITY1) 8 UNLOAD-PLANE (PACKAGE4, PLANE2, LOC1-1)

P

SAPA

[Do and Kambhampati 2001]

A Multi-Objective Heuristic Based

Temporal Planner

Sapa

Search through the space of time-stamped states

S=(P,M,Π,Q,t)

predicates pi and the time of their last achievement t_i < t.

Set of functions represent resource values.

Set of protected

persistent conditions.

Event queue containing delayed effects. Time stamp of S.

23

Sapa

Search through the space of time-stamped states

S=(P,

M,Π,

Q,t)

predicates pi and the time of their last achievement t_i < t.

Set of functions represent resource values.

Set of protected

persistent conditions.

Event queue containing delayed effects. Time stamp of S.

Main Flow-Diagram of Sapa

S=(P,

M,Π,

Q,t)

25

Some Important Concepts

■ Goal Satisfaction:

■ S=(P,M,Π,Q,t) ⇒ G if

∀<

∃ <pi,tj> ∈ P, tj < ti and no event in Q deletes pi

∃ e ∈ Q that adds pi at time te < ti

■ Action Application:

■ Action A is applicable in S =(P,M,Π,Q,t) if:

All preconditions of A are satisfied by P

A’s effects do not interfere with Q

■ When A is applied to S =(P,M,Π,Q,t) :

P is updated according to A’s instantaneous effects Delayed effects of A are put in Q.

■ Special Advance-Time Action

Advances time to next earliest event in the queue, and adds the event to P

Coordinated-Sapa

An Extension of Sapa that Finds

Coordinated Plans in Temporal

27

Handling Positive Interactions

■

Before starting planning for Agent

β

Create an event e = (st, et, effect) corresponding to every effect k of every action A in P

α

■ e.st = st(A) ■ e.et = et(A) ■ e.effect = k

Put all these events in the event queue Q

■

WHY?

At every state, Coordinated-Sapa will take into

account all the changes made by Agent

α

Advanced-Time action will add the effects generated by

Agent

α

, to

P

Handling Negative Interactions

■

Add another action applicability condition to the planning

mechanism of Agent

β

■ Action A is applicable in S=(P,M,Π,Q,t) if:

All preconditions of A are satisfied by P A’s effects do not interfere with Q

A does not threaten any causal link at t

29

Example Plan Generated

■ Agent β drives person P4 in Car1 to city C2

So that P4 can board the plane Pl1

Agent α is bringing the plane Pl1 to C2 from C3

■ Agent β makes person P4 board the plane Pl1 at C2

Experimental Results

■ Domains and problems taken from 3rd International Planning Competition

■ A multi-agent problem is generated by taking the original problem and dividing the

Proactive-Reactive Coordination

Problem

Proactive-Reactive Coordination

■

Prerequisite:

Reactive plan

P

of Agent

α

Proactive plan

P

of Agent

β

■

Our Aim:

Modify plan

P

such that:

■

It has no conflict with

P

■

Avails the cooperative opportunities offered by

P

■

Solution:

33

Case Study

■

Tasks of Rescue Agent

Rescue the Victims

■

Tasks of Analyzer Agent

Analyze the goal cells Call the central agent

■

Constraints

One agent in a cell Hyper energy cells

■ Needs fuel or energy to enter

Agent should have key to open door

Rescue Agent : α Analyzer Agent : β D 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 C B A

Conflict Resolution

Threat-Repair Link (

A1

,

A2

,

p

)

Action A1 deletes p

A2 is a subsequent action and adds p A1 is called Threat Action

A2 is called Repair Action

B1 p B2

A1 -p A2 p

Threat

35

Valid and Possibly Valid Time Stamps

Possibly Valid Time Slot for an action

A

All preconditions are met

No parallel actions interference

P[1] a b h b P[2] c -d P[3] c e e P[4] f f P[5] g P[1] b d -h P[6] g i P[7] i h g

■

Valid Time Slot for an action

A

All preconditions are met

No parallel actions interference Either:

■ No causal link threat

■ Repair Action exist before the deadline

P[1] a b h b P[2] c -d P[3] c e e P[4] f f P[5] g P[1] b d -h P[6] g i P[7] i h g P[2] d k k P[3] h

Plan Merging Algorithm

■

Fix all the actions of Reactive Plan

P

α

on timeline

■

For every action

CA

of Proactive Plan

Search for the first Possibly Valid Time Slot

T

on timeline

Reason about the time slot

T

37

Plan Merging Algorithm

Case 1: No causal link threat by CA at T

Assign Time Slot T to CA

EXAMPLE

■ Current Action: Move(A1, A2)

Returned Time Slot: 0 - 5

Any Threat? : No

Assign Time Slot 0 – 5 to CA

D

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

C B A

Plan Merging Algorithm

Case 2: CA threatens a Causal Link but Repair Action exist

Assign Time Slot T to CA

Save a Possible Threat <ThreatAction, RepairAction, Deadline>

EXAMPLE

■ Current Action: Move(A4, A5)

Time Slot: 20 - 25

Any Threat? : Yes (Agent α needs A5 at time 40-45)

Repair Action: Move(A5, A6)

Assign Time Slot 20 - 25 to Move (A4, A5)

Save <Move(A4, A5), Move(A5, A6), 40>

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

C B A

39

Plan Merging Algorithm

Case 3: It is a Repair Action but can not meet a deadline of some Threat Action

Backtrack to the Threat Action,find another time stamp

EXAMPLE

■ Current Action: Move (A8, A9)

Returned Time Slot: 50 - 55

Any Threat?: Yes (Agent α needs A9 at 85-110)

Repair Action : Move (A9, B9)

Save <Move(A8, A9), Move(A9, B9), 85>

■ Next Action: AnalyzeCell (A9)

Time Slot Assigned: 55 - 70

D

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

C B A

Plan Merging Algorithm

■ Next Action: CallCentral (A9)

Time Slot Assigned: 80 – 90

■ Next Action: Move (A9, B9)

Is it a Repair Action? : Yes

Meet all deadlines?: No (Agent α needs A9 at 85)

Backtrack to action Move(A8, A9)

■ Find another Time Slot

■ New Time Slot: 110 – 115 (Valid Time Slot)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

C B A

41

Plan Merging Algorithm

Case 4: All the effects of CA are already achieved

WHAT TO DO?

■ Mark CA as redundant

POST PROCESSING

■ Remove all redundant actions from the plan

■ Recursively remove all actions which achieve only the

Plan Merging Algorithm

EXAMPLE

■ Current Action: OpenDoor (C11)

Returned Time Slot: 172 - 175

Redundant(OpenDoor(C11)) true

■ Because openedDoor(C11) is true at time 172

When the plan is returned

■ Remove OpenDoor(C11) from plan

■ Also remove TakeKey(C11, key1) from plan

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

C B A

43

Plan Merging Algorithm

Case 5: Action CA’s preconditions can not be achieved

Remove action CA from the plan and compute a plan to achieve effects of CA

■ I = State just before CA

■ G = Effects (CA)

Plan should have no conflict with Reactive Plan P_β and if CA is a repair action, repair effects must meet their deadline

■ ReplacementPlan = Coordinated-Sapa (I, G, P

β)

If a plan is returned, replace the removed actions with the plan If a deadline is violated, backtrack to the threat action

If no plan possible, then remove another action CA + 1

■ G = G U Effects (CA + 1) \ Pre (CA + 1)

Plan Merging Algorithm

EXAMPLE

■ Current Action: TakeEnergy(B13, energy1)

■ Preconditions can not be achieved

Repair the plan

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

C B A

45

Plan Repair Algorithm

■

Create a CPP by removing

TakeEnergy(B13, energy1)

I = { at(β, B13), at(energy1, B13), at(energy2, B13) } G = { hasEnergy(β, energy), at(β, B13)}

■

Call Coordinated-Sapa to solve this CPP

Coordinated-Sapa returns fail

Why? energy2 is also needed by Agent α

D

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

C B A

Plan Repair Algorithm

■

Create another CPP by removing

Move(B13, A12)

I = { at(β, B13), at(energy1, B13), at(energy2, C15) } G = { at(β, A12) }

■

Call Coordinated-Sapa to solve this CPP

A plan is returned to enter A12 by taking the fuel from D14

POST PROCESSING

■ This plan will become a replacement for both

TakeEnergy(B13, energy1) and Move(B13, A12)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

C B A

47

Thanks for your attention!!!