Observation-Based Proactive Communication in Multi-Agent Teamwork

Volume8,Number1,pp.6377. http://www.spe.org ©2007SWPS

OBSERVATION-BASEDPROACTIVE COMMUNICATIONIN MULTI-AGENT

TEAMWORK

YU ZHANG

∗

Abstrat. Multi-agentteamworkisgoverned bythesamepriniplesthatunderliehumanooperation. Thispaper desribes

howto giveagents thesameooperativeapabilities,observabilityand proativity,that humansuse. Weshowhowagents an

useobservationoftheenvironmentandofteammates'ationstoestimatethe teammates'beliefswithoutgenerating unneessary

messages; we also show how agents an antiipate informationneeds among the teammembers and proatively ommuniate

the information,reduing thetotal volumeof ommuniation. Finally,wepresentseveral experimentsthatvalidate the system

developed,exploretheeetivenessofdierentaspetsofobservabilityandintroduethesalabilityoftheuseofobservabilitywith

respettothenumberofagentsinasystem.

Keywords. Multi-agentsystems,teamwork,agentommuniation,observability

1. Introdution. Reently, the fous of muh researh onmulti-agent systems(MAS) has shiftedfrom

strong ageny [26℄ to teamwork, whih is a ooperative eort by a team of agents to ahieve a ommon or

sharedgoal[23℄. Researh onmulti-agentteamworkbuilds onndingsabouteetivehumanteam behaviors

and inorporates them into intelligent agent tehnologies. Forexample, theshared mental model, oneof the

majoraspetsofthepsyhologialunderpinningsofteamwork,hasbeenadoptedwidelyasaoneptualbasisof

multi-agentteamwork.Basedonthesharedmentalmodel,aneetiveteamoftenanantiipatetheinformation

needs of teammatesand oerpertinent information proatively[18, 22℄. Consequently, supporting proative

information exhange among agentsin amulti-agent teamwork setting is ruial [29℄. Substantialhallenges

arise in a dynami environment beause agentsneed to deal with hanges. Although partial observability of

dynami,multi-agentenvironmentshasgainedmuhattention[17,11℄,littleworkhasbeendonetoaddresshow

toproesswhat isobservableand underwhihonditions;howanagent'sobservabilityaetstheindividual's

mental stateand wholeteam performane;and howagentsanommuniate proativelywitheahotherin a

partiallyobservableenvironment.

In this paper, we fous on how to inlude represent observability in the desription of a plan, and how

to inlude it into the basi reasoning for proative ommuniation. We dene several dierent aspets of

observability(e.g., seeingaproperty,seeing anotheragentperforman ation,andbelieving anotheranseea

propertyorationarealldierent),andproposeanapproahtotheexpliittreatmentofanagent'sobservability

that aimsto ahievemoreeetiveinformationexhange among agents. We employ theagent'sobservability

asthemajormeansforindividual agentstoreasonabouttheenvironmentand otherteammembers. Wedeal

withommuniationwiththe`right'agentaboutthe`right'thingat the`proper'timeinthefollowingways:

•

Reasoning about what information eah agent on a team will produe, and thus, what information

eah agentan oerothers. This is ahievedthrough: 1) analysis of the eets of individual ations

inthespeiedteam plans;2)analysisofobservabilityspeiation,indiatingwhatandunder whih

onditionseahagentan pereiveabouttheenvironmentaswellastheotheragents.

•

Reasoningaboutwhat informationeahagentwillneedintheproessofplan exeution. Thisisdone

throughtheanalysisofthepreonditionsoftheindividualationsinvolvedin theteam plans.

•

Reasoning aboutwhether an agent needs to at proativelywhen produing some information. The

deisionis made in termsof: 1) whether ornot theinformation is mutableaording to information

lassiation;2)whihagent(s)needsthisinformation;and3)whether ornotanagentwhoneedsthis

informationisabletoobtaintheinformationindependentlyaordingtotheobservationofenvironment

andotheragents'behaviors.

Wealso presentseveral experimentsthat validate thesystem developed, explore theeetiveness of dierent

aspetsofobservabilityand introduethesalabilityoftheuseofobservabilitywith respetto thenumberof

agentsinasystem.

Therestofthispaperisorganizedasfollows. Setion2reviewsrelatedwork.Setion3isanoverviewofthe

systemarhiteture, whih isalled CAST-O. Setion 4disusseshowanagent'sobservabilityis represented,

andhowanagent'sbeliefsaremaintainedin theourseofobservations. Setion5desribesobservation-based

∗

proativeommuniationamongagents. Setion6isanempirialstudybasedonamulti-agentWumpusWorld.

Setion7summarizesourworkanddisussesissuesforfurtherresearh.

2. Related Work. Asingleagent'sobservabilityand reasoninghavereeivedresearhers'attentionsfor

sometime. Pereptionreasoningisoneoftheseresearh diretions[16,24℄. Forexample,seeingisbelieving

hasbeenadopted forpereption-basedbelief reason[2,13℄. Inreentyears,observabilityhasbeenusedwidely

tounderstandbehaviorsof multi-agentsystems. Onestudy ofpartiular interestisalogiforvisibility,seeing

and knowledge (VSK), whih explores relationships between what is true, visible, pereived, and known; it

also investigates a number of interation axioms among agents, suh as under whih ondition agent a sees

everythingagentb sees or agentb knows everythingagentasees [27℄. However,VSK logi doesnotaddress

twomajor issuesregarding agentooperation: 1) anagent usesthe eets of ations in reasoning what

oth-ers are likely to know, but VSK does not provide a wayto treat ations through observation; 2) VSK does

not provide agentswith an eetive wayto utilize their observation to manage ommuniation. Isozaki and

Katsuno propose an algorithm to reasonaboutagents'nested beliefs (whih are one'sbelief aboutthe belief

ofanother),basedonobservatio[10℄. However,theydonotrepresenttheproess ofobservation, suh aswhat

anbe seenand under whih onditions. Tambeand Kaminkause observation to monitorfailed soial

rela-tionships betweenagents [12℄, but they do not give details abouthow agents' belief about their teammates'

mental statesare updated. Viroli andOmiini deviseaformal framework for observation that abstrats

on-ditions that ause agents' interative behavi [25℄. But, they don't say muh about how the observation to

environment is proessed. All of above fall into the ategory passive observation, in the sense that eah

agentevaluates observabilityonditions at theappropriate times. Ourwork also belongs to passive

observa-tion. However, we aim to redue the amount of ommuniation by reasoning aboutagent observability, the

apability to observe environment and ations. We relate an agent's observability to its mental state, and

then use observation and belief aboutothers' observabilities to estimate its teammates'mental states. That

is,anagentanexploit knowledgeaboutwhat itanditsteammatesanseetohelp deidewhenothersmight

ormight not know some information. Ioerger hasonsidered ativeobservation, in whih he invokes

addi-tional`nd-out'planstoseekvaluesforunknownonditionsknowledgeofwhosevalueswouldenablesituation

assessment[9℄.

Todate, ontrol paradigmsforooperativeteamworkhaveallowedagentstoommuniateabouttheir

in-tentions,plans,andtherelationshipsbetweenthem[23,21℄. However,thisomplexteamooperationbehavior

requires high-frequenyommuniationand omputationtime, whih weakensteamworkeieny. Moreover,

someresearhershavefound thatommuniation,while ausefulparadigm,is expensiverelativeto loal

om-putation [1℄; therefore tehniques that redue extraneous ommuniation during teamwork proesses are of

partiularimportane. Ontheotherhand,thereexistseveralommuniation-lessagentooperationtehniques

suhassoialonventions[20℄, foal points[14℄, planreognition[8℄, deision-theoretimodeling [15,28℄, and

game-theoretireursivemodeling[5℄. Ingeneral,these tehniques emphasizeinferringothers' ations

impli-itly orexpliitly, based onestablished normsfor behaviororonknowledgeaboutthe preferenesorinterests

of others. However, strategiessuh assoialonventionsorfoal pointstotallyeliminate ommuniationand

use onvention rules to guide agents' ations, strategies suh as plan reognition or deision-theoreti

nor-mallyhavehighomputationalomplexityindealingwithunertaintywhihweakensteamworkeieny,and

game-theoretireursivemodelingisprimarilysuitablefortwo-memberteams. Ourapproahtoproative

om-muniation is dierent in that agents are apable of prediting team-related information (by analyzingteam

plans) and distributing suh information only when it is neessary. The ommuniation need is redued, by

usingbeliefofwhatagentsanobserve,andhenedon'thavetobetold.

3. The CAST-O Arhiteture. The CAST-O arhiteture is an extension of CAST (Collaborative

Agents for SimulatingTeamwork) [29℄. There are three aspets to the extension: 1) representationof agent

observabilityabouttheenvironmentsandotheragents'ations;2)belief-maintenaneintermsofobservation;

3)observation-basedproativeommuniationamongagents.

An agent team is omposed of a set of agents. The team members share the team knowledge that is

represented in MALLET (Multi-Agent Logi Languagefor Enoding Teamwork), whih provides desriptors

for enoding knowledge about teamwork proesses (i. e. individual/team plans and operations), as well as

speiationsof teamstrutures(e.g., teammembersandroles)[30℄. Eahagenthasanindividual knowledge

theproessof planexeution,individual agentsanobservetheenvironmentandtheirteammates' behaviors,

infertheteammates'mentalstates,ommuniatewitheahother,and performations.

Plansareattheenterofativity. Theydesribehowindividualsorteamsangoaboutahievingvarious

goals. Plansarelassiedintoindividual plansandteam plans. Eah individualplan hasaproess onsisting

ofaset ofoperations,eahof whihiseither aprimitiveoperator,oraompositeoperation(e.g.,asub-plan).

Teamplansaresimilarto individualplans,but theyallowmultipleagentsoragentvariablesto beassignedto

arryoutoperationsorplans (someoftherequiringateam). ADOstatementisusedtoassign oneorseveral

agentsto arryoutspei operatorsorsub-plans. Thefollowingisanexampleteam planforthemulti-agent

versionof WumpusWorld(refertosetion6formoredetails):

(tplan killwumpus()

(proess

(par

(seq

(agent-bind ?a (onstraint (play-role ?a arrier)))

(DO ?a (findwumpus ?w))) // arrier is assigned

(seq

(agent-bind ?fi (onstraint ((play-role ?fi fighter)

(losest-to-wumpus ?fi ?w))))

(DO ?fi (movetowumpus ?w)) // fighter who is losest to

// wumpus is assigned

(DO ?fi (shootwumpus ?w)))))) // shootwumpus is an operator

wherefindwumpusandmovewumpusareindividual plans,andshootwumpusisanindividual operatorspeied

asfollows:

Generally, operators are dened by their preonditions and eets, whih are logial onjuntions. An

individualationistheexeutionofaninstantiatedoperatorinaDOstatement. Itisrepresentedas:

<ation> ::= (DO <doer> (<operator-name> <args>))

where<doer>istheagentassignedto theationand<operator-name>and <args>are orrespondentto the

nameandargumentsoftheoperator. SampleindividualationsintheextendedWumpusWorldareasfollows:

(DO ?fi (shootwumpus ?w))

(DO ?a (pikupgold ?g))

We assume that the preondition of the ation must be believed by <doer> before the ation an be

performed andtheeet mustbe believed aftertheationis performed. Sineationsaredomain-dependent,

whenagentsperformtheations,theysendasignaltotheenvironmentsimulation. Thentheationsarevisible

toanyteam memberwhoseobservability(seesetion4)permitsitat thetimetheationsareperformed.

Anessentialfeaturethatdierentiatesanagentteamfromasetofindividualagentsisthatateamofagents

may perform a joint ation, whih is the unionof simultaneous individual ations performed by individuals

sharingertain spei mental properties [4℄. MALLET provides a desriptor joint-dofor agentsperforming

thejointation,and speies threedierentjoint types: AND,ORorXOR[29℄. Forexample,wemaydene

followingjointationin theextendedWumpusWorld:

(joint-do AND

(DO ?a (move ?x ?y))

(DO ?fi (move ?x ?y)))

whih meansagents?aand?fimovesimultaneously.

GivenateamplanexpressedinMALLET,weanexpliitlydedueinformationneedsandprodutionfrom

thepre-ondsandeetsofoperatorsandimpliitlydedueothersfromtheplanstruture,e.g.,joint-dorequires

oordinationregarding startingtime, oroperationsin parallelneed oordination in termsof thestartingand

endingoftheparset ofbranhes. Thelatter, forexample,mightbedeterminablefrom observations,avoiding

theneedforexpliitommuniation. Inaddition, ifmultipleagentsareapableofperformingthesametasks,

theMALLET teamplanislikelytoontainagentseletionriteria(e.g.,thelosestagenttoawumpusshould

killit). Again,thisfallsintherealmofimpliitlydeterminableoordinationommuniation. Whilethispaper

hasfousedontheonlytheexpliitlydeterminablepartofthis(i.e.,thingsderivedfrompre-ondsandeets

onditions),thebasistruture oftheuseofobservationanbeappliedto moregeneralsituations.

<property> ::= (<property-name> <objet> <args>)

<objet> ::= <agent>|<non-agent>

where<objet>ouldbeeitheragentornon-agent,and<args>isalistofargumentsdesribingtheproperty.

SamplepropertiesintheextendedWumpus Worldareasfollows:

(loation fi ?x ?y),

(dead w1 ?state).

Theusefulnessof propertiesderivesfromtreatingthem asqueriesto theenvironment,using variablesfor

any orall ofthearguments. Uniationwill provide values,ifany, forthefreevariables that makethequery

true;ifthere arenosuhvalues,thenthevalueforthequerywillbefalse.

Duringateamworkproess,theenvironmentsimulationprovidesaninterfaethroughwhihtheagentsan

observetheenvironmentandtheirteammates'ations. Theenvironmentevolvesfromthestateatonetimeto

thestateatthenexttimewithanationpossiblybeingtakenduringthetimeinterval,savingonlytheurrent

environmentstates. EahagentmaintainsknowledgeoftheenvironmentinitsKB,updatingthisknowledgeas

neededtoarryoutitsplanor provideinformationtoteammembers.

4. Agent Observability. Toexpressagentobservability,wedeneaqueryfuntionCanSee(<observer>

<observable> <ond>), where <observer> speies the agent doing the observing, <observable>

identi-es what is to be observed, and <ond> speies the onditions under whih the <observer> an see the

<observable>. Whenneeded,the queryissubmittedto theknowledgebase forevaluation afterrst forming

theonjuntionof thearguments. As<observablea>and<ond>maybeprediates,missing valuesfor

vari-ableswillbesuppliedviauniationifthere areanysuhvaluesthat allowthe <ond>tobesatised,orelse

returnFALSE. This allowsanagent, forexample, todeterminethe loation(through variables)of atargetif

theonditionsaresatised(e.g.,thetargetiswithinrange). Timeisimpliitinthisqueryandistakentobethe

timeoftheurrentstep. Notethatstrongonstraintsweakenagents'observability;weakonstraintsstrengthen

agentsobservability. The strongest onstraint is FALSE, whih means that the agent ansee nothing. The

weakestonstraintis TRUE,whihmeansthattheagentanseeeverything.

Suessful teamwork requires interdependeny among theagents[6℄. This suggeststhat anagentshould

know at least some things about what other team members an see. However, an agent may not know for

sure that another agentan see something. Rather, an agentmay only believe, basedon its urrent beliefs,

that anotheragentansee something. WethenuseBelieveCanSee(<believer> <observer> <observable>

<ond>)tomeanthatoneagentbelievesanotheragentanseesomethingunder ertainondition.

Wealsomaketheassumptionofseeing isbelieving. Whilephilosophersmayentertaindoubts beauseof

thepossibilityof illusion,ommonsenseindiates that, otherthingsbeingequal,oneshouldbelievewhatone

sees[13,2℄. Thus, weassumethatanagentbelievesanobservedpropertypersistsuntilitbelievestheproperty

hasbeennegatedlater.

Inthefollowingsubsetions,wedesribethesyntaxandsemantisofobservabilityinmoredetail.

4.1. The Syntax of Observability. The syntax we use for observability is given in Table 4.1. For

example,theobservabilityspeiationforaarrierintheextended WumpusWorldisshownbelow,where a,

ra,,r representthearrier,arrier'sdetetionradius,ghter andghter's detetionradius,respetively.

(CanSee a (loation ?o ?x ?y)

(loation a ?x ?y) (loation ?o ?x ?y)

(inradius ?x ?y ?x ?y ra)

) // The arrier an see the loation property of an objet.

(CanSee a (DO ?fi (shootwumpus ?w))

(play-role fighter ?fi) (loation a ?x ?y) (loation ?fi ?x ?y)

(adjaent ?x ?y ?x ?y)

) // The arrier an see the shootwumpus ation of a fighter.

(BelieveCanSee a fi (loation ?o ?x ?y)

(loation fi ?xi ?yi) (loation ?o ?x ?y)

(inradius ?x ?y ?xi ?yi rfi)

Table4.1

TheSyntaxofObservability

1:

< observability >

:=

(

CanSee < viewing >

)

∗

2:

(

BelieveCanSee < believer >< viewing >

)

∗

3:

< viewing >

:=

< observer >< observable >< cond >

4:

< believer >

:=

< agent >

5:

< observer >

:=

< agent >

6:

< observable >

:=

< property >

|

< action >

7:

< property >

:=

(

< property

−

name >< object >< args >

)

8:

< action >

:=

(

DO < doer >

(

< operator

−

name >< args >

))

9:

< object >

:=

< agent >

|

< non

−

agent >

10:

< doer >

:=

< agent >

(BelieveCanSee a fi (DO ?f (shootwumpus ?w))

(play-role fighter ?f) ( ?f fi) (loation a ?x ?y)

(loation fi ?xi ?yi) (loation ?f ?x ?y)

(inradius ?xi ?yi ?x ?y ra) (inradius ?x ?y ?x ?y ra)

(adjaent ?x ?y ?xi ?yi)

) // The arrier believes the fighter is able to see the

// shootwumpus ation of another fighter.

Anagenthastwokindsofknowledge,sharedteamknowledge,enodedinMALLET,andindividual

knowl-edge, ontained in its knowledge base. Thesyntax of observabilityan be used either,as rulesin anagent's

knowledgebase[31℄,orasapabilityinorporatedintoMALLET.Inthispaper,weenodeobservabilityasrules

inagents'knowledgebases.

4.2. TheSemantisofObservability. Togiveoperationalsemantistoobservability,weneedtolarify

therelationshipsof: 1) what an agent an see, what it atuallysees, and what itbelievesfrom its seeing; 2)

whatanagentbelievesanotheragentansee,whatitbelievesanotheragentatuallysees,andwhatitbelieves

anotheragentbelievesfromitsseeing.

In order to properly disuss the semantis, we need to introdue a notionof time, as preonditions and

eetsrefertodierentpointsintime. Forpurposesofexposition,wewillsimplyassumethattimeisadisrete

and indexed in order by the natural numbers, and use the indies to referene pointsin time. Sine we are

dealingwithmultiple agents,multipleationsmayouratthesametimeinstant. Wedonottrytoelaborate

furtherontime inthis paper,asthereareanumberof usefuldierentwaysof dealingwithissuessuhasthe

synhronizationamongteam membersperformingations,andtheyarenotentral tothepointof thepaper.

Let

See

t

(a,

ψ

)expressthatagentaobserves

ψ

attimet. There aretwoasesto onsider,rstwhere

ψ

is

aproperty,and seondly, where

ψ

is anation. When

ψ

is aproperty,seeing

ψ

meansdeterminingthe truth

valueof

ψ

,withuniationofanyfreevariablesin

ψ

. If

ψ

isanation,seeing

ψ

meansthat theagentbelieves

thedoerbelievedthepreonditionof

ψ

immediatelybeforetheationourredandthedoerbelievestheeet

of

ψ

immediatelyafterperformingtheation. Weusethemeta-prediate

Hold

t

()tomeanholdsintheworld

(environmentsimulation)attimet. Wemaketheassumptionbelow:

∀

a, ψ, c, t, CanSee

(

a, ψ, c

)

∧

Hold

t

(

c

)

→

See

t

(

a, ψ

)

(4.1)

whihmeansthatiftheonditionholdsattimet andagentahastheapabilitytoobserve

ψ

underondition

,thenagenta atuallydoesdeterminethetruth-valueof

ψ

attimet.

Next,weonsider therelation betweenseeingsomethingand believing it. Belief isdenoted bythemodal

operatorBELandforitssemantisweadopt theaxiomsK,D,4,5inmodallogi. Theassumptionofseeing

isbelieving isagainstatedseparately forproperties andations. Intheaseof properties, itisformalizedin

theaxiombelow:

∀

a, ϕ, t, See

t

(

a, ϕ

)

→

[

Hold

t

(

ϕ

)

→

BEL

t

(

a, ϕ

)]

∧

[

¬

Hold

t

(

ϕ

)

→

BEL

t

(

a,

¬

ϕ

)]

(4.2)

whih saysthatforanyproperty

ϕ

seenbyagenta,if

ϕ

holds,agenta believes

ϕ

;if

ϕ

doesnothold, agenta

Agent a's beliefis moreomplexwhen anation,

φ

, is observed. Let

Doer

(

φ

)

,

P rec

(

φ

)

,

Ef f t

(

φ

)

denote

thedoer, thepreondition,and theeet of ation

φ

. Whenagenta seesation

φ

performedbysomeagent,

agenta believesthat theagentbelieved thepreonditionand believestheeet. This proessis expressedby

thefollowingaxiom:

∀

a, φ, t, See

t

(

a, φ

)

→

BEL

t

(

a, BEL

t

−

1

(

Doer

(

φ

)

, P rec

(

φ

)))

∧

BEL

t

(

a, BEL

t

(

Doer

(

φ

)

, Ef f

t

(

φ

)))

(4.3)

Fromthebeliefupdateperspetiveinoururrentimplementationwherebeliefsareassumedpersistent,forany

p

∈

P rec

(

φ

)

,agenta believesthat

Doer

(

φ

)

stillbelievesp attimet (i.e.

BEL

t

(

a, BEL

t

(

Doer

(

φ

)

, p

))))

unless

¬

p

isontainedin Et(

φ

). ThisissimilarforBelieveCanSee.

Anagent'sbelief aboutwhat anotheragentseesisbasedonthefollowingaxiom:

∀

a, b, ψ, c, t, t

′

_{, BelieveCanSee}

₍

_{a, b, ψ, c}

₎

_∧

_BEL

t

(

a, BEL

t

′

(

b, c

))

→

BEL

t

(

a, See

t

′

(

b, ψ

))

(4.4)

whihmeansthatifagentabelievesthatagentb isabletoobserve

ψ

underondition,andagentabelieves

attimet',thenagenta believesattimet that agentb saw(t'

<

t),sees(t'=t),orwillsee(t'

>

t,whih requires

somepreditionapabilityforagenta)

ψ

attimet'. Inourapproah,eahagentfousesonthereasoningabout

urrentobservability,notinthepastorinthefuture. Therefore,theaxiomaboveanbesimpliedasfollows:

∀

a, b, ψ, c, t, BelieveCanSee

(

a, b, ψ, c

)

∧

BEL

t

(

a, c

)

→

BEL

t

(

a, See

t

(

b, ψ

))

(4.5)

Notethatagenta evaluatesonditionaordingto itsownbeliefs.

Combiningthiswith thepreviousassumptionthat seeingisbelieving. weextendthistobelief. Wehave

twoseparateasesforpropertiesandations. Whenagentabelievesagentb seesaproperty

ϕ

,abelievesthat

b believes

ϕ

:

∀

a, b, ϕ, t, BEL

t

(

a, See

t

(

b, ϕ

))

→

BEL

t

(

a, BEL

t

(

b, ϕ

))

(4.6)

Whenagenta believesagent b seesan ation

φ

, a believesthatb believesthedoerbelievedthe preondition

attheprevioustimestepandbelievestheeetattheurrenttimestep. Thisonsequeneisexpressedbythe

following:

∀

a, b, φ, t, BEL

t

(

a, See

t

(

b, φ

))

→

BEL

t

(

a, BEL

t

(

b, BEL

t

−

1

(

Doer

(

φ

)

, P rec

(

φ

))))

∧

BEL

t

(

a, BEL

t

(

b, BEL

t

(

Doer

(

φ

)

, Ef f t

(

φ

))))

(4.7)

4.3. Belief Maintenane. Fromthesemantis,agents'observabilityisloselytiedtotheirbeliefsabout

theenvironmentandotheragents. Agentsmustupdatethesebeliefswhentheyperform,orreasonaboutothers',

observation.

4.3.1. Maintaining BeliefAboutSelf'sObservability. Theaxiomofseeingisbelievingbridgesthe

gapbetweenwhat anagentseesandwhat itbelieves. Anagentmaintainsitsbeliefsin twoaspets: 1) foran

observedproperty,theagentbelievestheproperty;2)foranobservedation, theagentbelievesthatthedoer

believedthepreonditionbeforetheationandthedoerbelievestheeetafter theation. Thealgorithmfor

updatingwhatanagenthasobserved,aordingto theobservabilityrules,isgivenin Figure4.1.

Thisalgorithmbuildsbeliefsinthebeliever's(i.e.,agentself's),knowledgebasebyhekingthefollowing:

Observingaproperty

•

Whenevaluatingobservability(CanSee self (<prop-name> <objet> <args>) <ond>),selfqueries

<ond>toenvironmentKB.Thequeryreturnsalistofsubstitutionsofvariables,ornullif <ond>are

notsatised. Whenthereturnedtupleisnotnull,ifthepropertyholdsintheenvironment,selfupdates

itsknowledgebasewithbelief (<prop-name> <objet> <args>)foreahvariablebindings,otherwise,

•

Observinganation

Intheaseof (CanSee self (<ation-name> Agd(

6

=

self) <args>) <ond>),thequery<ond>is

madewithrespetto environmentKBaswell. Iftheresultofqueryisnotnull,selfupdatesitsbeliefs

bythatself believesthat agentAgdknewthepreondition,and thatAgd inferstheeet. Tohandle

thetemporalissueorretly,selfupdatesAgd'sbeliefaboutthepreonditionrstandthenAgd'sbelief

abouttheeet. Thesebeliefsareusefulinommuniation. Forexample,ifagentaneedsinformation

I andbelievesagentb believesI,a mayask b forI.

updateSelfObs(self,KBself)

/*Letselfbetheagentinvokingthealgorithms. Wedenotetheknowledgebase

foragenta by

KB

a

,fortheenvironmentby

KB

env

.*/

1: foreahrulein

KB

self

oftheform(CanSeeself(propobjetargs)ond) 2:ifondistruein

KB

env

forsomebindingsofvariables

3: if(propobjetargs)istruein

KB

env

forsomebindingsofvariables

4: update(

KB

self

,(propobjetargs))

5: for eahsuhbindingofvaluestothevariables;

6: else

7: update(

KB

self

,(not(propobjetargs)))

foreahsuhbindingofvaluestothevariables;

8: foreahrulein

KB

self

oftheform(CanSeeself(ationdoerargs)ond), ifondistruein

KB

env

forsomebindingofvariables,

9: foreahonjuntofpreonditionofation

10: update(

KB

self

,(BELdoeronjunt));

11: foreahonjuntofeetofation

12: update(

KB

self

,(BELdoeronjunt));

Fig.4.1. AnAlgorithmofMaintainingSelfsBeliefbyDiretObservation

4.4. MaintainingBeliefAboutOthers'Observabilities. Figure4.2showsanalgorithmforupdating

whatanagentandetermineaboutwhatotheragentsansee.

updateSelfBel(self,

KB

self

)

1: foreahruleoftheform(BelieveCanSeeself Ag(propobjetargs)ond)that

2: ondistruein

KB

self

forsomebindingofargumentstoagents

Ag

6

=

self

3: foreahsuhbindingofargumentstothevariables

4: update(

KB

self

,(BELAg(propobjetargs)));

5: foreahruleoftheform(BelieveCanSeeself Ag(ationdoerargs)ond)that

6: ondistruein

KB

self

forsomebindingofargumentstoagents

Ag

6

=

self

7: foreahonjuntofthepreonditionofation

8: update(

KB

self

,(BELAg(BELdoeronjunt)));

9: foreahonjuntoftheeetofation

10: update(

KB

self

,(BELAg(BELdoeronjunt)));

Fig.4.2. Analgorithmofmaintainingbeliefaboutothersobservabilities

Thealgorithmreordswhihagentsareknowntobeabletoseewhat,andupdateswhatanagentbelieves,

aording to the preondition and eet of the ations it observes other agents performing. For the agent

to determinewhether a piee ofinformation is needed by others, it simulates theinferene proess ofothers'

observabilitytodeterminewhihisknownbyothers.

•

Observingaproperty

Intheaseof(BelieveCanSee self Ag(

6

=

self) <property> <ond>),aquery<ond>ismadewith

respetto

KB

self

. Iftheonditionissatised,selfbelievesAganseetheproperty. However,selfmay

•

ObservinganationIntheaseof(BelieveCanSee self Ag(

6

=

self) (<ation-name> doer(

6

=

self)

<args>) <ond>),<ond>isevaluatedwithrespetto

KB

self

. Selfaddstuplesto

KB

self

,indiating

thatAgbelievesthatdoerbelievedthepreonditionsoftheation,andbelievestheeetsoftheation 1

.

4.5. Exeution Model. Ateahtimestep,everyagent,denoted byself, hasafuntionyle: (possibly)

observe,reeiveinformationfromothers,beliefoherene,(possibly)sendinformationtoothers,andat. Ifself

needsaninformationitemorproduesanitemneededbyothers,itwillobservetheworldandotheragents. It

then heksmessagesandadjusts itsbeliefs forwhat itsees andwhat itis told. Selfkeepstrakof theother

agents'mental statesbyreasoningaboutwhattheyseefrom observation,inordertodeidewhentoassistthe

otherswiththeneededinformationproatively. Finally,selfatsooperativelywithteammatesandentersthe

nexttimestep.

An algorithm for overall belief maintenane along with the funtion yle is shown in Figure 4.3. The

algorithm begins with updateWorld by self's last ation. We will not elaborate on how updateWorld works

whih isbeyond the fous of this paper. Basially,the environment simulationupdates the environment KB

afterreeivinganyationfromtheagent. Beausetheagentaninfertheeetofitsownation,thealgorithm

savesthe eet asa newbelief. UpdateSelfObsevaluates observabilityrules with information obtainedfrom

KB

env

andupdates

KB

self

withtheresultsoftheobservation. UpdateSelfBelupdatesself'sbeliefsaboutwhat

others'beliefsbyobservingenvironmentand ations.

updateKB(self,ation,

KB

self

)

/*Thealgorithmisexeutedindependentlybyeahagent,denotedself below,

aftertheompletionofeahstepintheplaninwhihtheagentisinvolved.*/

1: updateWorld(ation,self);//notifytheenvironmenttoupdate

KB

env

2: foreahonjuntintheeetofation

3: update(

KB

self

,onjunt);

4: ifselfprodues/needsinformationI

5: updateSelfObs(self,

KB

self

);//update

KB

self

byobservability 6: updateSeBel(self,

KB

self

);//update

KB

self

bybeliefsabout

//othersobservabilities

7: foreahominginformationI

8: update(KBself,I);//update

KB

self

byommuniation

Fig.4.3.Anoverallbelief-maintenanealgorithm

Thefuntionupdatemanageshistoryandisresponsibleforohereneandpersisteneofbeliefinanagent's

KB. The agent's beliefs about the world are saved asprimitive prediates as theywere expressed originally

in the world. Suh beliefs are generated from three soures: (1) belief from observation, i. e., aproperty self

observes;(2)belieffrominferene,i.e.,onjuntsinferredfromtheeetoftheationselfperforms;(3)belief

from ommuniation, i. e., messagesother agents send to self by ommuniation. How does ommuniation

aettheagent'smentalstate? VanLinderet al. proposethattheommuniationanalsobetranslatedtoa

beliefsavedinthementalstateinthesamewayasobservationis[13℄. Inanysituationinwhihbeliefisrequired

frommultiplesoures,onitsmayarise,suhasselfsimultaneouslysees

¬

ψ

andhears

ψ

. Astrategyisneeded

thatpresribeshowtomaintaintheohereneoftheknowledgebase ofanagentintheaseofonitsamong

inominginformationfromdierentsoures. Castelfranhiproposesthat suhastrategyshould presribethat

more redible information should always be favored over less redible information [3℄. To dene a strategy

omplying withthis idea, we propose that eah soure isassoiatedwith aredit and the redit dereasesin

this order: sourefrom observation, soure from inferene, and soure from ommuniation. At ertain time

point,whenanagentgetsonitinformationfrom dierentsoures,italwaysbelieveswhat itsees.

Sine the numberof time steps ould be innite, an agent keeps only urrent beliefs in its mental state,

exeptthat themostreentoneiskept, evenifit isnotgeneratedurrently. That anagentdoesnotdiretly

observeorinfersomeprediatesfrom urrentobservationdoesnotmeanit doesnotbelievethem. The agent

hasmemoryofthemfrombefore. Memoryisusefulinproativeommuniation;thus,ifapieeofinformation

isinfrequentlyhanged,at thetimewhen agenta realizesthat agentb needs theinformation,even ifagenta

doesnothavetheinformation, agenta antellagentb theinformationin itsmemory.

1

5. Proative Communiation. The purpose of proative ommuniation is to redue ommuniation

overhead and to improvethe eieny orperformane of ateam. Inour approah, proativeommuniation

is based on two protools named proativeTell and ativeAsk. These protools are used by eah agent to

generateinter-agentommuniationswheninformationexhangeisdesirable. Proativeommuniationanswers

the following questions pertinent to agent proativity during teamwork. First, when doesan agent send the

informationtoitsteammatesifithasanewpieeofinformation(eitherfromperforminganationorobserving)?

A simplesolution ouldbe sending theinformation when requested. That is, the agent would only send the

information after it has reeived a request from another agent. Our approah is that the agentobserves its

teammates,andommitsto proativetelloneit realizesthat oneoftheteammatesneedstheinformationto

fulll itsrole anddoesnothaveitnow. Meanwhile,iftheagentneedssomeinformation,itdoesnotpassively

wait forsomeoneelsetotellit; itshouldask forthisinformationatively. Seond,what informationissentin

asessionofinformationexhange? Therearetwokindsofinformation thatanbeommuniated. Oneisthe

informationexpliitlyneeded byanagenttoomplete agivenplan, i. e.,onjunts inapreonditionof plans

or operatorsthat theagentis goingto perform. Theother istheinformation impliitlyneeded bytheagent.

Forexample,ifagenta needsprediatep andknowsp anbededuedfromprediateq,eveniftheproviding

agentdoesnotknowp,itstillan tellagenta aboutq oneithasq,beauseitknowsthatagenta andedue

pfrom q. This paper,however,dealsonlywithagentsommuniatinginformationthat isexpliitlyneeded.

TheproativeTellandativeAskprotoolsaredesignedbasedonfollowingthree typesofknowledge:

•

Information needers and providers. In order to nd a list of agents who might know or need some

information, we analyze the preonditions and eets of operators and plans and generate a list of

needersandalistofprovidersforeverypieeofinformation. Theprovidersareagentswhomightknow

suh information,andtheneeders areagentswhomightneedtoknowtheinformation.

•

Relativefrequenyofinformation needvs. prodution. Foranypiee ofinformation I,wedenetwo

funtions,

f

C

and

f

N

.

f

C

(I)returnsthefrequenywithwhihI hanges.

f

N

(I)returnsthefrequeny

withwhihI isusedbyagents. Welassifyinformationintotwotypes: stati 2

anddynami. If

f

C

(

I

)

≤

f

N

(

I

)

, I is onsidered stati information; if

f

C

(

I

)

> f

N

(

I

)

, I isonsidered dynami information. For

statiinformationweuseproativeTellbyproviders,andfordynami informationweuseativeAsked

byneeders 3

.

•

Beliefsgeneratedafterobservation. Agentstakeadvantageofthesebeliefstotrakotherteammembers'

mentalstatesandusebeliefs ofwhatan beobservedand inferredto reduethevolumeof

ommuni-ation. Forexample,ifaproviderbelievesthat aneeder seesorinfers informationI,theproviderwill

nottelltheneeder.

AnalgorithmfordeidingwhenandtowhomtoommuniateforativeAskandproativeTell 4

isshownin

Figure5.1.

Considering the intratability of general belief reasoning [7℄, our algorithm deals with beliefs nested no

morethan one-layer. This is suientfor our urrent study on proativebehaviorsof agents, whih fouses

on peer-to-peer proative ommuniation among agents. For ativeAsk, an agent requests the information

from other agents who may know it, having determined it from the information ow. The agent selets a

provideramong agentswhoknowI and ask forI. ForproativeTell, theagenttellsother agentswhoneed I.

Anagentalwaysassumesothersknownothinguntilitanobserveorreasonthattheydoknowarelevantitem.

Informationsensedandbeliefsaboutothers'sensingapabilitiesbeomethebasisforthisreasoning. First,the

agentdetermineswhatanotheragentneedsfrom theinformationows. Seond,theobservationrulesareused

todeterminewhetherornotoneagentknowsthat anotheragentansense theneededinformation.

6. EmpirialStudy. Whileonewouldthinkthatifonegivesanagentadditionalapabilities,its

perfor-manewouldimprove,andindeedthisturnsouttobeorret,thereareseveralotherinterestingaspetsofour

sheme to evaluate. Forexample, whenthere are several dierent apabilities, theinterestingquestion arises

of howmuh improvementeahapabilitygivesand whihapabilities are themost importantto addin

dif-ferentsituations. Moreover,whileitisobviousthatoneshould notseedereasingperformanefrominreasing

2

Here,statiinformationinludesnot onlytheinformationneverhanged,butalsothe informationinfrequentlyhanged but

frequentlyneeded.

3

Infuture work, we will address somestatistial methods to alulate frequenies and hene willbeable to provide more

omprehensiveproativeommuniationprotools.

4

Notethatthereisnoneedtosayanythinganoutprevioustimepoints,asthosewouldhavebeenhandledwhentheywererst

ativeAsk(self,I,

KB

self

,T)

/*LetTbethetimestepwhenthealgorithmisexeuted.

Independentlyexeutedbyeahagent(self) whenit

needsthevalueofinformationI.*/

1: andidateList=null;

2: if(Iisdynamiand(It)

∨

(

¬

It)isnottruein

KB

self

foranyt

≤

T)

3: ifthereexistsax

≥

0suhthat

4: ((BELAgIT-x)

∨

(BELAg

¬

IT-x))istruein

KB

self

5: letxsbethesmallestsuhvalueofx;

6: foreahagentAg

6

=

self

7: if((BELAgIT-xs)

∨

(BELAg

¬

IT-xs))istruein

KB

self

8: addAgtoandidateList;

9: randomlyseletAgfromandidateList;

10: askAgforI;

11: else

12: randomlyseletaprovider

13: asktheproviderforI;

proativeTell(

KB

self

,T)

/*Independentlyexeutedbyeahagent(self),afteritexeutesupdateKB.*/

14: foreahonjuntIforwhih(I,T)istruein

KB

self

andIisstati 15: foreahAgnneeders

16: if(BELAgnIT)isnottruein

KB

self

17: tellAgnI;

Fig.5.1. ProativeCommuniation Protools

apabilities, there are still interestingquestions of how muh performane inrease anbeobtained and how

oneaninorporate theapabilitiesintothesystemin aomputationallytratablemanner. And,onethere is

aninterest in how thesheme saleswith thenumber ofagentsinvolved. Our empirialstudy is intended to

addressthese questions.

Totest ourapproah, wehaveextendedtheWumpusWorldproblem [19℄into amulti-agentversion. The

world is 20 by 20 ells and has 20 wumpuses, 8 pits, and 20 piles of gold. The goals of the team, four

agents,one arrierand three ghters, are to kill wumpuses and get the gold. Thearrier is apable of

nd-ing wumpuses and piking up gold. The ghters are apable of shooting wumpuses. Everyagent ansense

a stenh (from adjaent wumpuses), a breeze (from adjaent pits), and glitter (from the same position) of

gold. When a piee of gold is piked up, both the glitter and the gold disappear from its loation. When

a wumpus is killed, agents an determine whether the wumpus is dead only by getting the message from

others, whokillwumpus orseeshooting wumpusation. The environmentsimulationmaintainsobjet

prop-erties and ations. Agents may also have additional sensing apabilities, dened by observability rules in

theirKBs.

There aretwoategories ofinformation neededby theteam: 1)an unknownonjunt that ispart ofthe

preonditionofaplanoranoperator(e.g.,wumpusloationandwumpusisdead);2)anunknownonjunt

that is part of aonstraint(e.g., ghter loation, for seletinga ghter losestto wumpus). The wumpus

loationandwumpusisdeadarestatiinformationandtheghterloationisdynamiinformation. Agents

use proativeTell to impart statiinformation theyjust learnedif theybelieveother agentswill need it. For

example,thearrierproativeTellstheghtersthewumpus'loation. AgentsuseativeAsktorequestdynami

informationiftheyneeditand believeother agentshaveit. Forexample,ghtersativeAskeah otherabout

theirloationsand whetherawumpusisdead.

Weusedtwoteams, TeamAandTeamB.Eahteam wasallowedtooperateaxednumberof150steps.

Exept for the observability rules, onditions of both teams were exatly the same. In the absene of any

targetinformation(wumpusorgold),allagentsreasonedabouttheenvironmenttodeterminetheirpriorityof

potentialmovements. If theywereawareofatargetloationrequiringationontheirpart(shootwumpus or

pikupgold),theymovedtowardthetarget. Inallases,theyavoidedunsafeloations.

Table6.1

TeamPerformaneandCommuniationFrequenyinSampleRun. T1:numberofwumpuses leftalive,T2: amountofgold

leftunfound,T3:totalnumberofavtiveAsksused,T4: totalnumberofproativeTellsused,T5:averagenumberofativeAsksper

wumpuskilled,T6: averagenumberofproativeTells perwumpus killed

T

1

T

2

T

3

T

4

T

5

T

6

T eamA

4.8 7.2 77.4 33.8 5.09 2.23

T eamB

15 14.6 67.6 28.8 13.6 5.9

belief generated from observability to the suesses of CAST-O asa whole. Finally, the third evaluates the

impatofobservabilityonhangingommuniationloadwithinreaseofteamsize.

Twoteamsaredened asfollows:

•

TeamA:Thearrieranobserveobjetswithinaradiusof5gridells,andeahghteranseeobjets

withinaradiusof3gridells.

•

TeamB:Noneoftheagentshaveanyseeingapabilitiesbeyondthebasiapabilitiesdesribedatthe

beginningofthesetion.

Weusemeasuresofperformane,whihreetthenumberofwumpuseskilled,theamountofommuniation

used andthe goldpiked up. Inorder to makeomparisonseasier, wehavehosento havedereasingvalues

indiateimprovingperformane,e.g.,smallernumbersofommuniationmessagesarebetter. Tomaintainthis

uniformity with someparameters of interest, we use the quantity not ahieved by the team rather than the

numberahieved,e.g.,thenumberofwumpusesleftaliveratherthanthenumberkilled. Theexperimentswere

performedon5randomlygeneratedworlds. TheresultsareshowninTable1.

Table1showsthat,asexpeted,TeamAkilledmorewumpusesandfoundmoregoldthanTeamB.From

other experiments we have learned that the further the agents an see, the more wumpuses they kill. It is

interestingthat theabsolutenumberofommuniationsishigherforTeamA withobservabilitiesthanthat of

TeamB,thus33.8vs. 28.8forproativeTelland77.4vs. 67.6forativeAsk.Thereasonfortheinreasednumber

ofproativeTellsisthat in TeamA, thearrier,whoisresponsible forndingwumpusesandproativeTelling

wumpuses'loationstoghters,hasfurthervisionthanthatofthearrierinTeamB.HenethearrierinTeam

Aansee morewumpuses. This feature leadstomoreproativeTellsfromthe arrierto theghters inTeam

A. ThenumberofproativeTellsan bereduedby thearrier'sbeliefs abouttheghters' observability, i. e.,

ifthearrierbelievestheghtersanseethewumpus'loation,itwillnotproativeTelltheghters. However,

sinethe ghters' detet rangeis smaller thanthat of thearrier, the redutionannot oset thenumber of

extraproativeTells. ThereasonfortheinreasednumberofativeAsksinTeamAisthat themorewumpuses

they nd, the more likely it beomes that messages are sent among ghters to deide who is losest to the

wumpuses. Sineghtersin TeamAmaynd wumpusesby themselves,they needto askother teammatesif

thewumpusisdead,todeidewhethertokillitornot. Althoughthenumberofthemessagesouldberedued

byfatorssuhasallowingtheghtertoseeotherghters'loationsandtoseeotherghterskillingawumpus,

theinreaseannotbetotallyosetbeauseoftheghters'shortvision. Hene,itmakesmoresensetoompare

theaveragenumberofmessagesperwumpuskilled. Intheseterms,theperformaneofTeamA,ismuhbetter

thanthatofTeamB,thus2.23vs. 5.9forproativeTelland5.09vs. 13.6forativeAsk. Hene,ouralgorithms

formanagingtheobservabilityofagentshavebeeneetive.

The results of this experiment produed a bit of a surprise. By introduing observabilities to agents,

the amount of ommuniation atually inreased slightly. This an be explained by the fat that beause

observability is a major means for an individual agent to obtain information about environment and team

members;themoreinformationobtainedbytheagent,themoremessageswere onveyedtohelp others. The

properwaytointerprettheresults,then, isto normalize thembythe performane oftheteam,whih inthis

aseistheaveragenumberofommuniationsperwumpuskilled,denotedbyACPWK,inthisexample. From

thisperspetive,theamountofommuniationwasredued,asexpeted,alsovalidatingourapproah.

6.1. EvaluatingDierentBeliefsGeneratedfromObservability. Theseondexperimenttestedthe

ontributionofdierentaspetsofobservabilitytothesuessfulredutionoftheommuniation. Theseaspets

arebelief aboutobservedproperty,beliefaboutthedoer'sbeliefaboutpreonditionsofobservedation,belief

Fig.6.1.AverageCommuniationPerKilledWumpusinDierentCombinations

ombiningthem. WeusedTeamA andTeamBin thisexperimentand keptallonditionsthesameasthose

oftherstexperiment. WeusedTeamB,asreferenetoevaluatetheeetivenessofdierentombinationsof

observabilitywithTeamA.Wenamedthistestombination0,sinethereisnoneofsuh fourbeliefsinvolved

in. ForTeamA,wetestedanother4ombinationsofthesebeliefstoshowtheeetivenessofeah,intermsof

ACPWK.Theseombinationsare:

•

Combination0: TeamB,whihinvolvesnoneofbeliefs.

•

Combination 1: In Team A, for eah agent, leaveo BelieveCanSee rulesand do notproess belief2

and belief3when maintainingbeliefs after observation. Thereforeeveryagent onlyhas belief1 about

theworld.

•

Combination 2: Keep everyonditionin ombination 1,exeptfor enablingthebelief2proess. This

ombinationtestshowbelief2improvesthesituation.

•

Combination 3: Enabling the belief3 proess in ombination 2. This ombination tests the eet of

belief3.

•

Combination 4: Add BelieveCanSee rules into ombination 3. This ombination tests the eet of

belief4as wellasshoweetiveness ofthebeliefsasawhole.

Eahombinationisrunintheverandomlygeneratedworlds. Theaverageresultsoftheserunsarepresented

inFigure 6.1,inwhihonebarshowsACPWKforoneombination.

Firstofallthat,agents'belief1(ombination1)isamajorontributortoeetiveommuniation,forboth

proativeTellandativeAsk. ForproativeTell,in(a),omparedtoombination0,ACPWKsigniantlydrops

from5.9to 3.52. For ativeAsk,in(b), ACPWKdropsfrom13.8to11.1.

Theseondase,belief2(ombination2)doesnotprodueanyfurtherredutionandheneisnoteetive

forproativeTell,but produesimprovementforativeAsk. ForproativeTell,whenaprovidersees anation,

thoughitbelievesthedoerknowsthepreonditionandeet oftheation,it doesnotknowthe preondition

andeetbyitself. Soforthisexamplebelief2anbeoflittlehelpinproativeTell. WhileforativeAsk,belief2

redues ACPWK from 11.1to 9.36, beausewith belief2, aneeder will knowwho hasa piee of information

expliitly. Thenitan ativeAskwithoutambiguity.

Third,forthesamereasonthatbelief2onlyworksforativeAsk,belief3(ombination3)ontributeslittle

toproativeTellbut furtherdereasesACPWKto7.97forativeAsk.

Fourth, belief4 (ombination 4) hasa majoreet on ommuniationsthat applies to both protools. It

furtherdropsACPWKto2.23forproativeTellandto5.39forativeAsk. Belief4ispartiularlyimportantfor

proativeTell. Forexample,ifthearrierbelievesthattheghtersseeawumpus'loation,itwillnottellthem.

Thisexperimentexaminedtheontributionofeahbeliefdeduedfromobservabilitytotheoverall

eetive-nessofommuniation. Theresultindiatesthree things. First,belief1andbelief4haveastrongeetonthe

eienyof bothproativeTelland ativeAsk. Therefore,CanSee/BelieveCanSeeaproperty,theobservability

Fig.6.2. TheComparisonofProativeTellwithDierentTeamSize

thatCanSeeanationmaybeappliedtoommuniationwhihinursmoreAskthanTell,suhasgoal-direted

ommuniation. Third,thesebeliefsworkbesttogether,beauseeahofthemprovidesadistintwayforagents

togetinformationfromtheenvironmentandotherteammembers. Furthermore,theyomplementeahother's

relativeweaknesses,sousingthemtogetherbetterservestheeetivenessoftheommuniationasawhole.

6.2. Evaluating the Eet of Observability Communiation Load with Inreased Team size.

Wedesignedthethirdexperimentto showhowommuniationloadsaleswithinreasedteam size. Basedon

theassumptionthatproativeTellbringsmoreommuniationintoplaythanativeAsk,wehooseto testthe

proativeTellprotool. AtiveAskisdiretedtoonlyoneprovideratertaintime,whiletheproativeTellgoes

to allneeders whodo nothavethe information. If thetest results are good for proativeTell, weanexpet

thattheyarevalidforativeAskaswell.

Weused thesame sensingapabilities forTeamsA and TeamB asin therst experiment. However, we

inreasedthenumberofteam membersby1,2and3,in twotests thatweran. Inthersttest, weinreased

thenumberof needers,(i. e. ghters) and kept thesamenumberof providers,(i. e. arriers). In theseond

test,wedidittheotherwayaround. Ineahtest,foreahinrementandeahteam,werantheverandomly

generatedworldsandusedtheaveragevalueofACPKWproduedineahworld.

Figure6.2showsthetrendofACPKWasafuntionofinreasingteamsize. In(a),TeamBhasanobvious

inreasein ACPKWwithinreasingtheteam size. However,TeamAkeepsthesameACPKW.Theausean

beattributedtotwofators: rst,theamountoftheinreasingproativeTellsishelddownbeauseifthearrier

believestheghtersanseewumpus,thearrierdoesnotperformproativeTell;seond,themoreghtersthere

are,themorewumpuseswillbekilled, whihenlargesthenumeratorofACPKW.

In(b), inreasingthe numberof providersbreaks the onstant trend in TeamA and showsan inreased

ACPWK. However,omparing this inreaseto that of Team B, it is amoderatenumber. InTeamB, every

provider inrement means almost double the number of proativeTells. The ommuniation load inreases

beause of dupliate proativeTells of the same information by dierent arriers. Forexample, eah arrier

always provides the wumpus' loation to ghters when observing a wumpus. The arriers lak an eetive

way to predit when a piee of information is produed and by whom, whih is one of our main onerns

of future work. This experiment shows that the team empowered with observability has a slowergrowth of

ACPWKwithinreaseofteamsize,whihmayindiatethatobservabilitywillimproveteamsalabilityinsome

sense.

7. Conlusion. In this paper, we have presented an approah to dealing with agent observability for

improvingperformaneandreduinginter-agentommuniation. EahCAST-Oagentisallowedtohavesome

observabilitytosee theenvironment,andto wathwhat othersaredoinginsideits detetionrange. Basedon

theobservation, theagentupdates its knowledge base andinfers what othersmayknowat the urrenttime.

Reasoningaboutwhat othersansee allowsagentsto deidewhetherto distributeinformation andtowhom.

evaluation in an extended Wumpus World, omparing the relativenumbers of proativeTell, ativeAsk, and

wumpuseskilled foragentteamswithandwithoutobservability.

Amajorpointtotheproativeommuniationapproahwithobservabilitiesisthat theunderlyingsystem

thatinterpretstheteam plansoftheagentsdoesmostoftheworkforhandlingtheobservation,infereneand

ommuniation. Thisneedonlybedesignedone. Itisre-usedasonemovesfromonedomaintoanother. Itis

onlytheexpliationoftheobservabilityonditionsthathangesfromonedomaintoanother,andthisis

essen-tiallylinearlyproportionaltothenumberofagentsandsize ofthedomainpropertiesthataretobeobserved.

Thoughurrentlyweareonsideringjustthetimesofinformationprodutionorneed,thesameapproahan

beextendedtounertaintyin observabilityaswell. Additionally, ourpresentproativeinformation algorithm

analyzesthepre-onditionsand eetsof operatorsforwhiheahagentisresponsible intheteam plan. The

purposeof doingsois todeterminepotentiallyusefulinformation owamong agents. However,thisapproah

is restritive. Wewould like tomakethereognitionof neededinformation moredynami. One wayto solve

thisproblemistoreognizetheplansofotheragentsbyobservingationsoftheotheragents,andtrakingthe

sequeneofsub-goalsonwhih theyareworkingdynamially. Usingthisinformationtogetherwiththeation

an agent hasmost reentlyperformed, themost likelyinformation needs of other agents anbe dynamially

estimated overanite time horizon. Then weansend other agents onlyunknown information that will be

neededinthenearfuture.

Aknowledgement. ThisworkwassupportedinpartbyDoDMURIgrantF49620-00-I-326administered

throughAFOSR.

REFERENCES

[1℄ T. Balh and R. C. Arkin, Communiation in reativemulti-agent roboti systems,, Autonomous Robots, 1 (1994),

pp.2753.

[2℄ J.BellandZ.Huang,Seeingisbelieving,inProeedingsofCommonSense98,1998,pp.391327.

[3℄ C.Castelfranhi,Guaranteesforautonomyinognitiveagentarhiteture,IntelligentAgents,(1996),pp.5670.

[4℄ P. R. Cohen and H.J. Levesque, Teamwork, Nous,Speial Issue on CognitiveSiene and Artiial Intelligene, 25

(1991),pp.487512.

[5℄ P. J. Gmytrasiewiz, E. H. Durfee, and D. K. Wehe, A deision-theoreti approah to oordinating multi-agent

interations.,inProeedingsof12thInternationalJointConfereneonArtiialIntelligene,1991.

[6℄ B.J.Grosz,Collaboratingsystems,AIMagazine,17(1996).

[7℄ J.Y.HalpernandY.A.Moses,Aguidetoompletenessandomplexityformodallogisofknowledgeandbelief,Artiial

Intelligene,(1992),pp.319379.

[8℄ M.J.HuberandE.H.Durfee,Deidingwhentoommittoationduringobservation-basedoordination,inProeedings

ofthe1stInternationalConfereneonMulti-agentSystems,1995,pp.163170.

[9℄ T. R. Ioerger andL. He, Modeling ommand andontrol in multi-agent systems, in8th International Commandand

ControlResearhandTehnologySymposium(ICCRTS),June17-192003.

[10℄ H.IsozakiandH.Katsuno,Observability-based nested beliefomputationformulti-agent systemsanditsnormalization,

IntelligentAgentIV,(2000),p.LNAI1757.

[11℄ G. A. Kaminka, D. V. Pynadath, andM. Tambe,Monitoring deployed agent teams, inProeedingsof International

ConfereneonAutonomousAgents,2001.

[12℄ G. A. Kaminka and M. Tambe, Robust agentteams via soially-attentive monitoring, Journalof ArtiialIntelligene

Researh,12(2000),pp.105147.

[13℄ B.V. Linder,W. V. D.Hoek, andJ.Meyer,Seeingis believing—and sohearing andjumping, TopisinArtiial

Intelligene,LNAI992(1995),pp.402413.

[14℄ S.K.M. FensterandJ.S.Rosenshein,Coordinationwithoutommuniation: Experimentalvalidationoffoalpoint

tehniques,,inProeedingsofthe1stInternationalConfereneonMulti-agentSystems1995,pp.102108.

[15℄ M.G.M.GeneserethandJ.Rosenhein,Cooperationwithoutommuniation,,teh.rep.,StanfordHeuristi

Program-mingprojet,ComputerSieneDepartment,StanfordUniversity 1984.

[16℄ D.Musto andK. Konolige,Reasoning aboutpereption,inProeedingsofthe AAAISpring SymposiumonReasoning

AboutMentalStates,1993,pp.9095.

[17℄ D.PynadathandM.Tambe,Multiagentteamwork: Analyzingtheoptimalityandomplexityofkeytheoriesandmodels,

inProeedingsofthe1stAutonomousAgentsandMultiagentSystemConferene,2002.

[18℄ W.B.Rouse,J.A.Cannon-Bowers,andE.Salas,Theroleofmentalmodelsinteamperformaneinomplexsystems,

IEEETransationsonSystems,Man,Cybernetis,22(1992),pp.12961308.

[19℄ S.RussellandP.Norvig,ArtiialIntelligene: AModernApproah,NJ:PrentieHall,2002.

[20℄ Y.ShohamandM.Tennenholtz,Onthesynthesisofusefulsoiallawsforartiialagentssoieties(preliminaryreport),

inProeedingsofthe9thNationalConfereneonArtiialIntelligene,1992.

[21℄ F.StoneandM.Veloso,Taskdeomposition,dynamiroleassignment,andlow-bandwidthommuniationforreal-time

strategiteamwork,ArtiialIntelligene,110(1999),pp.241273.

[23℄ M.Tambe,Towardsexibleteamwork,JournalofArtiialIntelligeneResearh,7(1997),pp.83124.

[24℄ A. D. Val, P. M.-R. II, and Y. Shoham, Qualitative reasoning about pereption and belief, in Proeedings of 15th

InternationalJointConfereneonArtiialIntelligene,1997,pp.508513.

[25℄ M.ViroliandA.Omiini,Anobservationapproahtothesemantisofagentommuniationlanguages,AppliedArtiial

Intelligene,16(2002),pp.775793.

[26℄ M.Wooldridge andN. R. Jennings,Intelligentagents: Theory and pratie,The KnowledgeEngneering Review,10

(1995),pp.115152.

[27℄ M.WooldridgeandA.Lomusio,Multi-agent vsklogi,Proeedingsofthe17th EurapeanWorkshop onLogisinAI,

(2000).

[28℄ P.Xuan, V.Lesser, andS.Zilberstein,Communiationdeisionsinmulti-agentooperation: Modelandexperiments,

inProeedingsofthe5thinternationalonfereneonautonomousagents,2001,pp.616623.

[29℄ J.Yen,X.Fan,andR.A.Volz,Atheoretialframeworkonproativeinformationexhangeinagentteamwork,Artiial

IntelligeneJournal,169(2005),pp.2397.

[30℄ J.Yen,X.Fan,R.Wang,S.Sun,andR. A.Volz,Context-entrineedsantiipationusinginformationneedsgraphs,

JournalofAppliedIntelligene,24(2006),pp.7589.

[31℄ Y.ZhangandR. A.Volz,Modelingooperationbyobservation inagentteam,inProeedingsoftheIEEEInternational

ConfereneonSystems,ManandCybernetis(SMC'05),2005,pp.536541.

Editedby: MarinPaprzyki,NiranjanSuri

Reeived: Otober1,2006