Contents lists available atScienceDirect
Science
of
Computer
Programming
www.elsevier.com/locate/scico
Formal
verification
of
autonomous
vehicle
platooning
Maryam Kamali
a,
∗
,
Louise
A. Dennis
a,
Owen McAree
b,
Michael Fisher
a,
Sandor M. Veres
baDepartmentofComputerScience,UniversityofLiverpool,UK
bDepartmentofAutomaticControl&SystemsEngineering,UniversityofSheffield,UK
a
r
t
i
c
l
e
i
n
f
o
a
b
s
t
r
a
c
t
Articlehistory: Received6June2016
Receivedinrevisedform25May2017 Accepted30May2017
Availableonlinexxxx
Keywords: Vehicleplatooning Agentprogramming Modelchecking
The coordination ofmultiple autonomousvehiclesinto convoysorplatoonsis expected on our highways in the near future. However, before such platoons can be deployed, the behaviours of the vehicles in these platoons must be certified. This is non-trivial andgoesbeyondcurrentcertificationrequirements,forhuman-controlledvehicles,inthat these vehicles can act autonomously. In this paper, we show how formal verification can contribute tothe analysis of thesenew, and increasingly autonomous, systems. An appropriate overall representation for vehicle platooning is as a multi-agent system in which eachagent captures the “autonomous decisions” carried outby each vehicle. In ordertoensure thattheseautonomousdecision-makingagentsinvehicleplatoonsnever violatesafetyrequirements,weuseformalverification.However,astheformalverification techniqueusedtoverifytheindividualagent’scodedoesnotscaletothefullsystem,and as theglobalsystemverification techniquedoesnot capturethe essentialverification of autonomousbehaviour,weuseacombinationofthetwoapproaches.Thismixedstrategy allowsustoverifysafetyrequirementsnotonlyofamodelofthesystem,butoftheactual agentcodeusedtoprogramtheautonomousvehicles.
©2017TheAuthors.PublishedbyElsevierB.V.Thisisanopenaccessarticleunderthe CCBYlicense(http://creativecommons.org/licenses/by/4.0/).
1. Introduction
While “driverless cars” regularly appear in the media, they are neither “driverless” nor fully autonomous.Legal con-straints,suchastheViennaConvention[37],ensurethattheremustalwaysbearesponsiblehumaninthevehicle.Although fullyautonomousroadvehiclesremainfuturistic,theautomotiveindustryisworkingonwhatarevariouslycalled
road
trains,car convoys,or
vehicle platoons
.Here,eachvehicleautonomouslyfollowstheoneinfrontofit,withtheleadvehicleinthe platoon/convoy/trainbeingcontrolled bya humandriver.Thistechnology isbeingintroducedby theautomotive industry in orderto improveboth thesafety andefficiencyof vehicles onvery congested roads [29]. It isespecially usefulifthe vehiclesaretrucks/lorriesandiftheroadisamulti-lanehighway.Intheseplatoons,each vehicleclearly needs tocommunicatewithothers,atleastwiththeonesimmediatelyinfront andimmediatelybehind. Vehicle-to-vehicle(V2V) communicationis usedata lower (continuouscontrol system) levelto
adjust each vehicle’s position in the lanes and the spacing between the vehicles. V2V is also used athigher levels, for
exampletocommunicatejoiningrequests,leavingrequests,orcommandsdissolvingtheplatoon.Soatraditionalapproach istoimplementthesoftwareforeachvehicleintermsofhybrid(andhierarchical)controlsystemsandtoanalysethisusing hybridsystemstechniques[4].
*
Correspondingauthor.E-mailaddress:maryam.kamali@liverpool.ac.uk(M. Kamali).
http://dx.doi.org/10.1016/j.scico.2017.05.006
However, astheseautomotiveplatoonsbecomemorecomplex,thereisamovetowardsmuchgreaterautonomywithin eachvehicle.Althoughthehumaninthevehicleisstillresponsible,theautonomouscontroldealswithmuchofthecomplex negotiation toallowother vehicles toleave andjoin,etc. Safetycertificationis an inevitableconcern inthedevelopment ofmoreautonomousroadvehicles,andverifyingthesafetyandreliabilityofautomotive platooningiscurrentlyoneofthe main challengesfacedby theautomotive industry.Traditionalapproachestomodellingsuch situationsinvolvehybrid au-tomata[18]inwhichthecontinuousaspectsareencapsulatedwithindiscretestates,whilediscretebehavioursareexpressed astransitionsbetweenthesestates.Adrawbackofthehybridautomatonapproachisthatitcanbedifficulttoseparatethe two(high-leveldecision-makingandcontinuouscontrol)concerns.Inaddition,therepresentationofautonomous decision-makingcanbecomeunnecessarilyopaqueinsuchhybridapproaches.
As isincreasingly commonwithin autonomoussystems, we use a hybrid architecture wherenot only isthe discrete decision-making componentseparatedfrom thecontinuous control system, but thebehaviour of thediscrete part is de-scribed inmuch moredetail; inparticular,using the
agent
paradigm [40].Thisstyleof architectureimproves thesystem design froman engineeringperspective andalsofacilitatessystemanalysisandverification[9].Indeed,weusethis archi-tectureforactuallyimplementingautomotiveplatoons,andwewillhereshow howformalverificationcanbe usedforits directanalysis.Theverificationofsuchsystemsischallengingduetotheircomplexandhybridnature.Separatingdiscreteand continu-ousconcerns,asabove,potentiallyallowsustoreasonaboutthedecision-makingcomponentsinisolationandensurethat nodecision-makingcomponenteverdeliberately choosesanunsafe state.However,theuseofthe‘agent’ conceptaloneis not enoughforourpurposes,sincethiscanstillmakeitsautonomousdecisionsinan‘opaque’way.Inordertobeableto reasonabout,andformallyverify,thechoicesthesystemmakes,weusea
rational
agent[41].Thisnotonlymakesdecisions, but hasexplicit representations ofthe reasonsfor making them,allowing us to describe not onlywhat the autonomous systemchoosestodo,butwhy
itmakesitsparticularchoices[16].Weutilisethe
Belief-Desire-Intention
(BDI)model,oneofthemostwidelyusedconceptualmodels,notonlyfordescribing these rational agents but for actually implementing them [32]. A BDI-style agent is characterised by its beliefs, desires and intentions: beliefs representthe agent’sviews aboutthe world; desires representthe objectives to be accomplished; while intentionsare thesetoftasks currentlyundertaken bythe agentto achieve itsdesires. A BDI-styleagent hasa set of plans, determining how an agent acts based on its beliefs and goals, andan eventqueue where events(perceptions from the environment and internal subgoals) are stored. There are several advantages in using this style of model for developingautonomoussystems:(a)itnaturallyseparatesfeedbackcontrollersfromhigh-leveldecisionmaking,asabove; (b)itfacilitatesreasoningandverifyingaboutthebehaviourofhigh-leveldecisionmaking[11];(c)itsupportsincremental andhierarchical developmentof plans;and(d)itprovides a clearseparationbetweenplan selectionandplan execution. However, a drawback ofthisformof approachis that itis essentiallya plan management andplanselection framework withnoin-built mechanismsforlearningorfirst-principles planning.Thismeans thataBDI agentdoesnot automatically learnfrompastbehaviourandadaptitsplansaccordingly.Asimilarlimitationislackofpredictabilityandforwardplanning, in itsbasic form. Asthe aimofthispaperis onverification
ofdecisionsconcerning platooning,we canutilise thismodel intheformoftheGwendolenprogramminglanguage[8],developedforverifiableBDI-styleprogramming, tocaptureand implementtheagent-baseddecision-makingineachvehiclewithinanautomotiveplatoon.Aspartofsafetyverification,weneedtoverifytheagent’sdecisions,especiallyincombinationwiththeothervehicles. Anautonomousrationalagentmakesdecisionsaboutwhatactionstoperform,etc., basedonthe
beliefs
,goals
andintentions
thattheagentholdsatthattime.Weuseamodel-checkingapproachtodemonstratethattherationalagentalwaysbehaves in line with the platoon requirements and never deliberately chooses options that end up in unsafe states. We verify propertiesoftherationalagentcodeusingtheAJPFmodel-checker[12],one oftheveryfew model-checkersabletocope with complex properties of BDI agents. Unfortunately, there are two drawbacks to using AJPF: currently, AJPF doesnot support verification of timed behaviours; and AJPF is resource heavy and cannot be used to verify the whole system. Consequently,we hereproposeacombinedmethodologyfortheverificationofautomotive platooning.Toevaluatetiming behaviour,weuseatimed-automataabstractionandverifythesystemusingUppaal[2];toevaluateindividualautonomous decisions,weuseAJPFtogether withan abstractionoftheothervehicles/agents.Furthermore,we describehowthesetwo approaches tomodelling canbe combined toprovide an appropriate basis forverifying thebehaviour ofboth individual agentsandthewholesystem.
Overviewofourcontribution.TheISO26262standardprovidesastandardfor
functional safety
managementinautomotive applications, and determines the safety requirements that should be fulfilled in design, development and validation of individual automotive units. Following the ISO 26262 guidelines for safety management in automotive applications we proposeaverifiableagent-basedarchitecturefordevelopmentofsafeautomotiveplatooningand,inthesameresearchline, we proposenewcombinedverificationtechniquesforautonomoussystemsdevelopedbasedonhybridagentarchitectures. Inparticular,weshowtheapplicabilityofourverificationtechniquesinthedevelopmentofplatooning.Foraclearpictureofourapproach,wesummarise ourfourfoldcontribution.
II Wedeveloppracticalplatooningprotocolsusingahybridagentarchitecturewherethehigh-leveldecision-making com-ponentoftheplatooningprotocolsisprovidedasGwendolenagentcodeandthecontinuouscontrolsystemisprovided as a Simulink model. This contributes to simulation and testing of protocols for validation. The Simulink based control systemwas developedundertheassumptionthat thephysicalmodelinthesimulationisrepresentativeofrealworld vehicledynamics.Exactcorrectnessisnotarequirement,asthecontrollerhasbeenshowntoberobusttounmodelled dynamics[36,27].
III We propose new combined verification techniques for the safety analysis of systems which are developed using such hybridagentarchitectures.Thisaimstoseparatedifferentaspectsofasystemandthesystemproperties,i.e.,BDI and real-timerequirementsinordertoempowertheverificationofcomplexautonomoussystems.
IV Weshowhowtheproposedtechniquesforverificationcanbeappliedto
automotive platoon verification
.Ourdescription ofthis includesa brief overviewofthe various typesof models we use,inparticular BDI agent models andUppaal models,thatareusedforverificationofthevarioussystemproperties.NotethattheverifiedBDIagentcodeistheactual code whichhas been usedfor simulation1 andthe realworld testing.2 Thisverification hasbeen carriedout under the assumption that the vehicle control and environment interactions are correct.This assumption is not restrictive becausethevehiclecontrolandenvironmentinteractionsabstractlyrepresentourSimulinkbasedcontrolsystemand, toensurethecorrectfunctionalityofthesystemcontrol,anengineeringanalysishasbeenused[36,27].Thevalidityof ourabstraction ischeckedby thesimulation andtesting,which isdiscussedin II andanyerrors foundthere leadto improvedabstractions.Necessarily,realsystemscannotbeformallyverified.Someabstractionsmust
beused,thoughit isimportanttocontinuallyimprovetheseabstractions.Overall,theapproachdescribedherecanbeusedtoformallyverifypracticalautonomousplatooningsystems.
Theremainder ofthepaperisorganisedasfollows.InSection 2theautomotiveplatoonandplatoonrequirementsare presented.InSection3thehybridagentarchitectureandtheagent-baseddecision-makingfortheplatoonaredescribed.In Section4theanalysisandverificationoftheplatoonisconsidered.Finally,inSection 5,concludingremarksareprovided.
2. Automotiveplatoons
An automotiveplatoon, enablingroad vehiclesto travel asagroup, isled bya vehicledriven by aprofessional driver
[35,39,34].Thefollowingvehicles,i.e., membersoftheplatoon,arecontrolledautonomously.Thesevehicles,equippedwith low-level longitudinal (controllingspeed)and lateral(controlling steering)control systems, travelin a platoon with pre-definedgapsbetweenthem.Inaddition,V2Vcommunicationalsoconnectsthevehicles.Theleadvehicleeffectivelycarries
outcoordination overtheplatoon:settingparameters, creatingcertificates forjoiningandleaving,etc. Eachindividual ve-hicleobservesitsenvironmentandfollowsincomingcommandsfromtheleadvehicle.Inwhatfollows,weoutlinetheset ofhigh-level automotiveplatoonconcepts andproceduresincludinghow tojoinandleave aplatoon [3,29]inSection 2.1
andSection2.2,respectively.Inaddition,thehigh-levelrequirementsontheseproceduresforthedevelopmentofsafeand reliableplatooningareexplained.Fromthesewederivetheformalpropertiestobeverified.
2.1. Joining the platoon
Avehiclecanjoinaplatooneitherattheendorinthemiddlewithdifferentcontrolstrategiesbeingused.Thejoining procedureisasfollows:
•
anon-membervehiclesendsajoiningrequesttotheplatoonleader,expressingtheintendedpositionintheplatoon;•
ifthevehiclehasrequestedtojoinfromtherear,theleadersendsbackanagreementprovidedthemaximumplatoon length has not been reached and the platoon is currently in normal operation, i.e., no other active joining/leaving proceduresarehappening;•
ifthevehiclerequeststojoininfrontofvehicleXandthemaximumplatoonlengthhasnotbeenreached,theleader sends an “increase space” command to vehicle X, andwhen the leader is informed that enough spacing has been created,itsendsbackanagreementtothejoiningvehicle;•
upon receipt of an agreement, the joining vehicle changes its lane (changing lane is a manual procedure which is performedbyahumandriver);•
oncethevehicleisinthecorrectlane,itsautomaticspeedcontrollerisenabledanditapproachestheprecedingvehicle;•
whenthevehicleiscloseenoughtotheprecedingvehicle,itsautomaticsteeringcontrollerisenabledanditsendsan acknowledgementtotheleader;and,finally,•
theleadersendsa“decreasespace”commandtovehicleX,andwhentheleaderisinformedthatspacinghasreturned tonormal,itrepliestotheacknowledgement.Fig. 1.Hybrid agent architecture.
In order to ensure safe joining operations, the following requirements should be preserved within the decision-making componentsofautomotiveplatoon.
1. Avehiclemustonlyinitiatejoiningaplatoon,i.e. changinglane,onceithasreceivedconfirmationfromtheleader. 2. Beforeautonomouscontrolisenabled,ajoiningvehiclemustapproachtheprecedingvehicle,inthecorrectlane. 3. Automaticsteeringcontrollermustonlybeenabledoncethejoiningvehicleissufficientlyclosetotheprecedingvehicle.
2.2. Leaving the platoon
Avehiclecanrequesttoleaveplatoonatanytime,i.e.,adriverinitiatesaleavingrequest.Theleavingprocedureis:
•
aplatoonmembersendsaleavingrequesttotheleaderandwaitsforauthorisation;•
uponreceiptof‘leave’authorisation,thevehicleincreasesitsspacefromtheprecedingvehicle;•
whenmaximumspacinghasbeenachieved,thevehicleswitchesbothitsspeedandsteeringcontrollerto‘manual’and changesitslane;and,finally•
thevehiclesendsanacknowledgementtotheleader.Note that the detailedcommunications withthe driver are abstracted away.It means that the drivershould be notified aboutthespeedandsteeringcontrollersthatareswitchedto‘manual’.AssuggestedintheEUlegislationCOMPANYON[29], a control andtell-taledevice wouldbeneededto letthe drivertake thevehiclecontrol backandto informthedriverif (s)heisin theplatoon.Thetwo followingrequirementsarealsonecessary propertiesforensuring safeleavingoperations thatshouldbemetwiththeagent-baseddecision-makingcomponentsofanautomotiveplatoon.
1. Exceptinemergencycases,avehiclemustnotleavetheplatoonwithoutauthorisationfromtheleader.
2. Whenauthorisedto leave,autonomouscontrol should notbe disableduntil themaximumallowable platoonspacing hasbeenachieved.
Bothleavingandjoiningprocedurestemporarilychangethespacingoftheplatoonindifferentpartsthatcouldendanger safety ofthe platoon.There could be caseswhere the distanceoftwo parts ofthe platoon wouldexceedthe maximum permitted distance or a crash could be possible. An example scenario of such cases can be as follows: when a vehicle increasesthedistancetothat precedingitto leavetheplatoonandatthesametime thevehiclerightbehind theleaving vehicleincreasesitsdistancetoletanothervehicletojointotheplatoon.Iftheleavingvehicleleavestheplatoonbeforethe joiningvehiclehasjoined,thedistancebetweentwopartswouldbeverylargeoracrashwiththejoiningvehiclewouldbe plausibleiftheplatoontriestodecreasethisdistance.Theplatoonleaderblockstheoccurrenceofbothleavingandjoining atthesametime.
In therestof thepaper,we first explain ouragent-based development ofan automotive platoon.The development is basedontheprocedures introducedinSection2.Wethenverifythecorrectnessofjoiningandleavingproceduresandin generalthesafetyofthewholeplatoon.
3. Agent-baseddevelopmentofautomotiveplatoon
Weemployahybridagentarchitecturebasedon[10]foreachvehicle(seeFig. 1).
Real-timecontinuouscontrolofthevehicleismanagedbyfeedbackcontrollers,implementedinMATLAB.Thereal-time continuouscontrolofthevehiclealsoobservestheenvironmentthroughitssensoryinput.Thisiscalledthe
Physical Engine
. The Physical Engine, inturn, communicateswith an Abstraction Agent that extractsdiscrete information fromstreams of continuousdataandpassesthisontoaDecision-Making Agent
.TheDecision-MakingAgentisarationalagentwhichdirects thePhysical Engineby passingitinstructionsthroughtheAbstractionAgent.InstructionsfromtheDecision-MakingAgent totheAbstractionAgentareinterpretedintomeaningfulinstructionsforthePhysicalEngine.To provide the complexenvironment necessaryfor effectivesimulation and testing, we use an automotive simulator, TORCS,3 to implementthe environment component ofthe architecture. The Physical Engineis implemented inMATLAB,
perform goal +!g [perform] addinganewgoaltoperformsomeactions
achievement goal +!g [achieve] addinganewgoalandcontinuouslyattemptingtheplansassociatedwiththegoaluntiltheagent hasacquiredthebeliefg
believe +b addinganewbeliefb disbelieve -b removingabeliefb on-hold believe *b waitingforbeliefbtobetrue
plan trigger : guard←body triggeristheadditionofagoalorabelief;guardgivesconditionsunderwhichtheplancanbe executed;bodyisastackofactionsthatshouldbeperformed
[image:5.561.34.504.55.155.2]guard condition B x checkingifbeliefxisperceivable guard condition G x checkingifgoalxhasbeenadded action perf(x) actionxcausingtheexecutionofx
Fig. 2.Gwendolen[8]syntax.
whilebothAbstractionandDecision-MakingAgents areprogrammedintheGwendolenprogramminglanguage. An inter-facebetweenTORCSandMATLAB/SimulinkhasbeendevelopedthatprovidesameanstocontrolvehiclesfromMATLABand Simulink.4 InSection 3.1,we overviewthe Gwendolenprogramminglanguage along withtheagent codeforthe joining scenario.
TheDecision-MakingAgentmakeshigh-leveldecisionsbasedonitsbeliefs,goals,etc.Thesedecisionsareatthelevelof
agreement,
coarse vehicle commands
,communications
,etc., andaretranslated,by theAbstractionAgent,toprovideinput for thePhysical Engine. Asthe PhysicalEngineruns, realdataandactivityis translated,againby theAbstraction Agent,into beliefsfortheDecision-MakingAgent.Animportantpartofthedesignprocessinvolvesensuringthatallrelevanthigh-level commandsandlowerlevelcontrolsarematched.3.1. Gwendolen
programming language
Gwendolen is a declarative logic-programming language incorporating explicit representations of goals, beliefs, and plans.Listing 1showssomeoftheGwendolencodefromthedecision-makingagentforthejoiningprocedureforafollower vehicle.5
Gwendolenuses manysyntactic conventionsfromBDI agent languages.We describe thoseneededto understandthe example(the syntaxisalsosummarised inFig. 2).: +!g indicatestheaddition ofthegoalg;+b indicatestheaddition of thebeliefb;while-bindicatestheremovalofthebelief.
∗
b indicatesthatexecutionofaplanissuspendeduntilthebelief,b,becomestrue.Plansconsistofthreeparts,withthepattern
t r i g g e r : guard <− body ;
The ‘trigger’isthe additionofagoal ora belief(beliefs maybeacquiredthanks totheoperation ofperceptionandasa resultofinternaldeliberation); the‘guard’ statesconditionsabouttheagent’sbeliefsandgoalswhichmustbetruebefore the plan canbecome active;and the‘body’ is a stackof ‘deeds’the agent performs in orderto execute the plan.These deedstypicallyinvolvetheadditionanddeletionofgoalsandbeliefsaswell as
actions
(e.g.,perf(changing_lane(1)))which referto codethatis delegatedto non-rationalpartsofthesystems.Plans maycontain uninstantiatedvariables whichwe representwithupper-caseletters,asiscommoninlogicprogramminglanguages.Whenaplan’striggerismatchedtoanew belieforgoalthiswill generateaunifier thatinstantiates anyvariablesinthe trigger,andevaluationofthe plan’sguard mayinstantiatefurthervariables.Thisunifieristhenappliedtothebodyoftheplanandthisbodyisaddedtoanintention
thatalreadycontainedthetriggerevent.
AfulloperationalsemanticsforGwendolenisavailablein[7,8]butinbriefaGwendolenagenthasa
reasoning
cyclethat followsthe characteristicsofreasoning outlinedby theBDI architecture [33].The agentmovesthrough thiscyclepolling anexternalenvironmentforperceptionsandmessages;convertingtheseintobeliefsandcreatingintentionsfromanynew beliefs; selectingan intentionforconsideration(bydefaultintentions arestoredasa queueandthefirst inthequeueis selected); ifthe intentionhas no associated plan body then the agent looks fora plan that matches the trigger event, extractedfromtheenvironment,(andbydefaultselectsthefirstfromtheprogramthatapplies)andplacesthebodyofthis plan ontheintention;the agent thenprocesses thefirstdeedon theintention(e.g.,addingorremoving a belief,adding orremoving asub-goal, orperformingan action) andthenplaces theintentionatthe endoftheintentionqueuebefore performingperceptiononcemore.PlanguardsareevaluatedusingProlog-stylereasoning.ThestatementB b istrueifeitherb unifieswithagroundliteral in the agent’sbelief base, or there exists a reasoning rule ofthe form h :
−
body where h unifies with b and body is a conjunctionof literalswhich in turneither unifywith a literal inthe agent’sbelief baseor can be deduced via further reasoningrules.SimilarlythestatementG g istrueifg
unifieswithagroundliteralintheagent’sgoalbase,orthereexists areasoningrulefromwhichitcanbededuced.Negationisindicatedwith~ anditssemanticsarenegationbyfailureasin Prolog.Listing 1containsonereasoningrule(line3)whichallowstheagenttodeducethetruthofjoining(X, Y) forsome X andY iftheagent’snameisX anditbelievesplatoon−
ok (abeliefthatisassertedbythefinalplaninthelisting).Listing1Afollowervehicle’scode.
1 2 Reasoning Rules
3 j o i n i n g (X , Y):−name(X ) , platoon−ok
4 5 Plans
6 +! j o i n i n g (X , Y ) [ achieve ] : {Bname(X) , ~B join_agreement (X , Y ) }
7 <− +! speed_contr ( 0 ) [ perform ] , +! s t e e r i n g _ c o n t r ( 0 ) [ perform ] ,
8 . send ( leader , :t e l l, j o i n _ r e q (X , Y ) ) , ∗join_agreement (X , Y ) ;
9 10 +! j o i n i n g (X , Y ) [ achieve ] : {Bname(X ) , B join_agreement (X , Y ) ,
11 ~B changed_lane , ~G s e t _ s p a c i n g ( Z ) [ achieve ] }
12 <− +! speed_contr ( 0 ) [ perform ] , +! s t e e r i n g _ c o n t r ( 0 ) [ perform ] ,
13 p e r f ( changing_lane ( 1 ) ) , ∗changed_lane ;
14 15 +! j o i n i n g (X , Y ) [ achieve ] : {Bname(X ) , B join_agreement (X , Y ) ,
16 B changed_lane , ~B speed_contr , ~ B s t e e r i n g _ c o n t r ,
17 ~B j o i n i n g _ d i s t a n c e , ~G s e t _ s p a c i n g ( Z ) [ achieve ] }
18 <− +! speed_contr ( 1 ) [ perform ] , ∗j o i n i n g _ d i s t a n c e ;
19 20 +! j o i n i n g (X , Y ) [ achieve ] : {Bname(X ) , B join_agreement (X , Y ) ,
21 B changed_lane , B speed_contr , ~B s t e e r i n g _ c o n t r ,
22 B j o i n i n g _ d i s t a n c e , ~G s e t _ s p a c i n g ( Z ) [ achieve ] }
23 <− +! s t e e r i n g _ c o n t r ( 1 ) [ perform ] ;
24 25 +! j o i n i n g (X , Y ) [ achieve ] : {Bname(X ) , B join_agreement (X , Y ) ,
26 B changed_lane , ~B speed_contr , ~B s t e e r i n g _ c o n t r ,
27 B j o i n i n g _ d i s t a n c e , ~G s e t _ s p a c i n g ( Z ) [ achieve ] }
28 <− +! speed_contr ( 1 ) [ perform ] , +! s t e e r i n g _ c o n t r ( 1 ) [ perform ] ;
29 30 +! j o i n i n g (X , Y ) [ achieve ] : {Bname(X ) , B join_agreement (X , Y ) ,
31 B changed_lane , B speed_contr , B s t e e r i n g _ c o n t r ,
32 B j o i n i n g _ d i s t a n c e , ~B platoon_m , ~G s e t _ s p a c i n g ( Z ) [ achieve ] }
33 <− . send ( leader , :t e l l, message (X , joined_succ ) ,
34 ∗platoon_m , +platoon−ok ;
Thus the first plan in Listing 1 (Lines 6–8) states that if the follower agent gains a goal to join the platoon (+! joining(X, Y)) andits nameisX (B name(X))andit doesnot yethavea joiningagreement(~B join_agreement(X, Y)), then it sets its speed andsteering controlsinto state 0(manual control) and sends a request to the leader to join the platoon.Itthenwaitsforanagreementtobereceived.
Gwendolenusestwotypesofgoals,
achieve
goalsandperform
goals(indicatedbythekeywords[achieve] and[perform] inplans).Anachievegoalindicatesastateoftheworldthattheagentwishestobringaboutandthesegoalspersistuntil they are achieved. So,in the caseofthe first plan,after the join agreementis received another planmay be attempted forthetrigger+! joining(X, Y) ifthebeliefjoining(X, Y) isnotyetdeduciblefromagent’sbeliefbaseandreasoningrules. Performgoals, ontheother hand,representsub-procedures.Theyarematched toaplan whichisthen executed andthe goalremovedwithoutanycheck thattheyhavesucceeded.+! speed_contr(0) and+! steering_contr(0) inthefirstplanare performgoals–theagentsimplyenactstherelevantactionsrequiredtoswitchthecontrolsystems.ThesecondplaninListing 1(Lines 10–13)canbeexecutedif,andonlyif,thefolloweragentbelievesithasajoin agree-mentbutdoesnotyetbelievethattheagenthassuccessfullychangedlanenorthattheagentcurrentlyhasagoaltoseta particularspacingbetweencars.Eachofthesubsequentplans inListing 1isexecutedina slightlydifferentcircumstance. When alane changegoessmoothly,eachplangetsexecutedatmostonce.Howeverifsomethinggoeswrong,forinstance thejoinagreementisrevokedhalfwaythroughthemanoeuvre,thenanearlierplanmaybeexecutedagain(e.g.,ifthejoin agreementisrevokedthentheagentreturnsthespeedandsteeringcontrolstomanualandrequestsanewjoinagreement fromtheleader).
The protocolembodied by theplans forachieving joining(X, Y) is that a lane change manoeuvre starts withmanual controlofspeedandsteering.Arequesttojointheplatoonissent.Whenanagreementisreceivedalanechangemanoeuvre is performed.Oncethelane issuccessfullychangedtherearetwo options,ifthecaristheappropriate distancefromthe one infront theautomated systemstake control ofbothspeed andsteering.Ifthe carisnot attheappropriate distance then theautomated systems takecontrol ofthe speeduntil an appropriatedistance isachievedandthentake control of the steering. At thispoint (the caris in the correct lane, thecorrect distance fromthe car in front andthe automated systems areinfullcontrol)theagent sendsa messageto theleaderconfirming asuccessfulmanoeuvre,awaitsreceipt of confirmationfromtheleaderandthenassertsaninternalbeliefthattheplatoonisnowOK.
Fig. 3.Overall platoon system.
byintegratingamoduleforthegenerationofplansviapartial observableMarkovdecisionprocess (POMDP)withtheBDI model[28].However,thisisoutofourpaper’sscopeandweassumetheagent’splanlibraryisfixed.
4. Verification
Thissectionpresentstheverificationoftheautomotiveplatooning.Westartbyoverviewingourverificationmethodology inSection4.1wheretheverificationtaskisdividedtotwomain parts:agentbehaviour andreal-timerequirementofthe system. We apply our methodology forverificationof automotive platoonin the restof sub-sections.In Section 4.2,the agentbehaviourisverifiedusingAJPF.Toverifythereal-timerequirementofthesystem,weusetheUppaalmodelchecker where a system is represented as timed automata. To generate timed automata for the system, first in Section 4.3 we introduceatranslationalgorithmthatgeneratesatimedautomatonfromanagentmodelbyabstractingtheinternalstates oftheagent.TheninSection4.4,wedescribeourUppaalmodelofthesystemfollowedwithitsverificationinSection4.5. Inthelastpartofthesection,wediscussthevalidationofverification.
4.1. Verification methodology
WecanvisualisetheoverallplatoonsystemasshowninFig. 3.
TheagentisaGwendolenprogram,the
Comms
componentisasimpletransferprotocol,andthevehiclerepresentsthe particularvehicularsystemthatweinteractwith.Thisistypicallyanautomotivecontrolsystemtogetherwithenvironmental interactions,andwehavevalidatedthisbothinsimulation(usingtheTORCSautomotivesimulation)andinphysicalvehicles (usingJaguaroutdoorrovervehicles).Limits to modelling/verification.Wearenotgoingtoformallyverifythevehicularcontrolsystems,andleavethistostandard mathematical (usually analytic) techniques from the Control Systems field [27]. These control components, for example involvingfollowingaprescribedpath,avoidinglocalobstacles,keepingdistancefromobjects,etc., arewell-establishedand standard. Instead,we willverifythe autonomousdecisionsthe vehiclesmake, capturedwithin each vehicle’s‘agent’ [16]. Eachagentrepresentstheautonomousdecision-makerwithineachvehicleandcorresponds,inpart,tothehumandriver’s decisions.Thesedecisionsinvolvedeciding
where to go
,when to turn
,when to stop
,what to do in unexpected situations
,etc.In thecaseofautonomousvehicleconvoys/platoons,theagent’s(and,hence,thevehicle’s)decisionsconcernwhen to join the
convoy,when to leave
,what to do in an emergency
,etc.So,webeginby abstractingfromall thevehiclecontrolsystemsandenvironmentalinteractions,representingtheseby one (potentially complex, depending on the vehicle/environment interactions) automaton. We also use an automaton to describethesimpletransferprotocolthatthevehiclesusefortheircommunication.Inboththesecaseswewilluse
Timed
Automata[1].Simplified,inthesensethattheabstraction aspects
,continuous control
,andenvironmental considerations
areall bundledinto“VehicleControlandEnvironment”,ourarchitectureis:Table 1
Anexampleoftimedanduntimedautomataoftheplatooning.
Simplified timedautomata ofourarchitecture are giveninTable 1.6 The communication automatonrepresentsthe re-quiredcommunicationsforanagenttoreceiveajoinagreementnamely,receivingajoinrequestfromanagentandsending back a join agreement. The vehicle automatonrepresents the behaviour ofa vehicle when it receives a “changing lane” command fromthe agent. The
agent
automaton representsa simplified behaviourof an agent forjoiningtothe platoon. Transitionsbetweenlocationsaresynchronisedthroughrcv
andsnd
channels.Forinstance,whentheagentautomatonsend ajoinrequest(snd(
j_req)
),itsynchroniseswithrcv
(
j_req)
ofthecommunicationautomaton.Indescribingagentbehaviour, theagentautomatonincorporatesthedimensionsof
belief
andintention
,allowing repre-sentation ofrationalagentbehaviour.Thus,theformalstructuresthatallow ustofullyrepresentthewholesystemabove are quite complex,combiningtimedrelations aswell asrelations foreach ofthe beliefandintentiondimensions [31,1]. We will not describe thisformal modelin detailbutjustnote that it isafusion
[15,17,25] oftimedandBDI structures, L,
A
,
C,
E
,
inv,
R
B,
R
I,
l.Here:L
isafiniteset oflocations; A isafiniteset ofactions;C
isafiniteset ofclocks,forthetimedaspect;
E
⊆
L×
(
C)
×
A×
2C×
L isasetof(timed)edgesbetweenlocations, where(
C)
isasetofsimpleclock constraints such as “c<
4”; inv:
L→
(
C)
is a function associatingeach location withsome clock constraint in(
C)
;RB
:
Ag→
(
L×
L)
provides thebelief relation betweenlocationsfor agentsin Ag; RI:
Ag→
(
L×
L)
,intention relationforagents betweenlocations; and
l
:
L→
2A P isa labellingfunction essentiallycapturing thosepropositionstrueateach location(A P isasetofatomicpropositions).The logic is then interpreted over such structures combining [17] the syntax of timed temporal logic, for example
♦
≤5finish(i.e.,“eventuallywithin5secondsfinish
holds”),andthesyntaxofmodallogicsofbelief,desireandintention,forexample Bxstarted(i.e.“agent
x
believesthatstarted
istrue”).In principle, though very complex,we could provide all ourplatoon requirements insuch a logic, build structuresof the above formforour platoonimplementation,andthen develop a model-checking approachforthiscombination [24]. However,thereareseveralreasonswechoosetoabstractandseparatethetimed/agentstrands,asfollows.
(I) For
certification
itisimportantthat weverifythe actualagentprogram usedineach vehicle,not aderived modelof this. Consequently,weutiliseaprogram model checking
[38] approachtoassessthecorrectnessofeachagentprogram. Programmodelcheckinganalysestheactualcodeindetailratherthangeneratingafurtherfinitestatemodelto rep-resent it.Forthisformalverificationoftheagent’sautonomousdecisions,weuseAJPF [12],an extension oftheJava PathFinder(JPF)programmodelchecker[23]forGwendolenthatallowsverificationofbelief/intentionproperties.This istheonlyprogrammodel-checkerforrationalagents.(II) We do not have thedetailed implementations ofall the communication protocols,the vehicular controls, and envi-ronmental interactions, and so must use an abstract, formal model to describe these, rather than actual code. This allowsustonarrowtheverificationtoonlytheagentprogrambutwiththecostoflosingthegeneralityofresults.Our abstractiondictateswhattheabovecomponentsshoulddowithoutlookingathowtheyperformthoseactions. (III) JPF isan explicit-stateprogram-checkerandisrelativelyslow;AJPF buildsaBDIprogramminglayerontopofJPFand
isatleastanorderofmagnitudeslowerthanJPF.Consequently,AJPFcannotrealisticallybeusedforverificationofthe wholesystem.Inaddition,asAJPFdoesnotyethavereal-timecapabilities,thenverifyingtimingaspectswithinAJPFis difficult.
Whiletheseappearproblematic,thereareseveralusefulsimplificationsinourcontext:
Fig. 4.Timed automaton to untimed automaton.
Table 2
AnexampleofuntimedautomataofCommandVi.
•
Whenverifyingautonomousbehaviour, theformalverificationwecarry outconcerns theinteractionofbeliefs, inten-tions,etc., withineach agent. Thesedonot extendbetweenagentsandso,checkingofbeliefs, intentions,etc,can be localisedwithineachagent.•
Intherequirementstobechecked,thetimedandBDIformulaearequiteseparate,i.e.♦
≤5finish∧
Bxstartedbutnever♦
≤5Bxstartedor Bx
♦
≤5finish.Astheoverall logicisafusion,andsincethereareno explicittiming constraintswithin
anagentprogram(agentshavefastinternalcomputation),thisallowsustodealwiththetimingandagentdimensions separately.
So,givenanoverallsystem,
S
,overwhichwewishtocheckϕ
,thenwereduceS
|=
ϕ
totwoproblems:1. for each individual agent, Ai, within S,verify theagent propertiesfrom
ϕ
,i.e.ϕ
a,on the agent within an untimedenvironment(anover-approximation)whichaddressestheconcernsgivenin(I)and(III);and
2. verify the timing properties from
ϕ
i.e.ϕ
t, on the whole system where the agent program is replaced by a timedautomatondescribingsolely its
input–output
behaviour (abstractingfrominternal BDI reasoning)whichaddressesthe problemsgivenin(II).Weexplainbothoftheseinmoredetail,beforegivingtherelevanttheorems.
Timed automaton
−→
untimed automaton.Thisisessentiallyachievedbyover-approximation.Byremovingtimingconstraints fromautomatatransitions,wecanstillmovebetweenthesamestatesbuttherearenotimingrestrictionsonthismovement. Ifwe do thissystematically, asinFig. 4,then the newsystemrepresentsan over-approximation oftheoriginal asmore behavioursareallowable.Notethat goingfromatimedautomatontoan untimedone,forexamplereplacingbehaviourssuch as
♦
≤3recei ve by♦
recei ve providesthisover-approximation. Hence, forexample,Comms|=
ϕ
impliesComms
|=
ϕ
, butnot necessarilythe converse. Thisthen allowsustoverify7 Vi
CommsAi|=
ϕ
a using AJPFasdetailedtiming isremoved inthevehicle andcommunicationsmodels.
TakingtheexampleinTable 1,theuntimedautomatainTable 2aretheresultoftimingremovalforthe
communication
and
vehicle
timedautomata.NotethattheComms
andV
icommunicateswithAithroughmessagepassing.Agent model
−→
untimed automaton.Formalverification,viamodelchecking,ofanagentprogram[11]carriesoutby explor-ingallpossiblestatesofanagentmodel.Whiletheexplorationisperforming,amodelofagentbehaviourcanbegardually built.ThismodelisaBDIautomatonincorporatingallthetemporal,belief,andintentionaspectsoftheagent.Nowwewill abstractfromtheagentaspects tobuilda simpleautomatonwhereallthebelief/intentionupdatesareignored.Thisthen justcorrespondstothebasicI/Obehaviouroftheagent.SeeFig. 5.Formal verificationusingUppaalcan then carriedout on thewhole systemwithall agentsabstracted by substituting withuntimedautomata.Inthisabstraction,
Comms
andVi communicatewith Aithroughsynchronisationchannels.Wenowprovideproofsketchesforpropertiesoftheaboveabstractions.Forsimplicity,weassumethat Sconsistsofjust twoagents/vehicles;thisresultcantheneasilybegeneralisedtogreaternumbersofagents/vehicles.Below,recallthat:
•
V1 andV
2aretimedautomatarepresentingthevehiclecontrol,whileV1 andV
2 areuntimedabstractionsofthese; [image:9.561.100.441.190.241.2]Fig. 5.Agent model to untimed automaton.
•
A1and A2areBDIautomatarepresentingtheagentsmakingdecisions,while A1and A2areabstractionsofthesewithBDIelementsremoved;and
•
Comms12isatimedautomatonrepresentingtheinter-vehiclecommunications,whileComms12
isanuntimed abstrac-tionofthis(astandardBüchiautomaton).Theorem1.
Let S
==
V1A
1Comms12
A
2V
2. Ifa) V1
A
1Comms12
|=
ϕ
a and b) V2A
2Comms12
|=
ϕ
a andc) V1
A
1Comms12
A
2V
2|=
ϕ
t.then S
|=
ϕ
a∧
ϕ
t.Proof Sketch. Since
V
1 andComms12
areover-approximations,thebehavioursofV
1 includeallbehavioursofV1 andthebehavioursof
Comms12
includeallbehavioursofComms12
,andsoV1
A1Comms12|=
ϕ
aimpliesV1A1Comms12|=
ϕ
a.
Similarly, (b) gives us V2
A2Comms12|=
ϕ
a. As the agent properties inϕ
a are local, we can compose these to give V1A1Comms12A2V2|=
ϕ
a andsoS
|=
ϕ
a.By(c)weknowthat V1
A1Comms12A2V2|=
ϕ
t yet,containnotiming constraintsto beginwith, weknowthat A1and A2 giveusexactlythesame
timed
behaviours.Thus A1 andA1 areequivalent wrtϕ
t,asare A2 andA2.Consequently, V1A1Comms12A2V2|=
ϕ
t andsoS
|=
ϕ
t.ThesetwotogethergiveusS
|=
ϕ
a∧
ϕ
t.2
Theorem2.
If V
1A
1Comms12
A
2V
2|=
ϕ
tthen V1A
1Comms12
A
2V
2|=
ϕ
t.Proof Sketch. Sincethetimingbehaviourofeach Aiisidenticaltoeach Ai then
V1
A1Comms12A2V2and
V1
A1Comms12A2V2areequivalent.
2
Aswementionedearlier,V1, V2,and
Comms12
areover-approximation,andsoV1
A1Comms12A2V2|=
ϕ
adoes not necessarily implyV1
A1Comms12A2V2|=
ϕ
a.
Insummary,weabstractfromtimingbehavioursinvehicleandcommunicationmodelssothatwecancarryout(untimed) verificationoftheseincombinationwiththeagent,and(separately)abstractformdetailedBDIbehavioursintheagentso wecanverifytimedbehavioursofthecombinationoforiginalcommunicationsmodel,originalvehiclemodel,andabstracted agent. This is feasible since, in our application, BDI behaviours only occur within agents and the agent itself does not
contributetotimingdelays.
4.2. Individual agent verification using AJPF
Toverifyindividualagentproperties,weusetheAJPFmodelcheckeronouragent,writtenintheGwendolenlanguage, asabove.Forinstance,weverifythat:
Thissafetypropertycorresponds tothefirstrequirementofjoiningaplatoon,asgiveninSection2,andcanbedefined as:
2(Gf3platoon_m(f3,f1)
→ ¬Df3perf(changing_lane(1))W Bf3join_agr(f3,f1))
(1)
Here f3 refersto anon-membervehiclewhichtriestojoin theplatoon,infrontofmembervehiclef1. Gx y standsfora
goal y that agent x triestoachieve, Bx z stands fora belief z of agent x, and Dx k stands foran action k that agent
x
performs/does.The standardLTLoperators, suchas
2
meaning “alwaysinthe future”andW meaning“unless”,are used. Aninstanceoftheabovewheretheagentnever
receivesajoinagreement,is:2(Gf3platoon_m(f3,f1)&¬Bf3join_agr(f3,f1))
→2¬Df3perf(changing_lane(1)) (2)
Tobe ableto checksuch a property,incoming perceptionsandcommunications shouldbe provided.One waytoprovide suchinputsistogeneratethemrandomly. However,thewholecombinationofincomingperceptionsandcommunications issignificantandconsequentlythepropertycheckingisinfeasible.Anotherwaytoprovideincomingperceptionsand com-munications istorepresentinputcombinationsthat areviablefromthepractical vehiclebehaviour.Thisoftenmakesthe property checking feasible. Thus, we supply two automata: Comm, representing communication to and from the other agents;andVi,representingvehicleresponsestoagent actions.Thelattersolutionforprovidingincomingperceptionsand communications,on one hand,removesall possibilities forproviding irrelevant inputsby mimickinga real modelofthe communicationandvehiclecontrolleranddecreasesthe verificationtime. Ontheother hand,itdoesnotensurethe cor-rectness ofthe agent code against completely unanticipated input. Under this configuration, we were able to carry out theagent verificationinaround20minutes. (RecallthatAJPF is
very
resource hungry.)Notethat wecan, withadditional resources,weakenexpectationsontheenvironmentandsoverifytheagentinincreasinglyunlikelyscenarios.We haveverifieda rangeof safetyandlivenesspropertiesandwe provide somejoining andleavingexamplesbelow. Note that thefollowing propertiescan also be similarlyexpressed interms ofthe weak until operator, W; however, we denoteaparticularinstanceofthesepropertiesforthesakeofbrevity.
If a vehicle ever sends a ‘join’ request to the leader and eventually receives the join agreement and it is not already in the correct lane, it initiates ‘joining’ the platoon by performing “changing lane”.
2(Gf3platoon_m(f3,f1)&¬Bf3changed_lane&
2(ItDf3send(leader,tell,message(f3,1,f1))→♦Bf3join_agr(f3,f1))) →♦Df3perf(changing_lane(1))
(3)
Property 3isalivenesspropertyensuringthateventually thejoiningprocedureinitiatesthechanginglane controlsystem onceitsconditionisfulfilled.Similarly,wecanverifyotherpropertiestoshow progresssuchas“eventuallythespeedand steeringcontrollersareswitchedtoautomaticifpre-conditionshold”.Someotherverifiedpropertiesensuringsafeoperation oftheplatoonareasfollows.
If a vehicle never believes it has changed its lane, then it never switches to the automatic speed controller.
2(Gf3platoon_m(f3,f1)&¬Bf3changed_lane)
→2¬Df3perf(speed_controller(1)) (4)
If a vehicle never believes it has received a confirmation from the leader, then it never switches to the automatic speed controller.
2(Gf3platoon_m(f3,f1)&¬Bf3join_agr(f3,f1) ) →2¬Df3perf(speed_controller(1))
(5)
If a vehicle never believes it is sufficiently close to the preceding vehicle, it never switches to the automatic steering controller.
2(Gf3platoon_m(f3,f1)& ¬Bf3joining_distance) →2¬Df3perf(steering_controller(1))
If a vehicle never believes it has received a confirmation from the leader to leave the platoon, i.e., increasing spacing has been achieved, then it never disables its autonomous control.
Note that the leader sends back the ‘leave’ agreement to follower3 if, andonly if, it received an acknowledgement from
follower3showingthatspacinghasbeenincreased.
2(Gf3leave_platoon& ¬Bf3leave_agr(f3))
→2¬Df3perf(speed_controller(0))
(7)
It isimportant torecall thatperceptionsandcommunications cominginto theagent are representedasinternal beliefs. Hence the proliferationofbeliefoperators.The AJPF programmodelchecker exploresallpossiblecombinations ofshared beliefsandmessagesandso,evenwiththerelativelylow numberofperceptionsabove,thecombinatorialexplosion asso-ciatedwithexploringallpossibilitiesisverysignificant.Therefore,verifyingthewholemulti-agentplatooningsystemusing AJPFisinfeasible.
AsexplainedinSection4.1,toverifytheglobalpropertiesofmulti-agentplatooning,weuseacomplementaryapproach. We generateamodelofthewholesystemastimed-automataandusetheUppaalmodelcheckertoestablishthe(timed) correctnessofmulti-agentplatooning.Inthefollowing,wereview therelevanttimed-automataandhighlightsome ofthe globalsafetypropertiesofvehicleplatooningthathavebeenverifiedusingUppaal.
4.3. From agent model to untimed automata
In thissectionwe define ouralgorithmforextractingan untimedautomaton fromanagent model.First,we generate a statemodelofanagent modelthroughtheexecutionoftheAJPF modelchecker.Ineverystatewe haveasetofbeliefs, intentions,actions,andmessageswhichcanrepresenteithertheinternalstateoftheagent,ortheinput/outputinformation from/totheenvironment.Theenvironment isan abstractrepresentationofavehicleandotheragents,inthiscontext.For automatongenerationonlytheinput/outputinformationisrequired.Therefore,weremovesuperfluousinformationfromthe statemodelandmergeequivalentstates,asdescribedinAlgorithm 1.Ourapproachissimilartotheautomataminimisation algorithms[21,22],whereequivalentstatesareidentifiedforadeterministicfiniteautomata
A
.Equivalentstatesaremerged tobuildthestatesoftheminimalautomaton A.The StripInternalBeliefs function updatesthe setof states(S) byfirstly removing all internal beliefs andintentions, andthenconvertingbeliefsandactionsrepresentinginput/outputinformationtopropositions.Typically,thisreplacesbelief formulaesuch asB xyz by propositionssuchas
bel
_xyz.TheRemoveEmptyfunction checksthepropositionsofstates.For every emptystate,thesetofedges(E)isupdatedbyredirectingtheincomingandoutgoingedgesoftheemptystate.The setofstates(S)isupdatedbyremovingtheisolatedemptystate.TheMatchAndMergeDuplicatesfunctionfindsallstates having equivalent outgoingedges andpropositions. Thesetof edges(E) isupdated by redirecting theincomingedges of equivalentstates.Theisolatedequivalentstateisthenremovedfromthesetofstates.The agent modelconsists of28 beliefs, 14actions/messages, and12 goals.In total, 22beliefs, actions, andmessages can be converted to propositions that represent input/output information.The generated state modelof the agent code consists of 1396 states and 2361 transitions. The failure of the communication component, represented by the Comms
automaton,hasnotbeenconsideredinthegenerationofthestatemodel.Reliablecommunicationplaysakeyroleinsafety ofplatooning.Ifthecommunicationisunreliable,theplatoondissolvesandconsequentlyanyjoiningandleavingoperation will be stopped. In other words, we can only guarantee the correctness of joining and leavingprotocols under reliable communicationcondition. Incontrast,some failuresofthe vehicle(represented asthe Vi automaton), namelythefailure to changelanes, havebeenincludedinthemodel.Changinglane isa procedurethatadriverperforms andcangowrong duetohumanerror.Therefore,inordertohavearealisticdesign,weconsiderhumanerror.Astheplatooningscenariosare sequential andinvolvedisjointsetsof events,ageneratedautomaton fromtheagent modelshould consistof atleast22 states.
The output of Algorithm 1 shows that 650 states, out of 1396 generated states ofthe agent code, are empty states whichareremoved bytheRemoveEmptyfunction.TheMatchAndMergeDuplicatesfunctionreducesthenumberofstates and transitions to 174and 519, respectively. To optimise the output of Algorithm 1, two more reduction functions are introduced that are shown in Fig. 6. The first function finds stateswhose outgoingedges are to states withequivalent propositions,andredirectsalltheincomingedgesandremovesisolatedstates.Twostates129and145inFig. 6(2–3)are mergedwith127usingthisreduction.ThesecondfunctionfindscasesasshowninFig. 6(4)whereastatewithequivalent propositions, i.e., state 126, redirects its incoming state 125to state 127 which is directly connectedto 125.As we do not takeintoaccount timingaspects here,moving betweenidenticalstates,thenthesecanbemerged.Thissimplification reducesthenumberofstatesandtransitionsto89and560,respectively.The reductionfunctionsare repeatedlyexecuted untilthenumberingofstatesconvergestoasizeof34.
The Uppaal agent automaton shown in Fig. 7 is composed of 29 locations which are extracted from the output of
Algorithm1Agentmodeltountimedautomatonalgorithm.
1:translateAgentToAutomaton(S,E)
Input:astatemodelA,representingasagraphG=(S,E) Output:aminimised statemodelA,representingas(S,E)
2:(S,E)=StripInternalBeliefs(S,E) 3:(S,E)=RemoveEmpty(S,E))
4:(S,E)=MatchAndMergeDuplicates(S,E)
5:functionStripInternalBeliefs(S,E) 6: S= ∅
7: foreachs∈Sdo
8: s=swithallinternalbeliefsandintentionsremoved
9: s=swithallremainingbeliefsandmessagesconvertedtopropositions 10: S=S∪ {s}
11: end for
12: return(S,E) 13:end function
14:functionRemoveEmpty(S,E) 15: S=S,E=E
16: foreachs∈Sdo
17: if prop(s)= ∅then
18: E=(E∪ {(x,y)|x∈in(s)∧y∈out(s)}) \({(x,s)|x∈in(s)}∪ {(s,y)|y∈out(s)}) 19: S=S\ {s}
20: end if
21: end for
22: return(S,E) 23:end function
24:functionMatchAndMergeDuplicates(S,E) 25: S=S,E=E
26: foreachsi∈Sdo 27: foreachsj∈S\ {si}do
28: if out(si)=out(sj)andprop(si)=prop(sj)then 29: E=(E∪ {(sin,si)|sin∈in(sj)})
\({(sj,sout)|sout∈out(sj)}∪ {(sin,sj)|sin∈in(sj)}) 30: S=S\ {sj}
31: end if
32: end for
33: end for
[image:13.561.38.506.53.538.2]34: return(S,E) 35:end function
Fig. 6.A reduction example.
agentcodecanbetransformedtotwoconsecutivelocationsinUppaalwithtwosynchronisationchannelswhichrepresent anewbeliefandaperforming action.Furthermore,theoutput ofAlgorithm 1isanuntimed automatonthatneeds tobe supplementedwithtimingrequirements.
4.4. Timed automata model of automotive platoons
We model vehicle platooning in Uppaalas a parallel composition of identical processes describing the behaviour of each individual vehicle in the platoon along with an extra process describing the behaviour of the platoon leader (the
leaderautomaton).Eachofthesevehicleprocessesisaparallel compositionoftwotimedautomata,the
vehicle control and
environmentandagent
.TheUppaalagent
automaton,8 inturn,comprisesbothComms
and Ai components,asdescribedinFig. 7.Example Uppaalagentautomaton.
Section4.1.Thismeansthatthetimingrequirementsof
Comms
componentareaddedtountimedtransitionsofA
i tomodel theUppaalagentautomaton.The vehicleautomaton suppliesincomingperceptionsfortheUppaalagent automaton.It describesthesensor models andaction execution. Thevehicle automatonreceives,andresponds to,theaction commands ofthecorresponding agent throughsixpairsofbinarychannelsmodelling
change-lane
,set-space
,speed-manual/auto
andsteering-manual/auto
commands andresponses.Tomodeltimingbehaviour,wedefineaclockforthevehicleautomatonwhichmodelsthetimeassessments for “changing lane”, “settingspace” and“speed auto” actions. Engineering analysisof thestability androbustness of the low level,continuouscontrollershasbeenusedtoensurethatthesesystemsfunctioncorrectly,withincertainbounds[36, 27].Theseboundscorrespondtoquantitiessuchasaccelerationandlateralvelocitywhichsubsequentlyimposeconstraints onthetimingofdecisions.Basedontheseanalysis,actionsspeed-manual
andsteering-manual/auto
happenimmediatelyand actionschange-lane
,set-space
andspeed-auto
take20±
5,10±
5 and10±
5 seconds,respectively.Forinstance,thespeed-auto
action completes in 5 seconds in the best case and in 15 seconds in the worst case. It means when the Uppaalagent
automatonsendsa
speed-auto
commandtothevehicle,itshouldwaitbetween5to15secondsforclose-enough
response. Notethattimingvaluesareparametersofthetimedautomataandcanbeadjustedbasedonthetypeofvehicles.Inother words,tuningthetimingvaluesinthetimedautomatamodelandsystempropertiesdonotimpactontheverificationresult oftheautomotiveplatoon.The Uppaal
agent
automatonmodelsanabstracted
versionoftheGwendolenagent byexcluding allinternal computa-tions ofthe agent andadded time constraintsof Comms. Theoverall structure of an Uppaalagent consistsof 5regions, showninFig. 7.TheseregionsareextractedfromtheoutputofAlgorithm 1.Fig. 8.Theleaderautomaton.
leader.Thesechannelsare usedtomodelthe messagepassingbetweenthe followingagentsandtheleader, modelledin thedecision-makingagent(Section3).Furthermore,eachagentautomatonalsocontainschannelstosendcommandstoits vehicleandreceiveacknowledgementsfromitsvehicle.Theagentautomatonhasaclock
process-time
thatisusedtomodel thetimeconsumptionforachievinggoals.4.5. Multi-agent platooning verification using Uppaal
Now we have timedautomata representations of the platoon, we can carry out verificationof their properties using Uppaal. Forsimplicity,weanalysetheglobalandtimingpropertiesofamulti-agent platooncomposedsimplyofaleader andthreevehicles(withthreecorresponding)agents.However,theverificationcanperformforarbitrarylengthofplatoon by simplyinstantiatingmorevehiclesandagents.Weassumevehiclescanalwayssetspacingandjoiningdistanceintime, i.e., 10
±
5, butcan fail to change lane intime, i.e., lessthan 20+
5.In the following, we firstgive examples ofglobal propertiesinvolvingthecoordinationbetweentheleader andthefollowers.Second,weevaluatetiming requirements:the safelower andupperboundsforjoiningandleavingactivities. Weobservedthat theverificationofthesepropertiestook less than 3 seconds using Uppaal. The reasonis that the behaviour of the system is sequential and the interleaving is restrictedintheleader
automaton.Unreliablecommunicationisnotacceptableforautomotive platoonsandifsuch errors are detectedtheplatoonisdissolved.Therefore,wedidnotconsidercommunicationerrorsinourautomata;however,the changinglanefailurecanoccur.Theleader
automatondetectsthefailurewhenitdoesnotreceiveanyacknowledgementin duetimeandreadjuststheplatoonandgoingbacktoidle
state,i.e.,readyforthenextrequest.If anagent everreceives a joiningagreementfromthe leader, thenthe precedingagent hasincreasedits spacetoits frontagent.Thispropertyisformulatedforagent
a3
asfollows(Arepresents“onallpaths”):A2((a3.rdy_ch_lane&&l.joining_vehicle.front==2)
imply(a2.incr_spacing&& a2.spacing_done)) (8)
where a3 is the agent which is in the rdy_ch_lane location, i.e., the agent has received a joining agreement, variable
joining_vehicle
.
f ront indicates the identification ofthe preceding agent, flag a2.
incr_spacing models that the preceding agent hasreceivedan “increasespace”commandfromtheleaderand, finally,flaga
2.
spacing_donemodels whetheragenta2 has successfully increased its space. Flag a2
.
spacing_done turns true when a2 and leader are synchronised throughset_spacing_f rom channel, i.e., the leader is informed that a2 has successfully increasedspace. We can also verify this property foragentsa2and
a4
.Property 8is asafetyrequirementensuringthat avehicleinitiates“changinglane” onlyif sufficientspacingisprovided.The next propertyof interest is whethera joiningrequest always ends up increasing space ofthe preceding vehicle. To expressthis property, we usethe leads to property form, written
ϕ
ψ
.It states that wheneverϕ
is satisfied, then eventuallyψ
willbe satisfied.Such propertiesarewrittenasϕ
ψ
inUppaal.We verifytheproperty(9)toshowthat wheneveragenta3
isinthewait_ j_ agr
location,i.e., hassenta joiningrequest,andagenta2
istheprecedingvehiclein theplatoon,theneventuallya3
willreceiveanincreasingspacecommandandwillperformtheaction.(a3.waiting_for_j_agr&&l.joining_vehicle.front==2)
(a2.incr_spacing&& a2.spacing_done) (9)
Toensurethatthespacingalwaysdecreasesafterajoiningprocedure,i.e.,platoonreturnsbacktoanormalstate,weverify that ifevertheleaderreceives ajoiningrequest,iteventually sendsadecreasing spacecommandtotheprecedingagent unlessthejoinedagentisthefinaloneintheplatoon.
A2((a3.join_completed&&l.joining_vehicle.front==2)
imply(!a2.incr_spacing&&a2.spacing_done)) (10)
Given the required time for a vehicle to carry out “set spacing”, “joining distance” and “changing lane” tasks, we are interestedinverifyingifanagentaccomplishesjoiningtheplatoonwithinanexpectedinterval:waitingtimeforagreement +changinglane+joiningdistance+waitingtimeforleaderconfirmation,representedinProperty 11.
A2(a2.join_completed imply
(a2.process_time≥50&&a2.process_time≤90)) (11)
Similarly,wecheckifanagentleavesaplatoonwithinan expectedinterval:waitingtimeforagreement+changinglane+ waitingtimeforleaderconfirmation.Waitingtimeforagreementisequaltothetimeneededtosetspaceandwaitingtime forleaderconfirmationiszerobecauseweassumeswitchingsteeringcontrollersisimmediate.
A2(a2.leave_completed imply
(a2.process_time≥30&&a2.process_time≤50)) (12)
4.6. Validating the verification
wefindapropertythatdoesnotholdwemustbesurethatthisisduetothesystemitselfandnottoafailureintroduced throughourabstractionprocess.Consequently,weprovideacloselinkbetweenformalverification,simulationand (increas-ingly)real-world testing. Thisis a generalproblemwithall formal verificationtechniquesapplied to real-worldsystems. Sincewecan neverpreciselyformalisethe“realworld” wemustprovideabstractionsoftheworldandcarryout verifica-tionwiththese.Theabstractionsmustinturnbevalidatedtoseeiftheycapturemostoftheviablerealworldbehaviour.If theydonot,thentheywillberefinedandverificationagaincarriedout.
It must be emphasisedthat the agent codethat we verifyis actually the codethat controlsthe vehicle,both inthe TORCSsimulationandintherealvehiclethatwearedeveloping.Animperfectenvironmentalabstractioncanmaskpossible problemsinanagent modelinawaywhichevenformalverificationoftheagentincludingtheenvironmentalabstraction, failstofind.Forinstance,inthefirstattemptto modeltheplatooning agentmodel,theenvironmentalabstractionalways deliveredachangelaneperceptionwhenachangelaneactionhadbeenrequested.Thisdisguisedthelackofrequiredplans for the agent to deal with situations where a vehicle failed to change lane. The inappropriate abstraction was detected duringsimulationandprovidedmeanstorefinetheenvironmentalabstractionandconsequentlytheagentmodel.Thus,as longastheenvironmentalabstractionsarecorrect,wecanbesureofthedecisionsmadebytheagent.
Thisclearlyleavesthequestionofwhethertheenvironmentalabstractionsareappropriateornot.Thisisnotsomething thatcanbe handledbyformalverificationandsoit isherethatwe utilise
validation
,boththrough TORCSsimulationand physical vehicle tests. These help us assess how realistic and appropriate the abstractions we use are, and provide the impetustorefinetheseifnecessary.5. Concludingremarks
Theverificationofsafetyconsiderationsforautomotiveplatooningisadifficulttask.Thereareseveralreasonsforthis.
•
Theseare non-trivialhybrid autonomoussystems,witheach vehicle mixingfeedback controllers andagent decision-making.•
There is astrong requirementto verify theactual
code used inthe implementation,rather than extractinga formal modeloftheprogram’sbehaviour—thisleadsontoprogram
model-checking,whichisresourceintensive.•
Rational agents are essential for capturinghigh-level autonomy and, asthere are no other practical systemsable to model-checktemporalandmodalpropertiesofcomplexBDIagents,weareledtoAJPF.•
AJPFisvery
resourceintensive(aswehaveseen,12hoursforsomeagentproperties)andcannotbepracticallyusedto verifywholesystempropertiesofautomotiveplatooning.•
Especially when interacting with real vehicles we need to verify timed properties,as timing considerations can be crucial.Thus it isperhaps not surprisingthat such formal verificationhasnever beenreportedbefore. Inorder toaddress all of theaboveconcerns,wehaveadoptedacombinedstrategy.WeuseAJPFtoverifyindividualagentproperties,givenrealistic abstractionsof environmentalinteractions. AJPF verificationisviablefor individualagents,though remains resource hun-gry.We thenabstractfromtheBDI codeandproducean abstractagentautomatonsuitable foruseinUppaalverification. Wehaveshownhowthiscan becarriedout automatically,producing smallertimedautomata withthe“internals”of au-tonomousdecision-makingremoved.Essentially,weremovethemodellingofthedecision-makingprocessandjustleavethe possibledecisionsthatcanbemade.Thisthenallowsustoformallyverifyplatoonrequirementsandsafetyconsiderations, asampleofwhichwehaveincluded.
5.1. Related work
Aspatialintervallogichasbeenpresentedin[20,19]toabstractfromthedynamicsofvehiclestoallowforspatial rea-soning. Theproposedlogicalframework aimstoprovecollisionfreedomoflane-changemanoeuvresbyintroducingalocal view ofeach vehicle.Similartoourapproach, vehicledynamicsareseparatedfromspatial reasoning;however,thefocus, in [20,19], is onreasoning aboutcontrollers not the high-level decisionmaking component ofthe system. A framework basedon
π
-calculushasbeenpresentedin[5]forformalmodellingofhigh-level decisionmakingofplatooningwhilethe closed-loopcontrolsystemsaremodelledusinghybridautomata.Aswithourapproach,thediscreteandcontinuousaspects ofthesystemareseparated, thoughnoverificationhasbeencarriedoutin[5]toensurethecorrectnessofthehigh-level decisionmakingcomponent.Acombinedverificationapproachforvehicleplatooningisproposedin[6]wherethesystembehaviourisspecifiedvia the CSPandB formalmethods.The approachin[6]isbasedonrefinementinwhicha systemisdevelopedinastepwise manner. The vehicle behaviour ismodelledas aB machine andthe communicationmodels asaCSP controller. The aim in[6]istorepresentacorrectmodelofarealphysicalvehicleforplatooningwhileourapproachaimstoverifyingthe co-operationbetweenvehiclesintheautomotiveplatoonandabstractthebehaviourofarealphysicalvehicle.Acompositional verificationapproachforvehicleplatooningisintroducedin[14]wherefeedbackcontrollersandagentdecision-makingare mixed.
Itshouldbenotedthattothebestofourknowledge,thereisnodirectlycomparableworkinliteraturewheretheactual agentcodeusedintheimplementationhasbeenverified.
5.2. Future work
Thereisclearlymuchfutureworktotackle.AnobviousoneistocontinueeffortstoimprovetheefficiencyofAJPF. Maintainingasafeplatoonincaseof
recoverable
latencyanddissolvingaplatooninthecaseofunrecoverable
latencyare two proceduresthatarenotyetimplementedinourverifiedagent code.Addingthesetwoprocedurestotheagentgrows thesystemspacetotheextentthatAJPFfailstoverifyanyproperty.Thus,weareinvestigatinganagentabstraction atthe levelofgoals,beliefsandintentionsinordertouseAJPFforverificationofmorecomplexagents.Sinceweareconcernedwithcertificationofautomotiveplatooninginpractice,weareaimingtoextractamore compre-hensivelistofformalpropertiesfromofficialplatoonrequirementdocuments.Relatedtothis,wearealsointheprocessof portingtheagentarchitectureontoarealvehicleandsoaimtocontinue andextendtestingoftheplatooningalgorithms inbothphysical,andsimulation,contexts.
Finally,animportantaspectofourtwoprongedstrategyistolinkthemodelsusedinUppaaltotheprogramsthatAJPF uses. Weprovided an algorithmto achievethisbut, inthispaperwe refined theUppaalmodelsby hand.Clearly, anext stepistofullyautomatethisprocess.
Acknowledgements
This research was funded by EPSRC via the research project “Verifiable Autonomy”, at Liverpool (EP/L024845) and Sheffield (EP/L024942). Further details of the project are available at http :/ /wordpress .csc .liv.ac .uk /va. We would like to kindlythankthereviewersfortheirdetailedandvaluablecommentsonourmanuscript.
References
[1]R.Alur,C.Courcoubetis,D.Dill,Model-checkingindensereal-time,Inf.Comput.104(1993)2–34.
[2]G.Behrmann,A.David,K.G.Larsen,AtutorialonUppaal,in:InternationalSchoolonFormalMethodsfortheDesignofComputer,Communication,and SoftwareSystems,RevisedLectures,in:Lect.NotesComput.Sci.,vol. 3185,Springer,2004,pp. 200–236.
[3]C.Bergenhem,Q.Huang,A.Benmimoun,T.Robinson,Challengesofplatooningonpublicmotorways,in:Proc.17thWorldCongressonIntelligent TransportSystems,2010,pp. 1–12.
[4]M.S.Branicky,Introductiontohybridsystems,in:HandbookofNetworkedandEmbeddedControlSystems,Birkhäuser,2005,pp. 91–116.
[5]J.Campbell,C.E.Tuncali,T.P.Pavlic,G.Fainekos,Modelingconcurrencyandreconfigurationinvehicularsystems:api-calculusapproach,in:Proc.9th InteractionandConcurrencyExperienceWorkshopofDicCoTec,ICE,2016.
[6]S.Colin,A.Lanoix,O.Kouchnarenko,J.Souquières,UsingCSPbcomponents:applicationtoaplatoonofvehicles,in:FormalMethodsforIndustrial CriticalSystems,Springer,2009,pp. 103–118.
[7]L.A.Dennis,GwendolenSemantics,Technicalreport,DepartmentofComputerScience,UniversityofLiverpool,UK,2017.
[8]L.A.Dennis,B.Farwer,Gwendolen:aBDIlanguageforverifiableagents,in:AISB’08WorkshoponLogicandtheSimulationofInteractionandReasoning, AISB,2008.
[9]L.A.Dennis,M.Fisher,N.Lincoln,A.Lisitsa,S.M.Veres,Reducingcodecomplexityinhybridcontrolsystems,in:Proc.10thInternationalSymposium onArtificialIntelligence,RoboticsandAutomationinSpace(i-Sairas),2010.
[10]L.A.Dennis,M.Fisher,N.K.Lincoln,A.Lisitsa,S.M.Veres,Declarativeabstractionsforagentbasedhybridcontrolsystems,in:DeclarativeAgent Lan-guagesandTechnologiesVIII,in:Lect.NotesComput.Sci.,vol. 6619,Springer,2011,pp. 96–111.
[11]L.A.Dennis,M.Fisher,N.K.Lincoln,A.Lisitsa,S.M.Veres,Practicalverificationofdecision-makinginagent-basedautonomoussystems,Autom.Softw. Eng.23 (3)(2016)305–359.
[12]L.A.Dennis,M.Fisher,M.P.Webster,R.H.Bordini,Modelcheckingagentprogramminglanguages,Autom.Softw.Eng.19 (1)(2012)5–63.
[13]E.Dolginova,N.A.Lynch,Safetyverificationforautomatedplatoonmaneuvers:acasestudy,in:Proc.InternationalWorkshoponHybridandReal-Time Systems,HART,1997,pp. 154–170.