Primitives. Ad Hoc Network. (a) User Applications Distributed Primitives. Routing Protocol. Ad Hoc Network. (b)

A Mutual Exclusion Algorithm for Ad Hoc Mobile Networks JenniferE.Walter  JenniferL.Welch  NitinH.Vaidya 

DepartmentofComputerScience,TexasA&MUniversity,CollegeStation,TX77843-3112

E-mail:[email protected],[email protected],[email protected]

Afault-tolerantdistributedmutualexclusionalgorithmthatadjuststonodemobilityispresented,alongwithproof

ofcorrectnessandsimulationresults. Thealgorithmrequiresnodestocommunicatewithonlytheircurrentneighbors,

makingitwell-suitedtotheadhocenvironment. Experimentalresultsindicatethatadaptationtomobilitycanimprove

performanceoverthatofsimilarnon-adaptivealgorithmswhennodesaremobile.

Keywords:Mobilecomputing,adhocnetwork,mutualexclusion,distributed algorithm.

1. Introduction

Amobileadhocnetworkisanetworkwhereinapair

ofnodescommunicatesbysendingmessageseitherover

a direct wireless link, or over a sequence of wireless

links including one or more intermediate nodes.

Di-rect communication is possible only between pairs of

nodesthatliewithinoneanother'stransmissionradius.

Wirelesslink\failures"occurwhenpreviously

commu-nicatingnodesmovesuchthattheyarenolongerwithin

transmissionrangeofeachother. Likewise,wirelesslink

\formation"occurs whennodesthat weretoofar

sepa-rated to communicatemovesuch that they arewithin

transmission rangeof each other. Characteristics that

distinguish ad hoc networks from existingdistributed

networks include frequent and unpredictable topology

changes and highly variable message delays. These

characteristicsmakeadhocnetworkschallenging

envi-ronmentsinwhichtoimplementdistributedalgorithms.

Past work on modifying existing distributed

algo-rithms for ad hoc networks includes numerous

rout-ing protocols (e.g., [8,9,11,13,16,18,19,22{24]),wireless

channelallocationalgorithms(e.g.,[14]),andprotocols

for broadcasting and multicasting (e.g., [8,12,21,26]).



SupportedbyGEFacultyoftheFutureandDept.of

Educa-tionGAANNfellowships. Phone:(979)845-3937. Fax:(979)

847-8578. Contactauthor.



SupportedinpartbyNSFPYIgrantCCR-9396098andNSF

grantCCR-9972235. Phone:(979)845-5076. Fax:(979)

847-8578.



Supportedin part byTexasAdvancedTechnology Program

grant010115-248 and NSFgrantsCDA-9529442 and

CCR-Dynamic networksarexed wirednetworks thatshare

some characteristics of ad hoc networks, since failure

and repairofnodes and linksis unpredictableinboth

cases. Researchondynamicnetworkshasfocusedon

to-tal ordering[17],end-to-end communication,and

rout-ing(e.g.,[1,2]).

Existingdistributedalgorithmswillruncorrectlyon

top of ad hoc routing protocols,since these protocols

are designed to hide the dynamic nature of the

net-work topologyfromhigherlayersintheprotocolstack

(see Figure 1(a)). Routing algorithmson ad hoc

net-works provide the ability to send messages from any

node to any other node. However, our contention is

that eÆciency canbe gainedby developing a coreset

ofdistributedalgorithms,orprimitives,thatareaware

oftheunderlyingmobilityinthe network,asshownin

Figure1(b). Inthispaper,wepresentamobilityaware

distributedmutualexclusionalgorithmtoillustratethe

layeringapproachinFigure1(b).

The mutual exclusion problem involves a group of

processes, each of whichintermittentlyrequiresaccess

toaresourceorapieceofcodecalledthecriticalsection

(CS).AtmostoneprocessmaybeintheCSatanygiven

time. Providingsharedaccesstoresourcesthrough

mu-tual exclusion is a fundamental problem in computer

science, and is worth consideringfor the ad hoc

envi-ronment,wherestripped-downmobilenodesmayneed

toshareresources.

Distributedmutualexclusionalgorithmsthatrelyon

themaintenanceofalogicalstructuretoprovideorder

User Applications

Distributed

Primitives

Routing

Protocol

Ad Hoc Network

(b)

Distributed

Primitives

Routing Protocol

Ad Hoc Network

User

Applications

(a)

Figure1. Twopossibleapproachesforimplementingdistributed

primitives.

inamobileenvironment,wherethetopologycan

poten-tiallychangewitheverynodemovement. Badrinathet

al.[3] solve this problem on cellular mobile networks,

wherethebulkofthecomputationcanberunonwired

portionsofthenetwork. Wepresentamutualexclusion

algorithmthatinduces alogicaldirectedacyclicgraph

(DAG)onthenetwork,dynamicallymodifyingthe

logi-calstructuretoadapttothechangingphysicaltopology

intheadhocenvironment. Wethenpresentsimulation

resultscomparingtheperformanceofthisalgorithmto

astaticdistributedmutualexclusionalgorithmrunning

ontopofanadhocroutingprotocol.Simulationresults

indicatethat ouralgorithmhasbetteraveragewaiting

time perCSentryandmessagecomplexityperCS

en-trynogreaterthanthecostincurredbyastaticmutual

exclusionalgorithmrunningontopofanadhocrouting

algorithmwhennodesaremobile.

Thenext sectiondiscussesrelated work. In Section

3, wedescribe oursystem assumptions and dene the

probleminmoredetail. Section4presentsourmutual

exclusionalgorithm. Wepresentaproofofcorrectness

and discussthe simulationresultsinSections 5and6,

respectively. Section7presentsourconclusions.

2. Related Work

Tokenbasedmutualexclusionalgorithmsprovide

ac-cesstotheCSthroughthemaintenanceofasingletoken

thatcannotsimultaneouslybepresentatmorethanone

nodeinthesystem. RequestsforCSentryaretypically

directedto whichevernodeisthecurrenttokenholder.

Raymond[25] introduced atokenbased mutual

ex-clusion algorithm in which requests are sent, over a

static spanning tree of the network, towardthe token

crashesand recoveries,but is not resilientto link

fail-ures. Chang et al.[7]extend Raymond'salgorithm by

imposing a logical directionon a suÆcient numberof

linksto induceatoken oriented DAGinwhich, for

ev-erynodei,there existsadirectedpathoriginatingati

andterminatingatthetokenholder. Allowingrequest

messagesto besentoveralllinksoftheDAGprovides

resilience to link and site failures. However, this

al-gorithm does not consider link recovery, an essential

featureinasystemof mobilenodes.

Dhamdhere and Kulkarni [10] show that the

algo-rithm of [7] can suer from deadlock and solve this

problembyassigningadynamicallychangingsequence

numbertoeachnode,formingatotalorderingofnodes

inthesystem. Thetokenholderalwayshasthehighest

sequencenumber, and,by deninglinksto pointfrom

anodewith lowerto highersequence number,atoken

oriented DAG is maintained. Due to link failures, a

nodei that wantsto sendarequest forthetokenmay

nd itselfwith no outgoing linksto the token holder.

In this situation, i oods the network with messages

to build a temporary spanning tree. Once the token

holderbecomespartofsuch aspanningtree,thetoken

is passed directly to node i along the tree, bypassing

otherrequests. Sincepriorityisgiventonodesthatlose

apathtothetokenholder,itseemslikelythatother

re-questingnodescouldbestarvedaslongaslinkfailures

continue. Also, oodinginresponsetolinkfailuresand

storing messagesfor deliveryafter linkrecoverymake

this algorithmill-suitedto the highlydynamicad hoc

environment.

Ourtokenbasedalgorithmcombinesideasfrom

sev-eral papers. The partialreversal techniquefrom [13],

usedtomaintainadestinationorientedDAGinapacket

radionetworkwhenthedestinationisstatic,isusedin

ouralgorithmto maintainatokenorientedDAGwith

adynamicdestination. Likethealgorithmsof[25],[7],

and [10], each node in our algorithm maintains a

re-quest queue containing the identiers of neighboring

nodes from which it has received requests for the

to-ken. Like[10],ouralgorithmtotallyordersnodes. The

lowest node is always the current token holder,

mak-ingita\sink"towardwhichallrequestsaresent. Our

algorithmalso includessomenewfeatures. Eachnode

dynamicallychoosesitslowestneighborasitspreferred

linkto thetoken holder. Nodes sense link changes to

statusofthepreviouspreferredlinktothetokenholder

andthecurrentcontentsofthelocalrequestqueue. All

requestsreachingthetokenholderaretreated

symmet-rically, so that requests are continually serviced while

theDAGis beingre-orientedandblockedrequestsare

beingrerouted.

3.Denitions

The systemcontains aset of n independent mobile

nodes, communicatingbymessagepassingovera

wire-less network. Each mobile node runs an application

processandamutualexclusionprocessthat

communi-catewitheachothertoensurethat thenodecycles

be-tween itsREMAINDER section (not interestedinthe

CS), its WAITING section (waiting for access to the

CS), anditsCRITICALsection. Assumptions 1

onthe

mobilenodesandnetworkare:

1. thenodeshaveuniquenodeidentiers,

2. nodefailuresdonotoccur,

3. communicationlinksarebidirectionalandFIFO,

4. alink-levelprotocolensuresthateachnodeisaware

ofthe set ofnodeswith which it cancurrently

di-rectlycommunicatebyprovidingindicationsoflink

formationsandfailures,

5. incipientlinkfailuresaredetectable,providing

reli-ablecommunicationonaper-hopbasis,and

6. partitionsofthenetwork donotoccur.

The rest of this section contains our formal

deni-tions. We explicitly model only the mutual exclusion

process at each node. Constraintson the behavior of

the application processes and the network appear as

conditions on executions. The system architecture is

showninFigure2.

We assume the node identiers are 0;1;:::;n 1.

Each node has a (mutual exclusion)process, modeled

asastatemachine,withtheusualsetofstates,someof

whichareinitialstates,andatransitionfunction. Each

statecontainsalocalvariablethatholdsthenode

iden-tierandalocalvariablethat holdsthecurrent

neigh-bors of thenode. The transitionfunctionis described

inmoredetailshortly.

1

Application Process

Mutual Exclusion Process

Network

node i

ReleaseCS

RequestCS

Recv(m)

LinkUp

Send(m)

LinkDown

EnterCS

Figure2.Systemarchitecture.

Acongurationdescribestheinstantaneousstateof

thewholesystem; formally,it is aset of nstates, one

foreach process. Inaninitialconguration,eachstate

isaninitialstateandtheneighborvariablesdescribea

connectedundirectedgraph.

A step of the process at node i is triggered by the

occurrenceofaninputevent. Inputeventsare:

 RequestCS

i

: the application process on node i

re-quests accesstotheCS,enteringitsWAITING

sec-tion.

 ReleaseCS

i

: the application process on node i

re-leases its access to the CS, entering its

REMAIN-DERsection.

 Recv

i

(j;m): nodeireceivesmessagemfromnodej.

 LinkUp

i

(l): nodeireceivesnoticationthatthelink

lincidentoniisnowup.

 LinkDown

i

(l): node i receivesnoticationthat the

linklincidentoniisnowdown.

Theeect of astep isto apply theprocess' transition

function, taking asinput thecurrent stateof the

pro-cess and the input event, and producing as output a

(possibly empty)set ofoutput eventsandanew state

fortheprocess. Output eventsare:

 EnterCS

i

: the mutual exclusion process on node i

informsitsapplicationprocessthat itcanenter the

CRITICALsection.

 Send

i

(j;m): nodeisendsmessagemtonodej.

Theonlyconstraintonthestateproducedbythe

tran-sition function is that the neighbor set variable of i

RequestCS i , EnterCS i , and ReleaseCS i are called

application events, while Send

i , Recv i , LinkUp i , and LinkDown i

arecallednetworkevents.

AnexecutionisasequenceoftheformC

0 ,in 1 ,out 1 , C 1 ,in 2 ,out 2 ,C 2 ;:::,wheretheC k 'sarecongurations, thein k

'sareinputevents,andtheout

k

'saresetsof

out-put events. An execution must end inaconguration

ifitisnite. A positiverealnumberisassociatedwith

eachin

i

,representingthetimeat whichthat event

oc-curs. Anexecutionmustsatisfyanumberofadditional

conditions,whichwenowlist. Therstsetofconditions

arebasic\syntactic"ones.

 C

0

isaninitialconguration.

 If in

k

occurs at node i, then out

k

and i's state in

C

k

are correct according to i's transition function

operatingonin

k

andi'sstateinC

k 1 .

 Thetimesassignedtothestepsmustbe

nondecreas-ing. Iftheexecutionisinnite,thenthetimesmust

increasewithout bound. Atmost onestep byeach

processcanoccurat agiventime.

Thenextset ofconditionsrequiretheapplication

pro-cesstointeractproperlywiththemutualexclusion

pro-cessandto giveuptheCSinnitetime.

 Ifin

k

isRequestCS

i

, then theprevious application

eventatnodei(ifany)isReleaseCS

i .  If in k is ReleaseCS i

, then the previous application

eventatnodeimustbeEnterCS

i .  Ifout k includesEnterCS i

, then there is afollowing

ReleaseCS

i .

The remainingconditions constrainthe behaviorof

the network to match the informal description given

above. First,weconsiderthemobilitynotication.

 LinkUp

i

(l)occursattimetifandonlyifLinkUp

j (l)

occursattimet,whereljoinsiandj. Furthermore,

LinkUp

i

(l)onlyoccurs ifj is currentlynota

neigh-bor of i (according to i's neighbor variable). The

analogousconditionholdsforLinkDown.

 ALinkDownneverdisconnectsthegraph.

Finally, we considermessage delivery. There must

existaone-to-oneandontocorrespondencebetweenthe

occurrencesofSend

j

(i;m)and Recv

i

(j;m),for alli, j

and m. This requirementimplies that everymessage

corrupt messages nor deliver spuriousmessages. F

ur-thermore, the correspondence must satisfy the

follow-ing:

 If Send

i

(j;m) occurs at some time t, then the

cor-respondingRecv

j

(i;m)occursatsomelatertimet 0

,

andthelinkconnectingiandjiscontinuouslyup

be-tweentandt 0

. ThisimpliesthataLinkDownevent

forlinklcannotoccurifanymessagesareintransit

onl.

Now we can state the problem formally. In every

execution,thefollowingmusthold:

 If out

k

includesEnterCS

i

, then the previous

appli-cationeventatnodeimustbeRequestCS

i

. I.e.,CS

accessisonlygivento requestingnodes.

 Mutual Exclusion: If out

k

includes EnterCS

i , then

anypreviousEnterCS

j

eventmustbefollowedbya

ReleaseCS

j

priortoout

k .

 No Starvation: If there are only a nite number

of LinkUp i and LinkDown i events, then if in k is RequestCS i

, thenthereisafollowingEnterCS

i .

Forthelastcondition,thehypothesisthatlinkchanges

cease is needed because an adversarialpattern of link

changescancausestarvation.

4.Reverse Link(RL)Mutual Exclusion

Algorithm

In this section we rst present the data structures

maintainedateachnodeinthesystem,followedbyan

overview of the algorithm, the algorithmpseudocode,

andexamplesofalgorithmoperation. Throughoutthis

section, data structures are described for node i, 0 

in 1. Subscriptsondatastructurestoindicatethe

nodeareonlyincludedwhenneeded.

4.1. DataStructures

 status: Indicates whether node is in the WAITING,

CRITICAL, orREMAINDER section. Initially,

sta-tus=REMAINDER.

 N:Thesetofallnodesindirectwirelesscontactwith

nodei. Initially,N containsallofnodei'sneighbors.

 myHeight: A three-tuple (h1,h2,i) representing the

heightofnodei. Linksareconsideredto bedirected

ifmyHeight 1 = (2, 3, 1) and myHeight 2 =(2, 2,2), then myHeight 1 > myHeight 2

and the link between

these nodes would be directedfrom node 1 to node

2. Initiallyatnode0,myHeight

0

=(0,0,0) and,for

alli6=0,myHeight

i

isinitializedsothatthedirected

linksformaDAGinwhicheverynodehasadirected

pathto node0.

 height[j]:Anarrayoftuplesrepresentingnodei'sview

of myHeight j for all j 2 N i . Initially, height[j] = myHeight j , for all j 2 N i

. In node i's viewpoint, if

j 2 N, then the link between i and j is incoming

tonode iifheight[j]>myHeight,andoutgoingfrom

nodeiifheight[j]<myHeight.

 tokenHolder: Flagsettotrueifnodeholdstokenand

settofalseotherwise. Initially,tokenHolder=trueif

i=0,andtokenHolder=falseotherwise.

 next:Whennodeiholdsthetoken,next=i,otherwise

nextisthenodeonanoutgoinglink. Initially,next=

0ifi=0,andnextisanoutgoingneighborotherwise.

 Q: Queue containing identiers of requesting

neigh-bors. OperationsonQincludeEnqueue(),which

en-queues an item only if it is not already on Q,

De-queue()withtheusualFIFOsemantics,andDelete(),

which removesaspecieditemfromQ,regardlessof

itslocation. Initially,Q=;.

 receivedLI[j]:Booleanarrayindicatingwhether

Link-Infomessagehasbeenreceivedfromnodej,towhich

a Token message was recently sent. Any height

in-formationreceivedat nodeifromanodej forwhich

receivedLI[j]isfalsewillnotberecordedinheight[j].

Initially,receivedLI

i

[j]=trueforallj2N

i .

 forming[j]: Boolean array set to true when link to

nodejhasbeendetectedasformingandresettofalse

whenrstLinkInfomessagearrivesfromnodej.

Ini-tially,forming

i

[j]=falseforallj2N

i .

 formHeight[j]:Anarrayoftuplesstoringvalueof

my-Height when new link to j rst detected. Initially,

formHeight i [j] =myHeight i forallj2N i .

4.2. Overviewof theRL Algorithm

Themutualexclusionalgorithmisevent-driven. An

eventat a nodei consists of receivingamessagefrom

another node j 6=i, oranindication of link failureor

formationfromthelinklayer,oraninputfromthe

ap-plicationon node i torequest orreleasethe CS. Each

thesender. Modulesareassumedtobeexecuted

atom-ically.

Thepseudocode triggeredby input events fromthe

applicationprocessisshowninFigure3.

When node i requests access to the CS:

1. status := WAITING

2. Enqueue(Q;i)

3. if (not tokenHolder)

4. if (jQj = 1) ForwardRequest()

5. else GiveTokenToNext()

When node i releases the CS:

1. if (jQj>0) GiveTokenToNext()

2. status := REMAINDER

Figure3.Pseudocodetriggeredbyinputeventsfromapplication

process.

Requesting and releasing the CS: When node i

re-quests access to theCS, it enqueues its own identier

onQ andsets status toWAITING.If node i doesnot

currentlyhold thetokenand ihasasingleelementon

its queue, it calls ForwardRequest() to send aRequest

message. If node i doeshold the token, i canset

sta-tusto CRITICALand enter theCS,sinceitwillbeat

the head of Q. When node i releasesthe CS, it calls

GiveTokenToNext() to send a Token message if Q is

non-empty,andsets statusto REMAINDER.

Thepseudocodetriggeredbynetworkinputeventsis

showninFigures4and5.

Requestmessages: WhenaRequestmessagesentby

aneighboringnodej isreceivedatnodei,iignoresthe

Request if receivedLI[j] is false. Otherwise, i changes

height[j],andenqueuesjonQifthelinkbetweeniand

j is incoming at i. If Q is non-empty, and status =

REMAINDER, i calls GiveTokenToNext(), provided i

holdsthetoken. Non-tokenholdingnodeicalls

Raise-Height() if thelinkto j is now incomingandi has no

outgoinglinksori callsForwardRequest()ifQ=[j]or

ifQisnon-emptyandthelinkto nexthasreversed.

Tokenmessages: WhennodeireceivesaToken

mes-sagefromsomeneighborj, i sets tokenHolder= true.

Thenilowersitsheighttobelowerthanthatofthelast

tokenholder,nodej,informsallitsoutgoingneighbors

of its new height by sending LinkInfo messages, and

callsGiveTokenToNext(). Node i also informs j of its

newheightsothatjwillknowthatireceivedthetoken.

When Request(h) received at node i from node j:

// h denotes j's height when message was sent

1. if (receivedLI[j])

2. height[j] := h

// set i's view of j's height

3. if (myHeight < height[j]) Enqueue(Q;j)

4. if (tokenHolder)

5. if ((jQj>0) and (status = REMAINDER))

6. GiveTokenToNext()

7. else // not tokenHolder

8. if (myHeight < height[k], 8 k2N)

9. RaiseHeight()

10. else if ((Q= [j]) or ((jQj>0)

and (myHeight < height[next])))

11. ForwardRequest() // reroute request

When Token(h)received at node i from node j:

// h denotes j's height when message was sent

1. tokenHolder := true

2. height[j] := h

3. Send LinkInfo(h.h1, h.h2 1;i)

to all outgoing k2N and to j

4. myHeight.h1 := h.h1

5. myHeight.h2 := h.h2 - 1 // lower my height

6. if (jQj>0) GiveTokenToNext()

7. else next := i

When LinkInfo(h) received at node i from node j:

// h denotes j's height when message was sent

1. N := N[fjg

2. if ((forming[j]) and (myHeight 6= formHeight[j]))

3. Send LinkInfo(myHeight) to j

4. forming[j] := false

5. if (receivedLI[j]) height[j] := h

6. else if (height[j] = h) receivedLI[j] := true

7. if (myHeight > height[j]) Delete(Q;j)

8. if ((myHeight < height[k], 8k2N)

and (not tokenHolder))

9. RaiseHeight()

10. else if ((jQj>0) and (myHeight < height[next]))

11. ForwardRequest() // reroute request

Figure4. PseudocodetriggeredbyRecvnetworkinputevents.

j's heightissavedinheight[j]. If receivedLI[j]is false,

i checks if the height of j in the message is what it

waswhen i sentthe Tokenmessage to j. If so,i sets

receivedLI[j] to true. If forming[j] is true,the current

valueofmyHeightiscomparedtothevalueofmyHeight

whenthelinktoj wasrstdetected, formHeight[j]. If

myHeight andformHeight[j]aredierent,thena

Link-Infomessageissenttoj. IdentierjisaddedtoN and

forming[j]is setto false. Ifj isanelementofQ andj

isanoutgoinglink,then j isdeletedfromQ. Ifnodei

RaiseHeight() sothat anoutgoinglinkwillbeformed.

Otherwise,ifQis non-empty,and thelinktonext has

reversed, i calls ForwardRequest() since it must send

anotherRequestforthetoken.

When failure of link to j detected at node i:

1. N := N fjg

2. Delete(Q;j)

3. receivedLI[j] :=true

4. if (not tokenHolder)

5. if (myHeight < height[k], 8k2N)

6. RaiseHeight() // reroute request

7. else if ((jQj>0) and (next 62N))

8. ForwardRequest()

When formation of link to j detected at node i:

1. Send LinkInfo(myHeight) to j

2. forming[j] := true

3. formHeight[j] := myHeight

Figure5. Pseudocodetriggered byLinkDown andLinkUp

net-workinputevents.

Link failures: When node i senses the failure of a

linktoaneighboringnodej,itremovesj fromN,sets

receivedLI[j]totrue,and,ifjisanelementofQ,deletes

j fromQ. Then, ifi isnotthetokenholder andi has

nooutgoinglinks,icallsRaiseHeight(). Ifnodeiisnot

thetokenholder,Qisnon-empty, andthelinktonext

hasfailed, i callsForwardRequest() sinceit mustsend

anotherRequestforthetoken.

Link formation: Whennodei detectsanewlinkto

nodej,isendsaLinkInfomessagetojwithmyHeight,

sets forming[j] to true, and sets formHeight[j] =

my-Height.

Thepseudocode for theprocedures of the RL

algo-rithmisshowninFigure6.

Procedure ForwardRequest: Selects node i's lowest

heightneighbortobenext. SendsaRequestmessageto

next.

Procedure GiveTokenToNext: Node i dequeues the

rstnodeonQandsetsnextequaltothisvalue. Ifnext

=i,ienterstheCS.Ifnext6=i,ilowersheight[next]to

(myHeight.h1, myHeight.h2 1; next), so any incoming

Requestmessageswillbesenttonext,setstokenHolder

=false,setsreceivedLI[next]to false,andthen sendsa

Tokenmessagetonext. IfQisnon-emptyaftersending

aToken messageto next,aRequest messageissentto

next immediatelyfollowing the Token message so the

Procedure ForwardRequest():

1. next := l2N : height[l ]  height[j], 8 j2N

2. Send Request(myHeight) to next

Procedure GiveTokenToNext(): // called when jQj>0

1. next := Dequeue(Q)

2. if (next 6=i)

3. tokenHolder := false

4. height[next]:=(myHeight.h1,myHeight.h2 1,next)

5. receivedLI[next] := false

6. Send Token(myHeight) to next

7. if (jQj>0) Send Request(myHeight) to next

8. else // next = i 9. status := CRITICAL 10. Enter CS Procedure RaiseHeight(): 1. myHeight.h1:=1+min k 2N fheight[k].h1g 2. S:=fl2N : height[l ].h1=myHeight.h1g 3. if (S6=;) myHeight.h2 := min l2S fheight[l ].h2g 1

4. Send LinkInfo(myHeight) to all k2N

// raising own height can cause links to be outgoing

5. for (all k2N such that myHeight > height[k])

6. Delete(Q;k)

// reroute request if queue non-empty,

// just had no outgoing links

7. if (jQj>0) ForwardRequest()

Figure6.Pseudocodeforprocedures.

nodeiwhenilosesitslastoutgoinglink. Nodeiraises

itsheight(inlines1-3)usingthepartialreversalmethod

of[13]andinformsallitsneighborsofitsheightchange

withLinkInfo messages. AllnodesonQtowhichlinks

arenowoutgoingaredeletedfromQ. IfQisnotempty

at this point, ForwardRequest() is called since i must

sendanotherRequestforthetoken.

4.3. Examples ofAlgorithmOperation

Werstdiscussthecaseofastaticnetwork,followed

byadynamicnetwork.Anillustrationofthealgorithm

onastaticnetwork(inwhichlinksdonotfailorform)is

depictedinFigure7. Snapshotsofthesystem

congu-rationduringalgorithmexecutionareshown,withtime

increasingfrom7(a)to 7(e). Thedirect wirelesslinks

are shown as dashed lines connecting circular nodes.

Thearrowoneachwirelesslinkpointsfromthehigher

heightnodetothelowerheightnode. Therequestqueue

at each node is depicted as a rectangle, the height is

shown asa 3-tuple, and the token holder asa shaded

Notethatwhenanodeholdsthetoken,itsnextpointer

isdirectedtowardsitself.

2

(0, 0, 0)

(0, 2, 2)

2

1

3

1

(0, 2, 1)

(0, 1, 3)

3

1

2

3

0

(b)

(0, 0, 0)

2

(0, 2, 2)

2

1

1

(0, 2, 1)

2

0

3

1

0

(0, −1, 3)

(c)

(0, 0, 0)

2

(0, 2, 2)

2

2

(0, −1, 3)

0

(0, −2, 1)

0

3

1

3

(d)

1

0

(0, −2, 1)

(0, −3, 3)

3

2

(0, −4, 0)

(0, −5, 2)

(e)

2

3

(0, 0, 0)

(0, 2, 1)

(0, 2, 2)

(0, 1, 3)

0

2

1

3

2

3

(a)

Figure7. Operationof reverselinkmutualexclusion algorithm

onstaticnetwork.

InFigure7(a),nodes2and 3haverequestedaccess

totheCS(notethatnodes2and3haveenqueued

them-selveson Q

2 and Q

3

)and havesentRequestmessages

to node 0, which enqueued them on Q

0

in the order

inwhich theRequestmessageswerereceived. Part(b)

depicts the system at a later time, where node 1 has

requestedaccesstotheCS,andhassentaRequest

mes-sagetonode3(notethat1isenqueuedonQ

1 andQ

3 ).

Figure 7(c)showsthesystem congurationafter node

0 has released the CS and has sent a Token message

tonode3,followedbyaRequestsentbynode0on

be-halfofnode2. Observethatthelogicaldirectionofthe

linkbetweennode0andnode3changesfrombeing

di-rectedawayfromnode3inpart(b),to beingdirected

towardnode3inpart(c),whennode3receivestheT

o-kenmessageandlowersitsheight. Noticealsothenext

pointersofnodes0and3changefrombothnodes

hav-ingnextpointersdirectedtowardnode0inpart(b)to

bothnodes having nextpointers directedtowardnode

3inpart(c). Part(d)showsthesystemconguration

after node3sentaTokenmessageto node1, followed

by aRequestmessage. The Request messagewassent

becausenode3receivedtheRequestmessagefromnode

0. Notice that theitems at the headof thenodes'

Part (e) depicts the system conguration after Token

messageshavebeenpassedfromnode1to3,node3to

0,and fromnode0to 2. Observethat themiddle

ele-ment,h2,ofeachnode'smyHeighttupledecreasesby1

foreveryhopthetokentravels,sothatthetokenholder

isalwaysthelowestheightnodeinthesystem.

WenowconsidertheexecutionoftheRL algorithm

on adynamicnetwork. The heightinformationallows

each node i to keep track of the current logical

direc-tion of links to neighboringnodes, particularly to the

node chosen to be next. If the link to next changes

andjQj>0,nodei mustrerouteits requestby calling

ForwardRequest().

(0, 1, 3)

3

3

2

(0, 0, 0)

(0, 2, 1)

(0, 2, 2)

0

2

1

2

(b)

2

(0, 0, 0)

(0, 2, 1)

(0, 2, 2)

0

2

1

2

3

3

(1, 1, 3)

(c)

2

(0, 0, 0)

(0, 2, 2)

0

2

1

2

3

3

(1, 1, 3)

(1, 0, 1)

3

1

(e)

2

(0, 0, 0)

(0, 2, 2)

0

2

1

2

3

3

(1, 1, 3)

(1, 0, 1)

3

(d)

2

3

(0, 0, 0)

(0, 2, 1)

(0, 2, 2)

(0, 1, 3)

₀

2

1

3

2

3

(a)

Figure8. Operationofreverselinkmutualexclusionalgorithm

ondynamicnetwork.

Figure 8(a)showsthe samesnapshotof thesystem

executionasisshowninFigure7(a),withtime

increas-ing from 8(a)to 8(e). Figure 8(b) depictsthe system

congurationafternode3hasmovedinrelationtothe

othernodesinthesystem,resultinginanetworkthatis

temporarilynottokenoriented,sincenode3hasno

out-goinglinks. Node0hasadaptedtothelostlinktonode

3by removing3fromitsrequest queue. Node 2takes

no action as a result of the lossof its linkto node 3,

sincethelinktonext

2

wasnotaectedandnode2still

hasoneoutgoinglink. Inpart(c), node3hasadapted

tothelossofitslinktonode0byraisingitsheightand

hassentaRequestmessageto node1(thathasnotyet

arrived at node 1). Part (d) shows thesystem

cong-urationafter node1has receivedthe Requestmessage

fromnode3,hasenqueued3onQ

1

,andhasraisedits

heightdue tothelossofits lastoutgoinglink. Inpart

(e), node 1 has propagated the Request received from

node3by sendingaRequesttonode2,also informing

node2ofthechangeinitsheight. Node2subsequently

enqueued1on Q

2

, but didnot raiseits ownheightor

send a Request, because node 2 has an intact link to

next

2

, node 0, to which it already sent an unfullled

request.

5.Correctness of ReverseLink Algorithm

The following theorem holds because there is only

onetokeninthesystematanytime.

Theorem1. Thealgorithmensuresmutualexclusion.

Toprovenostarvation,werstshowthat,afterlink

changes cease,eventuallythe systemreachesa\good"

conguration,andthenweapplyavariantfunction

ar-gument.

Wewillshowthat after linkchangescease,the

log-ical directionson the links imparted by height values

willeventuallyforma\tokenoriented"DAG.Sincethe

heightvaluesofthenodesaretotallyordered,there

can-notbe anycyclesinthelogicalgraph,and thus it isa

DAG.ThehardpartisshowingthatthisDAGistoken

oriented,denednext.

Denition1. Anodeiisthetokenholderina

cong-urationiftokenHolder

i

=trueorifaTokenmessageis

intransitfromnodeitonext

i .

Denition2. TheDAGistokenorientedina

congu-rationifforeverynodei;i2f0;:::;n 1g,thereexists

adirected path originatingat node i and terminating

atthetokenholder.

ToproveLemma3,thattheDAGiseventuallytoken

oriented,werstshow,inLemma1,thatthiscondition

isequivalenttotheabsenceof\sink"nodes[13],as

de-nedbelow. Wethenshow,inLemma2,thateventually

thereare nomorecallsto RaiseHeight(). Throughout,

weassumethat eventuallylinkchangescease.

Denition3. A node i is asink inaconguration if

(tokenHolder

i

= false) and ((myHeight

i

< height

i [j]),

Lemma1. In everyconguration of everyexecution,

the DAG is token oriented ifand only if there are no

sinks.

Proof: Theonly-ifdirectionfollowsfromthedenition

ofatokenorientedDAG.Theifdirectionisprovedby

contradiction. Assume,incontradiction,thatthere

ex-istsanodeiinacongurationsuchthattokenHolder

i =

false and for which there is no directed path starting

at i and ending at the token holder. Since there are

no sinks, i must have at least one outgoing link that

is incomingat someother node. Since thenumber of

nodes is nite, the network is connected, and alllinks

arelogicallydirectedsuchthatnologicalpathcanform

acycle, there mustexistadirectedpathfromito the

tokenholder,acontradiction.

Toshowthateventuallythere arenosinks (Lemma

3),weshowthat thereareonlyanitenumberofcalls

to RaiseHeight().

Lemma2. Ineveryexecutionwith anitenumberof

link changes, there exists a nite number of calls to

RaiseHeight().

Proof: Incontradiction,consideranexecutionwitha

nitenumberoflinkchangesbutaninnitenumberof

callsto RaiseHeight(). Then, after linkchangescease,

somenodecallsRaiseHeight() innitelyoften. Werst

notethatifonenodecallsRaiseHeight()innitelyoften,

theneverynodecallsRaiseHeight() innitelyoften. To

seethis,considerthatanodeiwouldcallRaiseHeight()

innitely often only if it lost all its outgoing links

in-nitely often. But this would happen innitely often

at nodeionlyifaneighboringnodej raiseditsheight

innitelyoften,andneighboringnodej wouldonlycall

RaiseHeight() innitelyoftenifitsneighborkraisedits

height innitely often, and so on. However, Claim1

showsthatat leastonenodecallsRaiseHeight() onlya

nitenumberoftimes.

Claim 1. No node that holdsthe tokenafter thelast

linkchangeevercallsRaiseHeight() subsequently.

Proof: Supposetheclaimisfalse,andsomenodethat

holds the token after the lastlink change calls

Raise-Height() subsequently. Letibetherstnodeto doso.

By the code, node i does not hold the token when it

ken to neighboringnode j at timet

1

, setting its view

ofjtobeoutgoing,andatalatertime,t

3

,nodeicalls

RaiseHeight(). ThereasonicallsRaiseHeight() attime

t

3

is that it lostits last outgoing link. Thus, at time

t 2 betweentimet 1 andt 3

,thelinkbetweeniandj has

reversed directionini's viewfromoutgoingto

incom-ing. Bythecode,the directionchangeat node imust

bedue tothereceiptof aLinkInfo orRequestmessage

fromnodej. Wediscussthesecasesseparatelybelow.

Case 1: The direction change at node i is due to the

receipt of a LinkInfo message from node j at time t

2 .

By the code, when i sends the token to j at t

1 , it

sets receivedLI[j] to false. Therefore, when the

Link-Info messageisreceivedat i fromj at time t

2

, nodei

musthavealreadyresetreceivedLI[j]totrueoriwould

still see the link to j as outgoing and would not call

RaiseHeight() at time t

2

. Since i called RaiseHeight()

afterreceivingtheLinkInfo messagefromj attimet

2 ,

imusthavereceivedtheLinkInfo messagenodej sent

when it received the token from i before time t

2 , by

theFIFO assumptiononmessagedelivery. Thennode

j musthavereceivedthetoken and sent itto another

node, k 6= i, after which j raised its height and sent

the LinkInfo message that node i received at time t

2 .

However,thisviolatesourassumptionthatiistherst

nodeto callRaiseHeight() afterthe lastlinkchange, a

contradiction.

Case 2: The direction change at node i is due to the

receipt of a Request message from node j at time t

2 .

Byasimilarargumentto case1, any Requestreceived

from node j would be ignored at node i as long as

receivedLI[j]is false. Butthismeansthat nodej must

have called RaiseHeight() after it received the token

fromnodeiandsubsequentlysenttheRequestreceived

by i at time t

2

. Again, this violates the assumption

that i is the rst node to call RaiseHeight() after the

lastlinkchange,acontradiction.

Therefore,nodeiwillnotcallRaiseHeight()attime

t

2

andtheclaimis true.

Therefore,byClaim1,thereisonlyanitenumber

ofcallsto RaiseHeight() inanyexecutionwithanite

numberoflinkchanges.

Lemma3followsfromLemma2,sinceifanode

be-comesasink,itwilleventuallybeinformedviaLinkInfo

Lemma3. Once linkchangescease,thelogical

direc-tiononlinksimpartedbyheightvalueswilleventually

alwaysformatokenorientedDAG.

ConsideranodethatisWAITINGinanexecutionat

somepointafterlinkchangesandcallstoRaiseHeight()

have ceased. We rst dene the \request chain" of a

nodetobethepathalongwhichitsrequesthas

propa-gated. Then wemodify thevariantfunctionargument

in [25] to showthat thenodeeventually gets to enter

theCS.

Denition4. Given a conguration, a request chain

for anynode lwith a non-emptyrequest queue is the

maximallengthlistofnodeidentiersp

1 =l;p 2 ;:::;p j ,

whereforeachi,1<ij,

 p

i

'squeueisnotempty,

 p i =next p i 1 ,

 thelinkbetweenp

i 1 andp i isoutgoingatp i 1 and incomingat p i ,

 noRequestmessageisintransitfromp

i 1 top

i ,and

 noTokenmessageisintransitfromp

i top

i 1 .

Lemma4givesusefulinformationaboutwhatis

go-ingonattheendofarequestchain:

Lemma4. The following is true in every

congura-tion: Letlbeanode withanon-empty requestqueue

andletp 1 =l;p 2 ;:::;p j

bel'srequestchain. Then

(a) lisinQ l ilisWAITING, (b) p i 1 isinQ pi ;1<ij,and (c) eitherp j

isthetokenholder,

oraTokenmessageisintransittop

j ,

oraRequestmessageisintransitfromp

j tonext

p

j ,

oraLinkInfomessageisintransitfromnext

pj top j withnext p j higherthanp j , ornext p j

seesthelinktop

j

as failed.

Proof: Byinductionontheexecution.

Property (a) can easily be shown to hold, since a

node enqueues its own identier when its application

requests access to the CS, at which point it changes

its status to WAITING.Bythe code, at no point will

a node dequeue its own identier until just before it

Properties(b)and(c)arevacuouslytrueintheinitial

conguration,sincenonodehasanon-emptyqueue.

Suppose (b)and(c) aretrueinthe(t 1) st

cong-uration,C

t 1

, ofthe execution. It ispossibleto show

thesepropertiesaretrueinthet th

conguration,C

t ,by

consideringin turn every possibilityfor the t th

event.

MostoftheeventsappliedtoC

t 1

areeasilyshownto

yieldacongurationC

t

inwhichproperties(b)and(c)

aretrue. Herewediscusstheeventsforwhichthe

out-come is less clear by presenting the problematiccases

that can appear to disrupt a request chain. We note

that, in the following cases, non-token holding nodes

areoften requiredto ndan outgoinglinkdue to link

reversalsorfailures.Itisnothardtoshowthatanodei

thatisnotthetokenholdercanalwaysndanoutgoing

linkdueto theperformanceofRaiseHeight().

Case1: Node ireceivesaRequest(h)fromnode j and

does not enqueue j on its request queue. To ensure

thatj'sRequestisnotoverlooked,causingpossible

star-vation,weshowthateitheraLinkInfooraToken

mes-sageis sent to j fromi if aRequest fromj is received

atiandj isnotenqueued.

Case1.1: receivedLI[j] isfalse at i. It mustbethat i

sentthe tokento j insome previousconguration

andihasnotyetreceivedtheLinkInfomessagethat

j mustsend to i upon receiptof the token. If the

token is not in transitfromi to j or heldby j in

C

t 1

, then earlier j had the token and passed it

on. TheRequest receivedbyi wassentbefore the

LinkInfomessagethatjmustsendtoiuponreceipt

of thetoken. So if j is WAITING inC

t 1 , it has

alreadysentanewerRequestandproperties(b)and

(c)holdforthisrequestchaininC

t

bytheinductive

hypothesis.

Case1.2: receivedLI[j] is true at i. Then if j is not

enqueued on i's request queue, it must be that

myHeight

i

> h. Since j viewed i as outgoing

when it sent the Request, node i must haveeither

calledRaiseHeight() after j wasinN

i

or the

rela-tive heights of i and j changed between the time

link(i;j)wasrstdetectedandbeforejwasadded

toN

i

. Ineithercase,nodejmusteventuallyreceive

aLinkinfo message fromi and see that its linkto

next

j

hasreversed,inwhichcasej willtakeaction

Case 2: Node i receives an input causing it to delete

identierj from itsrequestqueue. Toensurethat j's

Request is not forgotten when i calls Delete(Q;j), we

showthateithernodej receivedaTokenmessageprior

tothedeletion,inwhichcasej'sRequestissatised,or

nodejisnotiedthatthelinktoifailed,inwhichcase

jwilltaketheappropriateactiontoreroutetherequest

chain.

Case 2.1: Nodei callsDelete(Q;j)becauseitreceives

aLinkInfomessagefromjindicatingthati'slinkto

jhasbecomeoutgoingati. Then,sinceienqueued

j,itmustbethatinsomeearliercongurationisaw

thelinkto j asincoming. Since thereceipt of the

LinkInfo messagefromj caused thelinktochange

from incomingto outgoing ini's view, it must be

thattheLinkInfowassentbyj whenjreceivedthe

tokenandlowereditsheight. Ifthetokenisnotheld

byjinC

t 1

,thenearlierjhadthetokenandpassed

iton. IfjisWAITINGinC

t 1

,ithasalreadysent

anewerRequestandproperties(b)and(c)holdfor

thisrequestchaininC

t

bytheinductivehypothesis.

Case 2.2: NodeicallsDelete(Q;j)becauseitreceived

an indication that link (i;j) failed. Then j must

receivethesameindication,inwhichcaseitcantake

appropriateactionto advanceanyrequestchains.

Case 3: Node i receives an input which makes it see

the link to next

i

as incoming or failed. In this case,

anyrequestchainsincludingnodeiinC

t 1

endatiin

C

t

. We show that node i takesthe correct action to

propagatetheserequestchainsbysendingeitheranew

RequestoraLinkInfo message.

Case 3.1: Node i receives a LinkInfo message from

neighborj= next

i

indicatingthat i'slinkto j has

become incomingat i. If thelinkto j wasi'slast

outgoinglink, then inC

t

i willcall RaiseHeight().

Node i will delete the identiers of any nodes on

outgoinglinksfromits request queue. Node i will

send aLinkInfo messageto each neighbor,

includ-ing nodes whose identiers were removed from i's

requestqueue. Ifi's requestqueueis non-emptyit

willcallForwardRequest()andsendaRequest

mes-sageto thenodechosenasnext

i inC

t .

Case 3.2: Nodeireceivesanindicationthatthelinkto

next

i

hasfailed. InC

t

,iwilltakethesameactions

asitdidincase3.1,whenitslinktonext reversed.

Therefore,noactiontakenbynodeicanmake

prop-erties(b)and(c)falseandthelemmaholds.

Lemma5. Once link changes and calls to

Raise-Height() cease,foreverycongurationinwhichanode

l'srequestchaindoesnotincludethetokenholder,then

thereisalatercongurationinwhichl'srequestchain

doesincludethetokenholder.

Proof: ByLemma 3, after link changes cease,

even-tuallyatokenorientedDAGwillbeformed. Consider

a conguration after link changes and calls to

Raise-Height() cease in which the DAG is token oriented,

meaning that all LinkInfo messages generated when

nodesraisetheirheightshavebeendelivered.

Theproof isby contradiction. Assumenode l's

re-quest chain never includes the token holder. So the

token can only be held by or be in transit to nodes

that are not in l's request chain. By our assumption

on the execution, no LinkInfo messages caused by a

calltoRaiseHeight() willbe intransitto anode inl's

request chain, nor will any node in l's request chain

detect afailed linkto a neighboringnode. Therefore,

by Lemma 4(c), a Request message must be in

tran-sit from anode in l's request chain to a node that is

not in l's request chain, and the number of nodes in

l's request chain will increase when the Request

mes-sage is received. At this point, l's request chain will

eitherinclude the token holder, another Request

mes-sagewillbeintransitfromanode inl's requestchain

toanodethatisnotinl'srequestchain,orl'srequest

chain willhavejoinedthe requestchainof someother

node. Whilethe numberof nodes inl's request chain

increases,thenumberof nodesnotinl'srequestchain

decreases, since there are a nite number of nodes in

thesystem. Soeventuallyl'srequestchainincludesall

nodes. Therefore, if the token is not eventually

con-tained in l's request chain, it is not in the system, a

contradiction.

Letlbeanodethat isWAITINGafterlinkchanges

and calls to RaiseHeight() cease. Given a

congura-tion s in the execution, a function V

l

for l is dened

to be the following vector of positive integers. Let

p 1 =l;p 2 ;:::;p m bel'srequest chain. V l (s)haseither m+1ormelementshv 1 ;v 2

;:::i,dependingonwhether

aRequest messageis intransitfromp

m

ornot. In

1<jm,v j isthepositionofp j 1 inQ p j . (Positions

arenumberedinascendingorderwith1beingthehead

of thequeue.) If aRequestmessage isintransit, then

V

l

(s)hasm+1elementsandv

m+1

=n+1;otherwise,

V

l

(s)hasonlymelements. Thesevectorsarecompared

lexicographically.

Lemma6. V

l

isavariantfunction.

Proof: Thekeypointstoproveare:

(1) V

l

never has more than n entriesand every entry

is between1 and n+1,so therange of V

l

is

well-founded.

(2) Most eventscan be easilyseennot to increaseV

l .

Herewediscusstheremainingevents.

Whenthe Request messageat theend of l's

re-quest chain is received by node j from node p

m ,

l's request chain increases in length to m+1, V

l decreases from hv 1 ,:::, v m , n+1i to hv 1 ,:::, v m , v 0 m+1 ,:::i, where v 0 m+1 <n+1since v 0 m+1 is p m 's positioninQ j

aftertheRequestmessageisreceived.

Whena Token message is receivedby the node

p

m

attheendofl'srequestchain,itiseither

- keptatp m ,soV l decreasesfromhv 1 ;:::;v m 1 ;v m i tohv 1 ;:::;v m 1 ;v m 1i, - or sent toward l, so V l decreases from hv 1 ;:::; v m 1 ;v m ito hv 1 ;:::;v m 1 i,

- orsent away from l, followed by a Request

mes-sage, so V l decreases from hv 1 ;:::;v m 1 ;v m i to hv 1 ;:::;v m 1 ;v m 1;n+1i.

(3) ToseethattheeventsthatcauseV

l

todecreasewill

continuetooccur,considerthefollowingtwocases:

Case1: Thetokenholderisnotinl'srequestchain.

ByLemma5,eventuallythetokenholderwillbe

inl'srequestchain.

Case2: The token holder is in l's request chain.

Since no node stays inthe CS forever, at some

later time the token will be sent and received,

decreasing the value of V

l

, by part (2) of this

proof.

OnceV

l

equalsh1i,lenterstheCS. Wehave:

Theorem2. Iflinkchanges cease,theneveryrequest

iseventuallysatised.

6.Simulation Results

Inthissectionwediscussthestaticanddynamic

per-formanceoftheReverseLink(RL)algorithmcompared

toamutualexclusionalgorithmdesignedtooperateon

astaticnetwork. WesimulatedRaymond'stokenbased

mutualexclusionalgorithm[25]asifitwererunningon

topof a\routing"layerthat alwaysprovided shortest

pathroutesbetweennodes. Inthissection,wewillrefer

to thissimulationas\Raymond'swithrouting" (RR).

Raymond'salgorithmwasused becauseitis thestatic

algorithm from which the RL algorithm was adapted

andbecauseitdoesnotprovideforlinkfailuresand

re-coveryand mustrelyon theroutinglayerto maintain

logicalpathsifruninadynamicnetwork.

Complexitycomparisonofaroutingprotocolis

com-plicatedby the fact that thenumberof messagesand

amountoftimeneededtomaintainroutescanbe

amor-tizedoverthenumberofapplicationsusingthoseroutes.

Inordertomakeourresultsmoregenerallyapplicable,

we made best-case assumptions about the underlying

routingprotocolusedwithRaymond'salgorithm: that

italwaysprovidesshortestpathsanditstimeand

mes-sagecomplexityarezero. Ifoursimulationshowsthat

theRLalgorithmisbetterthantheRRcombinationin

somescenario,thentheRLalgorithmwillalsobebetter

than Raymond's algorithmin that scenario when any

realadhocroutingalgorithmisused. Ifoursimulation

showsthattheRLalgorithmisworsethantheRR

com-binationinsomescenario,thenitmightormightnotbe

worse in an actual situation,depending onhow much

worseit is inthesimulationand whatare thecostsof

theroutingalgorithm.

A30 node systemwassimulatedunder various

sce-narios. A 30 node system was chosen, in part,

be-causefornetworkslargerthan30nodesthetimeneeded

for simulation was very high. Also, ad hoc networks

aregenerallyenvisionedto bemuch smallerscalethan

wired networks like the Internet. Typical numbersof

nodes used for simulations of ad hoc networks range

from10to50[4{6,15,18,26].

Inall ourexperiments, each CS executiontook one

time unit and each message delay wasone time unit.

RequestsfortheCSweremodeledasaPoissonprocess

with arrival rate 

req

. Thus the time delay between

mean 1



r eq

timeunits. Linkchangesweremodeledasa

Poissonprocesswitharrivalrate

mob

. Hencethetime

delaybetweeneachchangetothegraphisan

exponen-tialrandomvariablewith mean 1

mob

timeunits. Each

change to thegraphconsisted ofthe deletionofalink

chosen at random (whose loss did not disconnect the

graph)and theformationofalinkchosenat random.

Ineach execution,wemeasuredtheaveragenumber

of timeunitsthat nodesspentintheirWAITING

sec-tionsandtheaveragenumberofmessagessentperCS

entry,while varyingtheloadonthesystem(

req ),the

degreeofmobility(

mob

),andthe\connectivity"ofthe

graph. Connectivitywasmeasuredasthepercentageof

possiblelinksthatwerepresentinthegraph. Notethat

acliqueon30nodeshas435(undirected)links.

Inthegraphsofourresults,eachplottedpoint

rep-resentstheaverageofverepetitionsofthesimulation.

ThusinplotsofaveragetimeperCS entry,eachpoint

istheaverageoftheaveragesfromveexecutions,and

similarly for plots of average numberof messages per

CSentry.

The same set of initial graphs were used on both

theRL andRRsimulationsforeachexperiment.

Dur-ing periods of mobility, linkchanges were not allowed

to change thepercentconnectivityofthe initialgraph

more than 10% in either the positive or negative

di-rection. For example, when starting with an initial

graph with a connectivity of 30%, theconnectivity of

the graphwasmaintainedat values between20% and

40%connectivityduringsimulationswithmobility.

Throughoutthis section,part(a)of each gure

dis-plays results when the graph is static, part (b) when



mob

= 510 2

(low mobility), and part (c) when



mob

= 510 1

(high mobility). Our choice for the

valueofthelowmobilityparametercorrespondstothe

situationwherenodesremainstationaryforafewtens

of seconds after moving and prior to making another

move. Our choice for the value of the high mobility

parameter represents a much more volatile network,

where nodes remain static for only afew seconds

be-tweenmoves.

6.1. Average waitingtimeper CSentry

Figure 9 plots the average number of time units

elapsedbetweenhostrequest and subsequent entry to

theCSagainstvaluesof

req

increasingfrom10 3

(the

3

timeunitsbetweenrequestsis1)fromlefttorightalong

thexaxis. Wechosethehighloadvalueof

req

because

atthisrateeachnodewouldhavearequestpending

al-mostallthetime. Thelowloadvalueof

req

represents

amuchlessbusynetwork,withrequestsrarelypending

atallnodesatthesametime. Plotsareshownforruns

with 20% (87 links)and 80%(348 links)connectivity

forboththeRLandRRsimulations.

Inthestaticcase,showninFigure9(a),RRhas

wait-ingtimeroughlyequaltoRLatthelowestandhighest

loadsandbetteraveragewaitingtimewhenloadis0.01.

In RL at aload of 0.001, the LinkInfo messages have

suÆcienttimeatthestartoftheexecutiontopropagate

betweentokenmoves,reducingthepathlengthbetween

potential requestersand thetoken holder. Whenload

is0.01,LinkInfo messagesinRLdonothavesuÆcient

time to propagate between token moves, resulting in

longerrequestpathsandmoretokenhopsbetween

con-secutiverequesters. Bothsimulationshaveroughlythe

sameaveragewaittimewhenloadis0.1orhigherinthe

staticcase. Thissimilaritycanbeexplainedbythe

ob-servationthat thenetworkbeginsto besaturatedwith

requestsat theseloadssothat agivennodemustwait

forthe tokento beused byagreater numberofother

nodesbetweenitsownconsecutiveCS entries.

Figure 9, parts (b) and (c), indicate that RL has

betterperformancethanRR intermsofaverage

wait-ingtime perCS entryfor mediumto high loadswhen

nodes are mobile. The waiting time advantage of RL

overRR increaseswith increasingload and increasing

mobility,particularlyatlowconnectivity. Athigh

con-nectivity,itislessprobablethataparticularroute

be-tweentwonodeswillbedisrupted,resulting insimilar

performanceoftheRLandRRsimulations.When

con-nectivitydecreasesinamobilenetwork,theRRaverage

waittimeincreasesbecauseRaymond'salgorithmsends

applicationmessagesoverastaticvirtualspanningtree.

Whenamessageissentfromanodetooneofits

neigh-bors in the virtual spanning tree, it may actually be

routed over a long distance, thus increasing the time

delay. In contrast,the RL algorithmuses accurate

in-formationaboutthe actualcurrenttopology, resulting

inless delaybetween each request and subsequentCS

entry.

In order to further study the eect of connectivity,

we ran the experiments shown in Figure 10: the

re-0.001

0.001

0.001

0.01

0.01

0.01

0.1

0.1

0.1

1

1

1

0

1

1

20

10

10

40

100

100

60

1000

1000

80

100

RL, 20% Connectivity

RL, 80% Connectivity

RR, 20% Connectivity

RR, 80% Connectivity

Load

Time Units/CS Entry

(a)

(b)

Load

Time Units/CS Entry

Load

Time Units/CS Entry

(c)

Figure9.Loadvs.timeperCSentryfor(a)zero,(b)low(

mob = 510 2 ),and(c)high( mob =510 1 )mobilityinRLvs.RR simulations.

questandsubsequententrytotheCSisplottedagainst

networkconnectivityincreasingfrom10%(43links)to

100% (435 links)alongthe x axis. Curvesareplotted

for lowload, where 

req =10

3

(the mean numberof

timeunitsbetweenrequestsis10 3

)andhighload,where



req

=1(1 meantimeunitbetweenrequests)forboth

FromFigure 10(a), thestaticcase,wecansee that,

at the loadstested, the RL and RR simulationshave

nearlythesameaveragewaitingtimeat all

connectivi-ties. ThiscorroboratestheresultsshowninFigure9(a).

AtlowloadintheRLalgorithm,thereissuÆcienttime

forLinkInfo messagestopropagatetoallneighbors

be-tweentoken moves,resultinginrequest pathsthatare

asshortasthoseexistingintheRRsimulation. Athigh

load,there is alwaysa requestpending at everynode,

resultinginaroundrobinpatternofCSentriesforboth

RRandRL,regardlessofconnectivity.

TheadvantagegainedbytheRLalgorithminterms

of waitingtime per CS entry becomes apparentwhen

nodesaremobile,asshowninFigure10,parts(b)and

(c). Becauserequest pathsand request queueschange

to match the physical connectivity in the network in

RL, the token spendsmore time inactualuse. In the

RR simulation, the token spends more time traveling

betweennodesbecauserequestqueuesarenotmodied

whenlinksfail,resultinginhigheraveragewaittimeat

allconnectivityranges. Atthehighestmobility,theRL

simulationhadaloweraveragewaittimeatbothtested

loadswhentheconnectivitywas20%orlower.

Theresultsofthesimulationsinthissectionare

sum-marizedinTable1. Thistableincludesdatapointsfrom

both sets of graphs depicted in this subsection. The

chosendata pointsshowaveragewaitingtime forhigh

(80%)andlow(20%)connectivityandforhighandlow

loadsinallmobilityscenarios.

6.2. Averagemessages per CSentry

TheRRalgorithmsendsrequestandtokenmessages

alongthe virtual spanningtree. Each messagefroma

nodetoitsvirtualneighborisconvertedintoasequence

ofactualmessages,thattraversethe(current)shortest

pathfromthesenderto therecipient.

TheRLalgorithmsendsRequestandTokenmessages

alongtheactualtokenorientedDAG.Inaddition,asthe

token traverses a path, each node on that path sends

LinkInfo messages to all its outgoing neighbors.

Ad-ditional LinkInfo messages are sent, and propagated,

whenalinkfailurecausesanodetoloseitslast

outgo-inglink.

Ourexperimentalresultsre ecttherelativenumber

ofhopstakenbyalgorithmmessagesforRRversusthe

0

0

0

20

20

20

40

40

40

60

60

60

80

80

80

100

100

100

0

1

1

20

10

10

40

100

100

60

1000

1000

80

100

RL, Low Load

RR, High Load

(a)

Time Units/CS Entry

Connectivity

Connectivity

(c)

Time Units/CS Entry

Delay between CS release

& next request = 1 time unit.

Delay between CS release

& next request = 1000 time units.

RR, Low Load

RL, High Load

Connectivity

(b)

Time Units/CS Entry

Figure10. Connectivityvs.time perCSentryfor(a) zero,(b)

low( mob =510 2 ),and(c)high( mob =510 1 )mobility inRLvs.RRsimulations.

When interpretingthese results, it is importantto

re-memberthatthe simulationofthe RRalgorithm isnot

charged for messages needed to recalculate the routes

duetotopologychanges. Thus,ifRL isbetterthanRR

insomesituation,itwillcertainlybebetterwhen

rout-Table1

SummaryofaveragetimeperCSentry.

Static Low

a

High b

Case Mobility Mobility

20% c 80% c 20% c 80% c 20% c 80% c RRhigh load d 85 85 193 114 401 125 RLhigh load d 85 86 82 84 70 79 RRlow load e 7 4 22 7 65 8 RLlow load e 7 4 7 4 10 4 a

Meantimeunitsbetweeneachlinkchange=500.

b

Meantimeunitsbetweeneachlinkchange=50.

c

Averagenetworkconnectivity.

d

Meantimeunitsbetweenrequests=1.

e

Meantimeunitsbetweenrequests=1000.

Also,ifRRisbetterthanRLinanothersituation,

de-pending on how much betterit is, RL might be

com-parableorevenbetterthanRRwhenroutingmessages

arechargedtoRR.

Figure 11plots theaveragenumberof messages

re-ceivedperCSexecutionagainstvaluesof

req

ranging

from10 3

(themeantimeunitsbetweenrequestsis10 3

)

to 1(the meantime unitsbetweenrequests is1) from

leftto rightalongthex axis. Plotsareshownforruns

with 20% (87 links)and 80%(348 links)connectivity

forboththeRLandRRsimulations.

Figure11 showsthat theRRalgorithmsendsfewer

messages per CS entry than the RL algorithm in all

simulation trials at these two particular connectivity

values.

In all situations studied, except the RL simulation

in the static case with high connectivity, the number

ofmessagesperCS entrytends todecrease asload

in-creases. Thereasonforthisdecreaseinnumberof

mes-sagesisthat, althoughtheoverallnumberofmessages

increaseswithloadinbothalgorithmsduetothe

addi-tionaltokenand requestmessages,the overallnumber

of CS entries increases proportionately faster as load

increases. Intheextreme,atveryhighload,everytime

thetokenmoves,itislikelyto causeaCSentry.

In the static case (Figure 11(a)) with 80%

0.001

0.001

0.001

0.01

0.01

0.01

0.1

0.1

0.1

1

1

1

0

0

1

5

10

10

10

20

100

15

30

1000

20

40

25

50

30

60

35

70

80

Messages/CS Entry

(b)

Load

(c)

Load

Messages/CS Entry

RL, 20% Connectivity

RL, 80% Connectivity

RR, 20% Connectivity

RR, 80% Connectivity

Messages/CS Entry

Load

(a)

Figure11. Loadvs.messagesperCSentryfor(a)zero,(b)low

( mob =510 2 ),and(c)high( mob =510 1 )mobilityin RLvs.RRsimulations.

is not apparent in the RL simulation at 20%

connec-tivity. An explanation for this peak may be that

to-ken movement more eectively shortens request path

length at high connectivity with load below 0.01. At

loadshigherthan0.01,thetokenisusedmoreoftenin

agivennumberofhopsbecausemorenodesrequestthe

ofmessagesperCSentry.

TheRL algorithmsends moremessagesper CS

en-try than the RR algorithmwhen mobility causes link

changes,andthenumberofmessagessentintheRL

al-gorithm grows very large under low loads, as can be

observed in Figures 11(b) and (c). When links fail

andform,theRLalgorithmsendsmanyLinkInfo

mes-sagestomaintainthetokenorientedDAG,resultingin

ahigher messageto CS entry ratioat lowloadswhen

thedegreeofmobilityremainsconstant.

Figure12showstheresultsofexperimentsdesigned

to understand the eect of connectivity on the

num-berof messagesperCS entry. In thegure, the

aver-agenumberofmessagesperCSentryisplottedagainst

networkconnectivityincreasingfrom10%(43links)to

100%(435links)fromlefttorightonthexaxis.Curves

areplottedforlowload,where 

req =10

3

(the mean

timeunitsbetweenrequestsis10 3

)andhighload,where



req

=1 (the mean time unitsbetween requests is 1)

forboththeRLandRRsimulations.

In the static case (Figure 12 (a)), the number of

RL messages per CS entry increases with

connectiv-ity. Asconnectivityincreases,thenumberofneighbors

pernodeincreases,resultinginmoreLinkInfomessages

being sent as the token travels. The number of

mes-sagessentperCSentryintheRRsimulationdecreases

with connectivity, since the shortest path lengths

be-tweenneighbors inthe virtualspanning treedecrease.

Athighload,simulationresultsfortheRRsimulationin

thestaticcasematchtheperformanceofapproximately

4messagesperCS entrycitedby Raymond[25] forall

connectivitylevels. This performance is also matched

bytheRRsimulationatlowerloadswhenconnectivity

isabove80%,duetotheextremelyshortrequestpaths

whenconnectivityishigh.

IntheRL algorithm,thereare twoopposingtrends

with increasing connectivity when nodes are moving

(Figure12,parts(b)and(c))thatappeartocanceleach

otherout: 1)higherconnectivitymeansmoreneighbors

pernode,whichmeansmoreLinkInfo messageswillbe

sentwitheachfailure,and2)moreneighborspernode

meansthatitislesslikelyforalinkfailureorreversalto

involvethelast outgoinglink,and thus LinkInfo

mes-sagesdue to failurewill propagateless. At high load,

the RL simulation sends nearly as few or even fewer

messagesthandoesRRwhenconnectivityisbelow20%

0

0

0

20

20

20

40

40

40

60

60

60

80

80

80

100

100

100

0

1

0

5

10

20

10

100

40

15

1000

60

20

80

25

100

30

120

35

140

160

Messages/CS Entry

Connectivity

(c)

RR, High Load

RL, Low Load

Messages/CS Entry

Connectivity

(a)

Messages/CS Entry

(b)

Connectivity

RL, High Load

RR, Low Load

Delay between CS release

& next request = 1 time unit

& next request = 1000 time units

Delay between CS release

Figure12. Connectivityvs.messagesperCSentryfor(a)zero,

(b) low ( mob = 510 2 ), and (c) high ( mob = 510 1 ) mobilityinRLvs.RRsimulations.

The results of the simulations measuring messages

per CS entry are summarized in Table 2. This table

includesdatapointsfrombothsetsof graphsdepicted

inthis subsection. Thechosen data pointsshow

aver-agenumberofmessagesforhigh(80%)and low(20%)

scenarios.

Table2

SummaryofaveragemessagesperCSentry.

Static Low

a

High b

Case Mobility Mobility

20% c 80% c 20% c 80% c 20% c 80% c RRhigh load d 4 4 11 6 29 7 RLhigh load d 10 27 13 27 43 44 RRlow load e 6 4 22 7 48 8 RLlow load e 13 17 73 48 496 331 a

Meantimeunitsbetweeneachlinkchange=500.

b

Meantimeunitsbetweeneachlinkchange=50.

c

Averagenetworkconnectivity.

d

Meantimeunitsbetweenrequests=1.

e

Meantimeunitsbetweenrequests=1000.

7.Conclusion and Discussion

We presented a distributed mutual exclusion

algo-rithmdesigned to be aware of and adapt to node

mo-bility,alongwithaproofofcorrectness,andsimulation

resultscomparingtheperformanceofthisalgorithmto

thatofastatictokenbasedmutualexclusionalgorithm

running on top of an ideal ad hoc routing protocol.

We assumed there were no partitions in the network

throughoutthis paperfor simplicity;partitionscanbe

handledinouralgorithmbyusingamethodsimilarto

thatusedintheTORAadhocroutingprotocol[22]. In

[22],additionallabelsareusedtorepresenttheheights

ofnodes,allowingnodestodetect,byrecognitionofthe

originatorofachainofheightincreases,whenaseriesof

heightchangeshasoccurredatallreachablenodes

with-outencounteringthe\destination". Asimilarpartition

detection mechanism could be encorporated into our

mutual exclusion algorithm at the expense of slightly

largermessages.

Ouralgorithmcomparesfavorablytothelayered

ap-proachusinganadhocroutingprotocol,generally

pro-vidingbetteraveragewaitingtimeperCSentryin