A Constrution of Distributed Referene
Counting
Lu Moreau 1
, Jean Duprat 2
1
UniversityofSouthampton,UK,e-mail:L.Moreaues.soton.a.uk
2
EoleNormaleSuperieure,Lyon,Frane,e-mail:Jean.Dupratens-lyon.fr
Reeived:date/Revisedversion:date
Abstrat. Distributedreferene ountingisageneral purpose
teh-nique, whih maybe used,e.g., to detet terminationof distributed
programsortoimplementdistributedgarbageolletion.Wepresent
a distributed referene ounting algorithm and a mehanial proof
of orretness arried out using the proof assistant Coq. The
algo-rithm is formalisedbyan abstrat mahine, and its orretness has
two dierent faets. The safetyproperty ensures that ifthere exists
a referene to a resoure, then its referene ounter will be stritly
positive. Liveness guarantees that if all referenes to a resoure are
deleted, its refereneounterwilleventuallybeome null.
1Introdution
Referene ounting is a general purpose tehnique that is able to
ount the number of referenes to a given resoure. Collins [5℄ was
therstto useitinorderto determinewhenlistellswerenolonger
needed. Operating systems relyon thistehnique in order to deide
when les may be deleted or when le desriptors may be losed.
Refereneountingisalsoamethodforimplementinggarbage
olle-tion,amemorymanagementtehniquethatautomatiallydetermines
when objets may be dealloated. We refer the readerto Jones and
Lins'book[19,setion2.1℄foradisussionoftheproandonsofthis
tehnique forgarbage olletionpurpose.
Distributed refereneounting isan extensionof referene
posi-ofwhetheraresoureisusedannolongerbetakenloally,butmust
involve a ollaboration with the dierent loations partiipating in
theomputation.Distributedreferene ountingmaybe usedto
im-plementdistributedgarbage olletion;a variant ofthistehnique is
inpartiularusedinJavaand RMI[27,18℄.Eventhoughdistributed
referene ounting is notable to deal with distributedyles, it has
beenapopularimplementationtehniqueofdistributedgarbage
ol-letionbeauseitissimpletoimplementandannielybeintegrated
withsequentialgarbageolletors[3,27,32,41℄.Moregenerally,itmay
beused fortraking referenes to resoures[15℄. A possibleuse isto
detet termination of distributed programs [40℄; referene ounting
may be used for suh an appliation beause proesses form a
hier-arhy. Groups [31℄ also have a hierarhial organisation and an be
referene ounted.
Therstauthorreentlypublishedanewalgorithmfordistributed
refereneounting[27℄.Ithasthepropertythatallreferenesmaybe
foundatanytime,whihan beusefulwhen theownerofaresoure
wishes to propagate information to the resoure users. In fat, this
algorithm desribes a family of implementations, aording to the
poliyadoptedtopropagatemessages.Inpartiular,Piquer'sIndiret
Referene Counting [32℄ an be seen as a partiular instane of our
algorithm.
Thepurposeofthispaperistopresentthisalgorithmandtoprove
itsorretness.Theorretnessofarefereneountingalgorithmhas
two dierent faets. Safety guarantees that if there exists a
refer-enetoaresoure,thenitsrefereneounterwillbestritlypositive.
Livenessguaranteesthatifallreferenestoaresourearedeleted,its
referene ounterwilleventuallybeome null.
The ontribution of thispaperis the desriptionof a mehanial
proof that has beenarried outusing thealulus of indutive
on-strutionsandtheproofassistantCoq[1℄.Wehavealsostudiedsome
optimisationsandhave onsideredtwoalgorithm variants.In
parti-ular,we present referenelisting,whihis avariantofthe algorithm
that not only ounts referenes to a resoure, but also remembers
wherethoserefereneswerepassed.Referenelistingisauseful
teh-nique to assist in building a fault tolerant version of the algorithm
[4℄.
The motivation for this work is threefold. First, even with the
bestintentions, itis easy to skipreasoningstepsin paperproofs, or
tooverlooknon-trivialproperties.Parallelanddistributedalgorithms
proofassistantCoqrequiresonstrutiveproofs, whihforedusnot
only to state properties, but also to provide a mehanial way to
derive them. Suh an exerise has proved to be suessful beause
we managed to speifyvery preisely thenotions of alternatequeue
and diusiontree,whihareentralto theproofof safety.Third,we
seethisworkaspartofa largerativityaiming toertifydistributed
software systems; the hope is that our formalisation may be reused
asa moduleformore omplexsystems.
The soure ode for the proof in Coq is available from [30℄. The
proofisabout13000lineslong,plusanextra3000 linesforalgorithm
variants.We present herea seletionof denitions,lemmasand
the-orems, ina notation that is very lose to the one in our Coq proof.
For the sake of oniseness, proofs are only skethed, but omplete
proof detailsmaybeobtained from[30℄.
This paper is organised as follows. First, we set the ontext in
whihthealgorithmwasdevelopedandpresentitsintuition (Setion
2).Thealgorithmisthenformallydesribedasanabstrat mahine,
whih we all the DRC-mahine (Setion 3). General properties of
the mahine are dened, inluding some basi invariants and a
no-tionofdiusiontree thatrepresentsthepathbywhihreferenesare
propagatedina omputation(Setion 4).Corretness isestablished,
involving bothsafetyandlivenessaspets(Setion 6).Then,
optimi-sations and algorithm variants are investigated (Setions 6 and 7).
Finally,we onludethepaperwith relatedwork.
2The Algorithm: Informal Presentation
The initial motivation for this work was the design and
implemen-tation of a distributedlanguage [29℄, based on the message-passing
libraryNexus[10℄.Thislibraryessentiallyprovidesanotionofglobal
pointer (GP), whih is a referene to a remote objet, and a form
of remoteproedure all,whihallows theprogrammerto ativatea
omputation on an objet pointed at by a GP; any data, inluding
global pointers, may be passed as argument to a remote proedure
all.
We assume that several loations partiipate to a omputation
andweallthemsites.Duringtheourseofaomputation, GPs are
reatedandommuniatedbyremoteproedurealls.Thesitewhere
aGP isinitiallyreatedisalleditsowner;theownerontainssome
datathataGP isreferringto.Newlyreatedglobalpointersmustbe
repre-adopt the following failure assumptions: there exists a reliable
mes-sage delivery, i.e. messages annot be lost, orrupted ordupliated;
mahinesneverrashandarenevertakenoutofservie;thereistrust
arossthe entiredomain.
The purposeof distributedreferene ounting is to keeptrak of
thedierentGPs.Morepreisely,eahGP willbeassoiatedwitha
refereneounter.OnaGP'sowner,arefereneounterisexpetedto
bestritlypositivewheneveraopyoftheGP isaessibleremotely.
Weusetablestomaintainassoiationsbetweenountersandglobal
pointers that were sent to remote sites. We all these tables
send-tables as they are used whenever GPs are sent remotely. Eah site
ontainssuha send-table.
Re-T
s
1 GP
Re-T s
2 Send-T
Send-T
0
0
Re-T
1 opy(GP)
s
1 GP
Re-T s
2 GP Send-T
Send-T
Re-T
de(GP)
s
1 GP
Re-T s
2 Send-T
Send-T
0
Fig.1 CopyingandDeleting aReferene
Letusonsidertwo sitess
1 ands
2
,some dataons
1
,and aglobal
pointer GP pointingat thisdata. Initially,theounter of GP is set
to zero inthe send-tableof s
1
.Every timea GP issent to a remote
site, its assoiated ounter is inremented by one. The reader will
notethatrefereneountersareusedforountingreferenesbetween
sites;other mehanismsmaybeused forountingreferenes loally.
The middle piture in Figure 1 shows that opying GP has
in-reased its referene ounter in the send-table of s
1
. To a rst
ap-proximation, the send-table indiates the number of times a global
pointer wassent remotely.The middlepiture indiatesthat a opy
ofGP isaessible ons
2
and thesend-tableons
1
isstritlypositive.
ventions.Eahpiturerepresentsasnapshotofthesystem,atagiven
point intime. A bold arrow from s
1 to s
2
indiates that a message
was sent bys
1
and reeived bys
2
;thesnapshot representsthestate
of thesystemafter the message hasbeenreeived and proessed.
In order to keep referene ounters up to date, eah site has to
be able to determine whether a GP has already been reeived. For
thispurpose,eahsitemaintainsaseondtable,alledreeive-table 1
,
whih ontains theglobal pointers that have already been reeived.
Byonstrution,aGP belongstoitsowner'sreeivetable.Aording
to themiddlepitureofFigure1,GP isinthereeive-tablesofboth
s
1
(itsowner)and s
2 .
Inaddition toreferene ounters,thedistributedreferene
ount-ing algorithm uses ontrol messages, whose purpose is to update
ounters. A derement message is aimed at a site and ontains a
global pointer GP. When the destination site reeives suh a
mes-sage, itderementstheounterassoiatedwithGP initssend-table;
iftheounterreahes0,theobjetassoiatedwiththepointeristhen
unreferenedbyremotesites.
Weusederementmessagesintwodierentsituations.First,when
a GP is no longerneeded bya site, GP is removed from thereeive
table and a derement message is sent to GP's owner. In Figure 1,
assoon asGP is unneededon s
2
,a derementmessage issentto s
1 ,
whihinthepresentasehastheeet ofresettingitsounterinthe
send-tableofs
1
.AGP an bedelaredunneededon asiteifitisnot
required by the loal omputation and its assoiated ounter in the
send-tableis null.
Seond,whenaGP isreeived byasitethatalreadyownsaopy
oftheGP (asindiatedbyitsreeivetable),aderementmessagehas
to be sent bak to the emitter so as to maintain aurate referene
ounters. Now, we an rene theounter desription: a ounter ina
send-tablerepresentsthe numberof dierent remote opiesofa GP
plusthenumberof messagesrelated to itintransit.
Letusnowonsiderthreesites.Figure2illustratesasenariothat
followsthemiddlepitureofFigure1,whereGP hasbeenopiedfrom
s
2 to s
3
.Usingthe same priniple,the ounterfor GP on s
1 and s
2
hasa value1,and theGP is also inthereeive-tablesof s
2 and s
3 .
In fat, the mehanism we desribe here bears some resemblane
with Indiret Referene Counting [32℄, where the sum of referene
1
Weallour tables send andreeive beause theyare usedwhensending or
reeivingglobalpointers,respetively.Othernamesmaybefoundintheliterature:
Re-T
1
Re-T
s
3 GP
Re-T 1
s
1 GP Send-T s
2 GP
Send-T opy(GP)
Fig.2 ThreeSites
ounters aross the diusion tree of a GP is the number of its
re-moteopies.Theanalogydoesnotextendfurtherbeausederement
messagesare useddierently.
Let usreall that, when a GP is no longer needed, a message is
senttoitsowner.Thisdesigndeisionismotivatedbythefatthata
NexusGP onlyreferstoitsownersite,andhasnoinformationabout
thesitesittransitedby.Unfortunately,untimelyderementmessages
may be the onsequene as illustrated in Figure 3. If s
3
, whih
re-eived GP, deletes its referene to GP, then s
3
sends a derement
message to s
1
, that is,the GP's owner. The eet of thederement
messageistoresettherefereneounterons
1
.Thislearlyresultsin
an inonsistent situation asGP may stillbe ative on s
2
, whilethe
referene ounteron s
1
is null.
Besides theinorretness related to thederement message,suh
an indiretreferene ounter tehnique may keep some pointers
a-tive longer than expeted; in other words, this results in a form of
memoryleak. Indeed, GP remainsneededbys
2
inFigure 2 beause
the ounter for GP in s
2
send-table is not null, even if the loal
omputationdoesnotuse thispointeranylonger.
Our solutionto boththe untimely arrival of messagesand
mem-oryleaksinvolvesanewtypeofmessage,alledinrement-derement,
writtenin de .An inrement-derement message involves three
dif-ferent sites: s
1 ;s
2 ;s
3
, respetively, the owner, the emitter and the
reeiverofaGP.WhenGP reahesthereeiverforthersttime,an
inrement-derement message is sent to its owner. When the owner
s
1
reeivesan inrement-derement message,itinrementsGP's
Re-T
s
3
Re-T
s
1 GP Send-T
0 Re-T
s
2 GP Send-T
1
de(GP)
Fig.3 UntimelyDerement
onerningGP (Figures4and5).Theinrement-derementmessage
an beseenasa formof registration,whih hasto be performed the
rst time a GP is reeived; as a onsequene, thisallows theowner
to beaware ofall thesites thathave reeived opiesof aGP.
2
s
1 GP
Re-T Send-T
s
3 GP
Re-T
s
2 GP
Re-T Send-T
1
inde(GP ;s
2 )
Fig.4 DiusionTreeReorganisation(1)
Introduingtheinrement-derement message isnot suÆient to
avoid untimely message arrivals. The inrement-derement message
from thereeiver s
3
should arrive at theowner s
1
before any
dere-mentmessage fromthe reeiver s
3
0
2
s
1 GP
Re-T Send-T
Re-T
s
3 GP
s
2
de(GP) GP
Re-T Send-T
Fig.5 DiusionTreeReorganisation(2)
mission of messages. We therefore assume in-ordermessage delivery
ofmessagesbetweenanypairofsites(inSetions6and7,wedisuss
howsuh a onstraint maybepartiallyrelaxed).
InFigure5,we an observe thatifGP isno longerneededons
2 ,
its owner s
1
may be informedbya de message. Suh a property is
partiularlyimportantinthepreseneofmobileomputations
jump-ingfrom sites to sites.The diusiontree reorganisation providedby
theinrement-derementmessagepreventstheformationofhainsof
pointers abandoned bymobileomputations.
RemarkWe have presenteddistributedreferene ounting as
a general purpose tehnique. It may be used to implement a
distributed garbage olletor. The send-table must be dened
asarootoftheloalgarbageolletor.AGP willbeenteredin
asend-tableonlyifits ounteris stritlypositive.Asa result,
by its presene in the send-table, GP remainsreahable from
theloalolletorroots, whihensures thatthespae usedby
thedatareferened byGP annot be relaimed.As soon asa
refereneounterreaheszero,itsentrymaysafelyberemoved
fromthesend-table.Inontrast, thereeive table mustnotbe
denedasa root ofloal olletor.
3The Algorithm: The DRC-Mahine
Letusnow presentthealgorithm,followingourenodingintheCoq
state spae is displayed in Figure 6. In the DRC-mahine, we only
model messages exhanged bythe distributedreferene ounting
al-gorithm, and we do not model any form of omputation whih it
would be usedin.
S=fs0;s1;:::;snsg (SetofSites)
G=fgp0;gp1;:::;gpn
g
g (SetofGlobalPointers)
M=opy:G!M j de:G!M (SetofMessages)
j inde:GS!M
K=SS!Queue(M) (SetofMessageQueues)
ST =SG!Z (SetofSendTables)
RT =SG!Bool (SetofReeiveTables)
C=ST RT K (SetofCongurations)
Charateristivariables:
s2S; GP2G; m2M; k2K ; send T 2ST; re T 2RT; 2C
Fig.6 StateSpaeoftheDRC-mahine
A nitenumber of sites are involved in a DRC-mahine, and we
onsider a nite number of global pointers. The set of messages is
dened by an indutive type, whose three onstrutors are named
aording to the messages presented in Setion 2, namely opy, de
and in de. Communiation hannels are represented by queues of
messages betweenpairs of sites. We use thefollowing notations and
operationsonqueues:
q;q
1
;::: : denotequeues;
; : theemptyqueue;
first(q): headofa non-emptyqueue q;
tail(q) : non-emptyqueue q exeptits head;
q x fmg: queueq afteradding a message mat itstail;
q
1 x q
2
: queueobtained afteronatenating q
1 and q
2 .
SendandReeiveTablesarerepresentedbyfuntionsassoiatingsites
andglobalpointerswithnumbersorbooleans,respetively.Counters
arerepresentedbyintegers;weshallestablishthatountersarealways
positive or null. A DRC-onguration is given by a tuple of send
tables, reeive tables, and message queues. This abstrat mahine
is a suitable abstration of a distributed system as send-tables and
reeive-tablesmayeasily be distributedarossseveral sites.
assoi-denea funtion
owner:GP !Site;
whih maps eah globalpointeronto its ownersite.
Thedistributedreferene ounting algorithm itself is enodedby
transitions of the DR C-mahine displayed in Figure 7. Transitions
are dened as indutive types, whose onstrutors are make opy,
reeive opy,reeive in de ,reeive deanddelete.Atransition
fun-tionmaps aongurationand atransitiontto anewonguration
0 : 7! t 0 ;
wheretisanyofthevepermittedtransitions.Inaoniseform,
Fig-ure 7 displays the denitions of transitions and the transition
fun-tion.We used some notations suh aspost,reeive ortable updates,
whihgiveanimperativelooktothealgorithm,andwhosedenitions
areasfollows.
- sendT(s;GP) := V denotes hsend T 0
;re T;ki, suh that
send T 0
(s;GP) = V and send T 0
(s; GP 0
) = sendT(s;GP 0
) for
anyGP 0
6=GP.
- re T(s;GP) :=V issimilar.
- post(s
1 ;s
2
;m) denotes hsend T;re T;k 0
i, with k 0 (s 1 ;s 2 ) = k(s 1 ;s 2
)xfmg,and k 0
(s
i ;s
j
)=k(s
i ;s j ), 8(s i ;s j )6=(s
1 ;s 2 ). - reeive(s 1 ;s 2
) denotes hsendT;reT;k 0
i, with k 0 (s 1 ;s 2 ) = tail(k(s 1 ;s 2
)), and k 0
(s
i ;s
j
)=k(s
i ;s j ),8(s i ;s j )6=(s
1 ;s
2 ).
In eah rule of Figure 7, the onditions that appear to the left
hand sideof an arrow are guards that must be satised in order to
performthetransition.Theright-handsidedenotestheonguration
thatis reahedafter transition.
Therst transitiondenotesthetransitionthatisperformedwhen
a GP is opied from s
1 to s
2
. We assume here that the two sites
are dierent. Furthermore, it is a requirement for s
1
to \have
a-ess" to GP, otherwise sendingGP to s
2
wouldbe impossible;suh
a onditionis modeled by GP's presene in the reeive-table of s
1 .
Theresulting ongurationsees thesend-tableof s
1
inreasedanda
message opy sent betweens
1 and s
2 .
Theseond transition isonernedwith s
2
handlingan inoming
opy(GP) message from s
1
. The following ases are possible: (i)
If s
2
has aess to the global pointer GP, i.e. GP is present in s
2
reeive-table,thena demessageis sent bakto theemitters
1 . (ii)
Otherwise, s
2
reeive table is set to true; furthermore, if s
1 and s
Givenaonguration=hsendT;reT;ki,ve transitionsare per-mitted: make opy(s 1 ;s 2 ;GP):
s
1 6=s
2
^ re T(s
1 ;GP)
! fsendT(s
1
;GP):=sendT(s
1
;GP)+1
post(s
1 ;s
2
;opy(GP)) g
reeive opy(s
1 ;s
2 ;GP):
first(k(s
1 ;s
2
))=opy(GP)
! freeive(s
1 ;s
2 )
ifre T(s
2
;GP) then
fpost(s
2 ;s
1
;de (GP))g
else
freT(s
2
;GP):=true
post(s
2
;owner(GP);in de (GP;s
1 )) ifs
1 ;s
2
6=owner(GP)gg
reeive in de (s
1 ;s
2 ;GP;s
3 ): first(k(s 1 ;s 2
))=inde (GP;s
3 )
! freeive(s
1 ;s
2 )
sendT(s
2
;GP):=sendT(s
2
;GP)+1
post(s
2 ;s
3
;de (GP)) g
reeive de (s
1 ;s
2 ;GP):
first(k(s
1 ;s
2
))=de(GP)
! freeive(s
1 ;s
2 )
sendT(s
2
;GP):=sendT(s
2
;GP) 1 g
delete (s;GP):
sendT(s;GP)=0; re T(s;GP);owner(GP)6=s
! fre T(s;GP):=false
post(s;owner(GP);de(GP)) g
to the owner as displayed in Figure 4. Consequently, a neessary
onditionto sendanin demessageistohavereeivedaGP thatis
notloallyaessible 2
.Letusnotethatthereeivedmessagehasbeen
\onsumed" and isno longerpresent intheresulting onguration.
Thethirdtransition dealswithan inomingin de (GP;s
3 )
mes-sage: thesend-tableisinreasedand ade messageis sent tosites
3 .
Thefourth transition reatsto aninoming demessage by
dereas-ingthesend-table fortheonernedglobal pointer.
Deiding when a referene is lost is appliation dependent. For
instane,adistributedgarbageolletormayusealoalgarbage
ol-letortodetetsuhanevent;indistributedtermination[40℄,thelost
of a referene is triggered by the end of a loal omputation. As a
result,weannotmodelsuhriteria,butweanestablishthe
ondi-tionsthat musthold inthedistributedreferene ounting algorithm
whenarefereneisdeleted,asformalisedbythefthtransition.This
transition istypiallyredwhen theappliation deides torelease a
referene. It an only be red if the site is not the GP's owner, if
the send-table is nulland if the reeive-table ontains the GP.The
transition sets the reeive table to false and sendsa de message as
intheright-hand sideofFigure 1.
The initialongurationisdenedasfollows.Reeive-tables
on-tainfalseentriesexeptforGPowners;Send-tablesaresetto0;
Com-muniationhannels areempty.Formally,theinitial onguration
i
isdened bythetuplehreT
i
;send T
i ;K
i i.
re T
i
=sGP:if(s=owner(GP))thentrue elsefalse
send T
i =s
1 s
2 GP:0
K
i =s
1 s
2 :;
A ongurationis saidto belegal ifthereis asequene of
tran-sitionst
1 ;t
2 ;:::;t
n
suh that is reahable from the initial
ongu-ration:
i 7!
t
1
1 7!
t
2
2
::: 7! t
n
:
4Algorithm Properties
Ourgoalistoprovetheorretnessofthedistributedreferene
ount-ing algorithm,whihhas two dierent faets. Safety is theproperty
2
Note that thedeision ofsending aninde message is basedon the
aes-sibility of the GP atthe time a opy message is reeived,independentlyofthe
previous history. A site, dierent from the owner, may therefore reeive a rst
aordingtowhihtherefereneounterofaGP onitsowneris
guar-anteedto bestritlypositivewheneveraopyof theGP isavailable
on aremote site.Liveness isthepropertythatguarantees thatifall
referenestoa globalpointeraredeleted,theowner'ssend-tablewill
eventuallybeome null.
These properties willbe establishedin Setion5,butbeforehand
we presentsome generalpropertiesof thealgorithm.First,we
estab-lishsomeinvariantsrelatingsend-tables,reeive-tablesand messages
intransit.Seond,we analyse theuseofin demessages, whihare
onlyfound on hannels aimed at a GP's owner; we showthat these
hannels have a regular struture. Third, we investigate the notion
of diusiontree,whih is,we previouslylaimed, reorganisedbythe
in demessage.
4.1Invariants
Messages may be assigned a weight, as a measure of their overall
absolute eet on referene ounters. We assign 1 to de and opy
messages beause their eet is respetively to derease or inrease
ounters. On the other hand, we assign a null weight to an in de
message,beauseitinreasestheowner'ssend-table,butitisfollowed
byade message whih dereasesanotherounter.
Weight(de(GP))=1
Weight(opy(GP))=1
Weight(in de (GP;s))=0
Similarly,we an onvert thebooleanvalue storedin areeive table
into an integer.
INT(true)=1
INT(false)=0
The rst invariant establishes that the ounters stored in
send-tables are diretly linked to reeive table values and the weight of
messagesintransit.
Lemma 1.Let = hsend T;reT;ki be a legal onguration. The
following equality holds. For any GP 2G:
X
s
i 2S
send T(s
i
;GP)= X
s
i 2S
INT(reT(s
i
;GP)) 1
+ X
wherem2K #GP denotes the setof messages in K that arerelated
to GP.
Proof. The detailed proof appears in le invariant1.v. It uses an
indution on legal transitions and a ase analysis on the dierent
typesof transitions. ut
owner
s2
s1
opy(GP) inde (GP;s
2 )
de (GP)
opy(GP)
de (GP)
Fig.8 MessagesUnderControlofs
2
The seond invariant denes the value of referene ounters on
sites that dier from the owner. In Figure 8, we identify messages
that update the send-tables of s
2
,or whih result from a hange in
the send-table of s
2
. Indeed, the send-table of s
2
is inreased every
timeaopy(GP)messageissenttoaremotesite;suhaopymessage
maybefollowedbyademessageoranin demessage(towardsthe
owner);thelatterisfollowedbyademessage bakto s
2
.Inreality,
wehaveto onsiderallsitess
1
towhihs
2
sendssuhopymessages.
Denition 1.Let k bea setof queues ofa DRC-mahine
ongura-tion. Let s
i
be a site of S. The set of messages under ontrol of s
i ,
written ontrol(GP;s
i
), is dened as:
ontrol(GP;s
i
)=f mj m=opy(GP);m2 k(s
i ;s
j )
m=de (GP);m2 k(s
j ;s
i ) or
m=inde (GP;s
i
); m2k(s
k ;s
j )
for any s
j ;s
k g:
The seond lemma is stated asfollows:the value of a send-table
on a site s
i
that diers from the owner is given by the number of
Lemma 2.Let hsend T;re T;ki bea legal DRConguration. The
following property holds. For any GP 2G, for any s
i
2S suh that
s
i
6=owner(GP):
send T(s
i
;GP)=#(ontrol(GP;s
i ));
where# denotes the ardinality of a set.
Proof. The equality is initiallytrue and is preserved byeah
transi-tion.The aseanalysis isavailableinleinvariant2.v. ut
Bothinvariantsmay be ombined togetherinorder to obtainthe
value of the owner's send-table in terms of the messages in transit
andreeive-tables.Wewillthenbeableto derivethesafetyproperty
byprovingthat theowner's send-tableis positivewhenevera global
pointer isaessible remotely.However, we needto establishfurther
properties about ontents of messages queues within de messages
and thenotionof diusiontree.
4.2Alternate Queues
Amessage indeissent ifasitesreeivesamessage opy(GP)and
thereeive table fortheGP isemptyon s. Siteswillsend againan
in demessageonlyafterithasperformedadeletetransition,whih
learedthereeive-tableforthatGP.Consequently,wean ndtwo
messagesin de (GP;s
i
) andin de (GP;s
j
)ina same queueonlyif
there is(at least)one de message betweenthem.
Weharaterisesuhabehaviourbythenotionofalternatequeue,
whih speieshowin deand demessages mustbe interleaved.
Denition 2(Alternate). An alternate queue for a given GP is
dened indutively asfollows:
{q is alternate for GP if it does not ontain messages related to
GP;
{q x fin de (GP;s)g is alternate for GP if q does not ontain
messages related to GP;
{ if q is alternate for GP, so is qxfmg provided that m is not an
inde message related to GP;
{ ifqisalternateforGP,soisqxfde (GP)gxq
1
xfin de (GP;s)g,
provided that q
1
isa queue of messages not related to GP.
Lemma 3.Let hsend T;re T;ki bea legal DRC-onguration. For
any GP 2G and for any s2S, k(s;owner(GP)) is alternate for
GP.
Proof. The proof appears in le invariant5.vand proeeds by
in-dutiononthelegaltransitions,andbyaaseanalysisofthedierent
transitions. ut
4.3DiusionTree
In a distributedappliation, global pointers are exhanged between
sites taking part in the omputation. Suh operations are modeled
byopymessagesintheDRC-mahine.Oneaneasilyderiveagraph
struture whose nodes are sites and edges represent the presene of
a opymessage betweentwosites.
However, our motivation is not so muh about understanding
whereopymessagesaresent to,whihisappliation-spei,butto
investigate therole ofin de messagesin the algorithm.An in de
message indiatesthatasiteshasreeived a newglobal pointer,i.e.
s has reeived a global pointer that was not aessible on s. From
this idea, we an derive a notionof diusion tree, whih formalises
thepathtaken byglobalpointers to reah newsites.
We dene the root of the diusion tree as the owner of a global
pointer.Adirethild isasitethatreeivesanewglobalpointerGP,
diretlyfromitsowner.Anindirethild isasitethatreeivesanew
GP froma sitedierent fromits owner. Aordingto thealgorithm,
assoonasanindirethildreeivesanewGP,an indemessage is
postedto its owner.
We an therefore dene a relation diuse(;GP;s
1 ;s
2
), read as
s
1
has diused GP to s
2
in onguration , indiating that s
2 has
reeived thenew GP froms
1 .
Denition 3(Diuse). Given a onguration and a GP,
diuse(;GP;s
1 ;s
2
)holdsifre T(s
2
;GP)=trueandthelastin de
message related to GP in the queue between s
2
and owner(GP) is
in de(GP;s
1 ).
Let us note that we ould nd several in de messages for a given
GP ina given queue,butthe diuse relationisdened bythemost
reentindemessagefortheGP thatwaspostedinthatqueue.The
relationdiuse hanges overtime as inde messages are proessed
Denition 4(Indiret Child). Given a onguration , a global
pointer GP, a site s
2
is an indiret hild if there is a site s
1 suh
that diuse(;GP;s
1 ;s
2
) holds.
Denition 5(Diret Child). A site s that has aess to a GP is
a diret hild ifthereis no s
i
suh that diuse(;GP;s
i
;s) holds.
We dene an anestor as the transitive losure of the relation
diuse. An important property of the anestor relation is its
non-reexivity,whih ensures that thisrelation may be used to denea
tree,and willnotresult ina graph.
Lemma 4(Not Reexive). For any legal onguration , for any
global pointer GP, and for any sites s
1 ;s
2
,if anestor(;GP;s
1 ;s
2 ),
then s
1 6=s
2 .
Proof. The proof,availablein invariant6.v,proeeds byindution
onthelegaltransitionsandbyaseanalysison thedierent kindsof
transitions. ut
s
1 ow ner(GP)
s2 s
1
s
2 inde(GP;s
1 )
IndiretChild
DiretChild
diuserelation
ow ner(GP)
Fig.9 DiusionTreeReorganisation
Inthe left-hand sideof Figure 9, GP wasdiused from s
1 to s
2 ,
as visualised by the in de message towards the GP's owner. The
eet ofan inde messageisto \register" asitethathasreeiveda
newGP.Assoon asthein demessageisreeived bytheowner, s
Whenallin demessageshavebeenproessed,allsiteswillbediret
hildren.The eet of thein demessage is therefore to atten the
diusiontree.
More importantly for our proof, we an prove that for any site,
one an ndananestor that isa direthild.
Lemma 5.For anylegalonguration ,anyglobal pointer GP,and
any site s, if s is an indiret hild of GP's owner, then there exists
a site s
1
suh that s
1
isa diret hild and s
1
is an anestor of s.
Proof. This isa longproof byindutionon thelegaltransitionsand
byaseonthepossibletransitions.Inpartiular,thetransitionsthat
produe oronsume inde messages have theabilityto hange the
diusiontree;theyneeda arefulaseanalysis.Theproof alsorelies
on Lemma4 to guaranteethat we deal witha treeand nota graph.
u t
Intuitively this Lemma speies that if a site s reeives a new
GP from a site that is not the owner, this global pointer had to be
propagated froma sites
1
that isa direthildoftheowner.
5Corretness
Wearenowreadytoestablishthesafetyandlivenessofthealgorithm.
5.1Safety
The safety property guarantees that the referene ounter of a GP
onits ownerisstritlypositiveifGP isaessible remotely.AGP is
said to be aessible on a site ifit ispresent ina site's reeive-table
orifitis present inaopymessage intransit.
Itisnowratherstraightforwardtoderivethesafetyproperty.
Sub-stitutingLemma2into Lemma1,weobtainthevalueof theowner's
send-table.
Lemma 6.Let hsend T;re T;ki bea legal DRC-onguration. The
following property holds:
send T(owner(GP);GP) =
X
si2S;si6=owner(GP)
site weight(s
i ;GP)
withsite weight dened as:
=fINT(reT(s
i ;GP))
+#(fm jm=opy(GP); m2k(owner(GP);s
i )g)
+#(fm jm=de (GP); m2k(s
i
;owner(GP))g)
#(fm jm=in de (GP;s
i
); m2k(s
j
;owner(GP)); 8s
j g) g:
Proof. Theproofanbefoundinleinvariant4.v.Itisimmediately
derivedfrom Lemmas 1and 2. ut
Wean seethattheowner'ssend-tabledependsonthenumberof
remote sites that have aess to the pointer, on the numberof opy
messagesleavingtheowner,onthenumberofdemessagesaimedto
theowner,and on thenumberofin demessages intransit.
Lemma 3establishedthatevery queuek(s
i
;owner(GP))is
alter-nate for GP. It follows that theowner send-table is always positive
ornull.
Lemma 7.Let hsend T;re T;ki bea legal DRC-onguration. For
any global pointer GP, sendT(owner(GP);GP)0.
Proof. Theproofappearsinleinvariant5.v.Lemma6denesthe
owner'ssend-tablevalueasa sum,forwhihwe proveherethateah
summandis positiveornull.UsingLemma3,wean derive thatthe
numberof in de messages ina queue k(s
i
;owner(GP)) isat most
equaltothenumberofdemessagesplus1.Furthermore,itisatmost
equaltothenumberofdemessageswhenreT(s
i
;GP)isfalse.We
therefore onludethat siteweight isalways positiveornull. ut
We arenowreadyto establishthe safety property.
Theorem 1(Safety). Let hsend T;re T;ki be a legal
DRC-on-guration.
8 GP 2G;let s=owner(GP);8s
i 6=s;
if re T(s
i
;GP); then send T(s;GP)>0:
Proof. Theproofofthistheoremmaybefoundinleinvariant8.v.
First, siteweight(s
i
;GP) > 0 for any site s
i
that is a diret hild;
indeed, by denition, the reeive-table of a diret hild is true and
thereisnoin demessage inthequeue k(s
i
;owner(GP))ofadiret
hilds
i
.FromLemma7,weknowthatsite weight isalways positive
or null. We therefore have to prove that, if there is a site s
i suh
thatre T(s
i
;GP),thenthere existsatleastone sitethatis adiret
hild. UsingLemma 5, we know that ifs
i
Thepurposeofthesafetypropertyistoguaranteethattheowner's
send-table is stritly positive when a referene is available in the
distributed system. Theorem 1 proved suh a propertywhen a GP
isexpliitlypresentinasite'sreeive-table.Westillhave toonsider
thease wherethereferene is intransitina opy message.
Theorem 2(Safety2).Lethsend T;re T;kibealegal
DRC-on-guration.
8 GP 2G;8s
i ;s
j 2S;
if opy(GP)2k(s
i ;s
j
); then sendT(owner(GP);GP)>0:
Proof. Theproofofthistheoremmaybefoundinleinvariant8.v.
We an prove that ifa opy message is in transitbetween two sites
s
i and s
j
,thenthe send-tableofs
i
is stritlypositive,whih implies
thatitsreeivetableisalso true.UsingTheorem1,weonludethat
theowner'ssendtable is stritlypositive. ut
5.2Liveness
Liveness guarantees that if all referenes to a GP are deleted, its
owner'ssendtable willeventuallybeome null. Inorder to establish
liveness, we rst show that wheneverthere is a message in a queue,
a transitionmayberedto onsume thismessage.
Lemma 8.Letbea legalongurationhsend T;re T;ki,suhthat
k(s
1 ;s
2
) = fmgxq, for some m;s
1 ;s
2
and q. Then, there exist a
transition t and a onguration 0
= hsend T 0
;re T 0
;k 0
i suh that
7! t
0
, with k 0
(s
1 ;s
2 )=q.
Proof. The proof appears in le liveness.v. It proeeds by ase
analysis onthetypeof themessage m known to be ina queue. ut
Lemma 8 ensures that the algorithm itself does not prevent the
proessing of messages.
Ournext step isto prove that thedistributedreferene ounting
ativity generates a nite number of transitions. We however need
to beverylearaboutwhat wemean bydistributedreferene
ount-ingativity.Thetransitionmake opyisinitiatedbytheappliation,
whih is beyond this algorithm. So, we show that there an only
be a nite number of transitions that do not involve a transition
make opy.
termina-is from terminating its transitions related to distributed referene
ounting. Thetermination measure is denedin terms of ameasure
of thereeivetable and a measure ofmessages.
Denition 6(Termination Measure). The termination measure
of a onguration =hsend T;reT;ki isdened as:
termination measure()= X
GP X
s2S
rt measure(reT(s;GP))
+ X
si2S X
sj2S X
m2k(s
i ;s
j )
msg measure(m);
with
msg measure(opy(GP))=5
msg measure(in de (GP;s))=2
msg measure(de (GP))=1
rtmeasure(true)=2
rt measure(false)=0
Intuitively, a opy message an update a reeive table and reate a
newin demessage,whihitselfmayreateanewdemessage.The
terminationmeasureoftheseeventswasdesignedinsuhawaythat
the measure of an event is bigger that the umulative measure of
ausallydependentevents.
Lemma 9.For any legal ongurations ; 0
and for any transition
t, suh that 7! t
0
, and t 6= make opy(s
1 ;s
2
;GP), the following
inequality holds:
0termination measure( 0
)<termination measure():
Proof. The proofan be foundinleliveness.v.Itproeedsbyan
analysis ofthedierent possibleases fortransitiont. ut
Knowing that the termination measure is positive or null, and
having proved that it dereases for every non make opy transition,
we an derivethe followingterminationLemma.
Theorem 3(Termination).For any legal onguration, all
transi-tion paths that donot involve make opy transitions terminate.
Proof. Theproofappearsinleliveness.v.Letusdenearelation
suessor onthesetoflegalongurations;l
2
isasuessor ofl
1 if
l
2
is obtained from l
1
by a transition that diersfrom make opy.
Using the termination measure (Denition 6) and the fat that it
dereases(Lemma 9), we an establish that thesuessorrelation is
well-founded.Therefore, we an derivethat, foranylegal
Let usonsideragiven globalpointer GP.UsingTheorem 3and
Lemma6,aterminalstateoftheDRC-mahinedoesnotontainany
message related to GP, whih implies that the owner's send-table
valueisequalto thenumberofremotesitesthathavea reeive-table
set to true. In addition, if all sites have red the delete transition,
beause the globalpointer was no longer needed, the owner's
send-table beomeszero. Consequently, ifwe assume fairness[26℄ of
mes-sage delivery,and ifall referenesto a GP arelost, thenits owner's
send-tablebeomesnull,whihproves livenessofour algorithm.
6Loal Optimisations
In this setion, we present two loal optimisations, whih give new
insights to the algorithm. The rst optimisation relaxes the FIFO
onstraintforopy messages,whereastheseond optimisationshows
thatour algorithmdesribesafamily ofdistributedreferene
ount-ing, inludingPiquer'sIndiret RefereneCounting[32℄.
6.1Unordered Copy Messages
The distributed referene ounting algorithm was formalised by an
abstrat mahine, whih assumes FIFO ommuniation queues
be-tweenanypair of sites.We relied on suh apropertyto haraterise
the regular struture of a queue between a site and a GP's owner
(Denition 2). In addition, we know that if de messages were
al-lowed to overtake inde messages, send-tables mayprematurely be
deremented, whihwouldbreakthesafetyproperty.
However, opy messages have a dierent nature than de and
in demessages. Aopymessage representstheappliation ativity
whih ommuniates referenesto remotesites, forinstanethrough
remoteproedureall,whereasthelattermessagesrepresentreal
dis-tributedrefereneountingativity.
Depending on the spei need of the appliation, it may be of
primaryimportaneto proess appliationmessages fasterthan
dis-tributed referene ounting messages. For instane, it is generally
admitted that garbage olletion ativity shouldnot slow down the
mandatoryappliation.
The FIFO handling of messages fores the distributed referene
ounting ativity to proeedsynhronouslywith the appliation.As
suh a behaviour may not be aeptable to some appliations, we
mahine andintroduequeueswhosespeipurposeisto transport
opymessages.Instead,weprefertointrodueanewrulethatallows
anyopy message tobe propagatedindividuallybyanystrategy.
propagate opy(s
1 ;s
2 ;GP;q
1 ;q
2 ;q
3 ;q
4 ):
k(s
1 ;s
2 )=q
1
x fopy(GP)g x q
2 ^ q
1 xq
2 =q
3 xq
4
! fk(s
1 ;s
2 ):=q
3
x fopy(GP)g x q
4 g
Rule propagate opy should be read as follows. If there is a opy
message between two sites s
1 and s
2
with q
1 and q
2
the sequenes
of messages respetively preeding and following the opy message,
the message opy may be positioned at any loation in the queue
between s
1 and s
2
; theonatenation of q
3 and q
4
,the sequenes of
messages respetively preeding and following the opy message in
thetransformedqueue,mustbeequaltotheonatenationofq
1 and
q
2 .
Rulepropagate opyallowsanyopymessageappearinginaqueue
to be putat anyother positioninthat queue,providedtheorder of
theothermessagesremainsunhanged.Suhatransitionallowsopy
messages to be proessed at a dierent speed than other messages.
Notethat thistransition isnot intendedto beeasilyimplementable,
but its purposeis to speifya range of possible behavioursforopy
messages.
Afteradding a new transitionto theabstrat mahine,all proofs
thatuseanindutiononthetypeoftransitionhadtobeextendedto
supportthenewase.NomajordiÆultywasenountered,exeptfor
thealternate queues(Denition2). The denitionhad to be revised
sothat opymessages maybeallowed at anyposition.
Denition 7(Alternate 2). An alternate queue is dened
indu-tively asfollows:
{ q is alternate for GP if it does not ontain messages related to
GP;
{ if q is alternate for GP, so is qxfmg provided that m is not an
inde message related to GP;
{ if q isalternate for GP, so isq x finde (GP;s)g, provided that
thereisade (GP) messageafter the last ourreneof an in de
messagerelated to GP in q,if any.
Weonjeturethatothersimilarloaloptimisationsmaybeproved.
6.2Indiret RefereneCounting
Let us onsider a senario where a opy message was reeived by
s
1
from s
2
,followed bys
1
postingan inde message to the owner;
shortly afterwards, let us assume that s
1
deleted the global pointer
referene, whih resulted in an de message from s
1
to the owner,
immediatelyfollowingthein demessage.There isroomforaloal
optimisationinsuhirumstanes.Indeed,aording totheurrent
algorithm,theindemessagewouldbedelivered,wouldinreasethe
owner's send-table, would be followed by a de message that would
derease thesend-tableons
2
;on theotherhand, theotherde
mes-sage would derease the owner's send-table. In other words, the net
eet of these three messagesisto dereasethesend-table ofs
2 .
Asimilareetmaybeahievedbyasingledemessagefroms
1 to
s
2
diretly.This optimisationmaybeformalisedbyanewtransition
rule.
rediret in(s
1 ;s
2 ;GP;q
1 ):
k(s
1
;owner(GP))=q
1
xfin de (GP;s
2
)g x fde (GP)g
! fk(s
1
;owner(GP)):=q
1 ;
k(s
1 ;s
2
):=k(s
1 ;s
2
) xfde(GP)g g
The new rulesatises theinvariantsformalisedinLemmas 1and
2; furthermore, it is also safe beause the safety Theorems 1 and 2
are stillvalid. However, this innoent hangein surfae had quite a
deep reperussion on the proof. Indeed, rule rediret in potentially
hangesthediusiontreeasitonsumesthelastindemessageofa
queue.Rulerediret inisuniqueinthealgorithmbeauseitextrats
messagesfrom theendof thequeue and notits beginning.
Inpartiular,Lemma4,andonsequentlyLemma5,ouldnotbe
derivedimmediatelyinpreseneofthenewrule.Wehadtogeneralise
Denition3 and introduea notionof multiplediusion.
Denition 8(Multiple Diusion). Given a legal onguration
and a GP, the prediate multiple diuse(;GP;s
1 ;s
2
) holds if
re T(s
2
;GP) = true and there is a message in de (GP;s
1
) in the
queue k(s
2
;owner(GP)).
Denition8diersfromDenition3beauseitregardsall in de
messages as indiators of the diuse relationship,as opposedto the
lastoneonly.Wedeneanmultiple anestor asthetransitivelosure
Lemma 10(Not Reexive 2). For any legal onguration , for
any sites s
1 ;s
2
, if multiple anestor(;GP;s
1 ;s
2
), then s
1 6=s
2 .
Proof. Proofappearsinleinvariant6.vandproeedsbyindution
on thelegaltransitionsand byaseon thepossibletransitions. ut
The multipleanestor relationis a supersetof the anestor
rela-tion.Therefore,fromLemma 10,wean derivethatLemma4isstill
validinthepreseneof rulerediret in.
Let us observe again that rediret in is not intended to be
eas-ily implementable, but its purpose is to speify a new behaviour of
the abstrat mahine. Indeed, in terms of implementation, it seems
diÆultto rediretmessages thatwere alreadysent.
Morerealistially,thisrulemaybeimplementedasfollows.Instead
of sending an inde message when a new GP is reeived, one an
assoiate the GP with a \redireting information", ontaining the
sitethatsentit. When ade message hastobe sent tothe owner, it
hasto beredireted ifsome rediretinginformationisavailable.
Inreality, suh a systemati avoidane of in de messages is
Pi-quer's Indiret Referene Counting algorithm [32℄. We an see our
algorithm as an abstrat speiationof a family of distributed
ref-erene ounting algorithms. At one endof thespetrum,we nd
Pi-quer's Indiret Referene Counting (IRC) that does not use in de
messages at all. At the other end of the spetrum, we nd an
al-gorithm that eagerly sends inde messages in order to atten the
diusion tree. In between those extremes, there is a range of
im-plementationstrategies, whih ombine both IRC and diusiontree
attening.
IndiretRefereneCountingforeseahparenttomaintaina
send-tableentryforeahglobalpointerpassedtoitshildren,untilhildren
have ompletely released the referenes to this pointer. This may
result in \zombie pointers" [33℄, where the pointer is only kept live
onasitebeause itisneededina send-table.Thisinfatresultsina
formofmemoryleak,whihmaybeavoided bytheuseofthein de
message.
7Algorithm Variants
In thisSetion, we onsidertwo variants of thealgorithm. (i)The
rst one handles messagesto the owner dierently,so thatde
somefundamentalpropertiesofthealgorithm,whihwedisusshere.
(ii)Theseond variantof thealgorithm usesreferenelisting,whih
notonlyountsthe numberof timesreferenesare opiedto remote
sites,butalso remembersthesites wheretherefereneswere opied.
Referene listing is a tehnique that is usefulto assist in deninga
fault-tolerant version ofthealgorithm.
7.1NoCopy to the Owner
Amake opytransitioninreasestheemitter'ssend-table.Iftheopy
messageisemittedtotheowner,itwillbefollowed byademessage
bakto theemitter,whihwillderease its send-table.Thissenario
ould be optimised: if we do not inrement the send-table before
sending a opy-message to the owner, we an avoid sending a de
message bakto theemitter.
We have investigatedthisapproah, whihrequires an extra
pre-onditionintheguardof rulemake opy.
make opy(s
1 ;s
2 ;GP):
s
1 6=s
2 ^ s
2
6=owner(GP) ^ re T(s
1 ;GP)
! fsend T(s
1
;GP):=send T(s
1
;GP)+1
post(s
1 ;s
2
;opy(GP)) g
Rulemake opymayberedonlywhens
2
isdierentthantheowner.
Letusobserve thatthisruleismore radial thanthedesriptionwe
justgave. Indeed,inthisalgorithm,wenolongersendopymessages
to the owner at all. Let us remember that opy messages represent
the informationthat must be ommuniated to ouralgorithm when
referenes are opied between sites. The absene of opy messages
to the owner does not prevent an implementation from performing
remote proedure allsto theowner, butit simplyindiates thatno
information has to be passed to the distributed referene ounting
module in suh irumstanes.We deided to adopt suh a rule
be-auseitfailitatestheproof;ifwehadaeptedopy-messagestothe
owner without inreasing the send-table, we would have had to
in-trodue anullweight forthese messages, whih wouldhave required
longerase analysesintheproofs.
Theinvariant Lemmas 1and 2 and thesafety Theorems1 and 2
are all valid for thisalgorithm, without any major dierene in the
proofsthemselves.
However,propagate opyindiatedthatappliationmessagesarrying
referenesdidnothaveto besynhronisedwithdistributedreferene
ounting messages. Thispropertyisnolongervalidwiththeurrent
algorithm,and we give aounter example.
Letusonsidertwo sites:theownerofaGP ands.Letusassume
thatthesend-tableofsisnull.SitessendsaopyGP witharemote
proedure allto the owner, and immediately afterwards deletes its
referene of GP, whih generates a demessage to the owner. If the
remote proedure all is delayed, the de message an derease the
owner'ssend-table,whihbeomesnull,whereasarefereneisstillin
transit. Suh a senario would have been impossible in the original
algorithm,beauseshadto inreaseitssendtable whensendingthe
opy message,whihpreventedsto rethe deletetransition.
It does not imply that this variant of the algorithm is less
use-ful than the previous one. FIFO order must be stritly followed in
ordertopreservesafety,andtheappliationwillditateifsuha
on-straint is aeptable. We onjeture that some asynhronism is still
permitted:itis always safe to proess a opy message early, beause
it inreases referene ounters; symmetrially, de messages may be
proessedlater.
7.2Referene Listing
In orderto denea fault tolerant version ofthe algorithm,itis
on-venient to maintain not only a ounter representing the number of
times referenes were opied, but also the sites to whih they were
sent.
Thestatespaehastobehangedaordingly.Send-tablesrequire
anextraargumentrepresentingthesitewhereaglobalpointerissent
to. In addition,de and in demessage onstrutors take one more
argument, whih isthesite-entry of asend-table they operate on.
M=opy:G!M j de:GS !M (Setof Messages)
j in de:GSS !M
ST =SGS !Z (Set ofSendTables)
Figure 10 displays the transitions. Rule make opy updates the
table on site s
1
, for an entry identied by GP and s
2
. Similarly,
rulesreeive in deand reeive de update theentry ofa send-table
indexed by the new site ontained in the reeived message. Other
hangesare similar.
Givenaonguration=hsendT;reT;ki,ve transitionsare per-mitted: make opy(s 1 ;s 2 ;GP):
s
1 6=s
2
^ re T(s
1 ;GP)
! fsend T(s
1 ;GP;s
2
):=sendT(s
1 ;GP;s
2 )+1
post(s
1 ;s
2
;opy(GP)) g
reeive opy(s
1 ;s
2 ;GP):
first(k(s
1 ;s
2
))=opy(GP)
! freeive(s
1 ;s
2 )
ifre T(s
2
;GP) then
fpost(s
2 ;s
1
;de (GP;s
2 ))g
else
freT(s
2
;GP):=true
post(s
2
;owner(GP);in de (GP;s
1 ;s 2 )) if s 1 ;s 2
6=owner(GP) g g
reeive inde(s
1 ;s
2 ;GP;s
3 ;s 4 ): first(k(s 1 ;s 2
))=inde (GP;s
3 ;s
4 )
! freeive(s
1 ;s
2 )
send T(s
2 ;GP;s
4
):=sendT(s
2 ;GP;s
4 )+1
post(s
2 ;s
3
;de (GP;s
4 )) g
reeive de (s
1 ;s
2 ;GP;s
3 ): first(k(s 1 ;s 2
))=de (GP;s
3 )
! freeive(s
1 ;s
2 )
send T(s
2 ;GP;s
3
):=sendT(s
2 ;GP;s
3
) 1 g
delete(s;GP):
8s
j
; sendT(s;GP;s
j
)=0; reT(s;GP);owner(GP)6=s
! fre T(s;GP):=false
post(s;owner(GP);de(GP;s)) g
Denition 9.Let k bea setof queues ofa DRC-mahine
ongura-tion. Let s
i ;s
j
betwo site of S.Theset of messages underontrolof
s
i via s
j
, written ontrol(GP;s
i ;s
j
), isdened as:
ontrol(GP;s
i ;s
j )
= f mj m=opy(GP);m2 k(s
i ;s
j );
m=de (GP;s
j
);m2 k(s
k ;s
i ) or
m=in de (GP;s
i ;s
j
); m2k(s
k
;owner(GP))
for any s
k g:
The numberofmessagesunderontrolofs
i vias
j
ispreiselythe
valueof thesend-tableof s
i
,formessagessent to s
j .
Lemma 11.Lethsend T;re T;kibealegalDRConguration.The
followingpropertyholds.For anyglobalpointer GP 2G,foranysites
s
i ;s
j 2S:
sendT(s
i ;GP;s
j
)=#(ontrol(GP;s
i ;s
j )):
Proof. The equality is initiallytrue and is preserved byeah
transi-tion.The aseanalysis isavailableininvariant2.v. ut
Other properties suh as safety and liveness still hold for this
al-gorithm. The algorithm presented here ombines referene ounters
and referene listing.By using referene listing,Birrel etal. [4℄ and
PlainfosseandShapiro[34℄ mademessagesidempotentandtherefore
resistent to message failure.
8Related Work
8.1Comparison with OtherRelated Mehanial Proofs
Jakson [17℄ hasveriedtheorretnessof agarbage olletion
algo-rithmusingthePVStheoremprover.Thealgorithmthatwasstudied
isastop-and-ollet,nonopyingolletor.ItusesDijkstra,Lamport,
Martin, Sholten, and Steens'[7℄ triolourmarking sheme,butno
onurreny(ordistribution)wasallowedinthealgorithm.The
algo-rithm wasformalisedasa labelledtransitionsystem. An embedding
of linear temporal logi in PVS was used for reasoning. Safety and
livenessproperties,similarto ours,werederived forhisalgorithm.
Goguen, Brooksby and Burstall [11℄ present an abstrat
formu-lation of memorymanagement based on a graph-theoreti
represen-tationofmemoryand relatedoperations.Theyalso formalised
Russino[36℄ usedtheBoyer-Moore theoremprovertoverifythe
safetyandlivenesspropertyofBen-Ari's[2℄mark-and-sweepgarbage
olletion algorithm. Ben-Ari's algorithm is a two olour solution
to Dijktra et al's initial problem. He proves that a state prediate
remains invariant, i.e. true for all reahable states. Havelund and
Shankar [14℄ use renement tehniques to prove the safety of
Ben-Ari's algorithm,inPVS.
Gonthier and Doligez [8,13℄ proved the safety of a onurrent
garbage olletorusedinCaml-light.Theproofwasarriedoutwith
theLarhProver.
8.2Referene Counting Algorithmsfor Garbage Colletion
Referene-ountinggarbageolletionwasinitiallydevelopedfor
uni-proessorsystems [5℄.Itsprinipleisasfollows:every timea pointer
is opied or deleted, a referene ounter is respetively inremented
or deremented. It might seemthat this algorithm an be extended
straightforwardly to distribution by using two typesof messages. A
derement message is sent to GP's ownerwhen GP is disarded;an
inrement message is sent to GP's owner when GP is dupliated.
However, thisnave extension failsto behave properly beause
non-ausal [20℄ message delivery may reset the ounter even though
re-mote referenesmay stillbe ative.
Numeroussolutionstothisproblemhavebeenproposed.Themost
famous are weighted referene ounting [3,41,9℄ and its optimised
version [6℄, generational referene ounting [12℄,or Piquer's Indiret
RefereneCounting[32℄, whih wehave already disussedinSetion
6.2. However, Lermen andMaurer's [23,40℄and Birrel's[4℄ solutions
are the losest to our work; we present them in the next two
para-graphs.
In Lermen and Maurer's algorithm [23,40℄, when a GP is
dupli-ated, a reate message is sent to its owner. The owner then sends
anaknowledgement totheGP'sreeiver.WhenaGP isdisardeda
derement message is sent onlyafter theaknowledgement hasbeen
reeived for this pointer. Lermen and Maurer's tehnique also
in-volves three sites (emitter, reeiver, and owner), but it diersfrom
ours: (i) The owner is involved every time the emitter dupliates
a GP to thereeiver in Lermenand Maurer's algorithm,whereas it
is involved only if the GP is not aessible on the reeiver in our
ofaknowledgements reeived. Derementmessages anonlybesent
when bothare equal.
Birreletal.[4℄presentnetworkobjets,adistributedobjet-based
languagewithagarbage olletor.The ownerofan objetmaintains
a \dirty" set, whih ontains identiers for all the proesses that
have GPs to theobjet. When a lientrst reeives a GP,it makes
a dirty all to the owner. When the GP is no longer reahable, as
determinedbythelient's loal g,thelientmakesa lean alland
deletesGP.Withthedirtyalls,Birreletal.reinstatetheequivalent
of an inrement message. In order to avoid onits between dirty
and lean alls,an aknowledgement message from the reeiverof a
GP to its emitter guarantees theimpossibilityof freeingthe pointer
on the emitter; the atualimplementation preventsthe method all
fromterminatingontheemittertilltheaknolwedgementisreeived.
In Birrel's algorithm, distributed referene ounting ativity is
synhronous withthe appliation. In partiular, unmarshallingmay
besuspendedbydirtyalls.Furthermore,theemitterofaGP isonly
allowed to free its referene after the method invoation has
termi-nated on the reeiver: this may potentially reate a zombie pointer
forthedurationoftheomputation.Ouralgorithmrequiresless
syn-hronisationswiththeappliation;itismoreexiblesine,fullylazy,
it behaves as indiretreferene ounting, and fully eager it behaves
more likeBirrel's;theonlydiereneisthatouraknowledgement is
sentbytheownerintheformofaderementmessageand notbythe
reipient ofthe referene.
ThedistributedvariantoftheTrainGC[15℄isalsoabletoollet
yles;it ombines areferene-ounting stylepointer-traking
meh-anismwitha substitutionprotool.The latter algorithmbearssome
resemblane with Birrel's but minimizes the number of exhanged
messages: asa onsequene, the owner of a GP may not be able to
nd(diretlyorindiretly)all thesites thathave a opyofthe GP.
Ouralgorithmhasamajorbenetasitisabletoreorganise
diu-sion trees: when GCmessages are all proessed, thediusion tree is
ompletelyattened,andeverysiteowningaGP diretly \depends"
fromitsowner.Inthepreseneofmobileomputationsjumpingfrom
sitetosite,thisallowssitestorelaimthespaethatwasoupiedby
a mobile program, hereby avoiding zombie referenes as in indiret
refereneounting[32℄.Tothebestofourknowledge,Shapiro,Gruber
and Plainfosse [39℄, and subsequentlyShapiro, Dikman, and
Plain-fosse [37,38,34℄ were the rst to address the issue of short-utting
when sending the referene of a loal objet, or when migrating an
objet to some other site. In addition, they propose a tehnique to
short-utSSP-hains,herebyavoidingtheequivalentofzombie
refer-enes. They regardmigration as a primitive notionto be supported
by the GC; in this paper, we do not deal with migration, however,
wehaveshowedthatsupportformobilityouldbeaddedasanextra
layer, like a library, on top of the our garbage olletion algorithm
[27℄.InShapiro'salgorithm,hainsofpointers areollapsedinasafe
fashion by side-eet on remote invoations;speially,
piggybak-ing new loation information onto invoation results, and loation
exeptionraisingare usedto thisend. Garbage olletiontakes are
of leaningobsolete indirethains.
Weighted referene ounting (WRC) [3,41,9℄ assoiates a weight
with eah objet and pointer. It maintains the invariant that the
weightofanobjetisequaltothesumofweightsofpointerspointing
toit.Whenapointerisopied,itsweightis(equally)dividedbetween
the two opies. When the weight of a pointer reahes one, several
solutionsarepossible. (i)Anindiretionellmaybeintrodued,but
itbehavesasa\zombiepointer"asinPiquer'sIRC. (ii)Amessage
may be sent to theownerin order to request formore weight. Suh
a message may be regarded as a form of inde message, and we
ould see Weighted Referene Counting as a systemati method to
deide when in de messages must be sent. Whenever a pointer is
deleted, the objet weight must be updated, whih involves sending
a \derement" message to theowner.
ManiniandShrivastava[25℄investigateafault-tolerantversionof
distributed referene ounting. They also onsider a triangular
pro-tool, betweentheowner, thesenderand thereeiverof a referene,
whih diers from ours. It is an open question whether their
fault-tolerantextension areappliableto ouralgorithm.
In[28℄,weinvestigatethesalabilityofreferenelistinginthe
pres-ene of massively distributed omputations. The size of send-tables
is proportionalto the number of sitespartiipating to the
omputa-tion.In order to redue theburden on individualsites,we presenta
hierarhialorganisation of the referene listing algorithm bywhih
we areable to give aboundto thesizeofsend-tables.
JAVARemoteMethodInvoationomeswithadistributedgarbage
olletor[18℄.ItextendsBirrel'sreferenelistingtehniquewithanew
approah to fault tolerane, where remote pointers are leased for a
listingsothat send-tables ontain thesites to whih GPs weresent,
and a similarleasetehniqueould also be adopted.
Refereneountinggarbageolletionisonlyabletorelaimayli
datastrutures.Severalauthorshaveombineddistributedreferene
ounting withother algorithmsto provide yligarbage olletion;
forinstane, Le Fessant,Piumarta, and Shapiro[22℄, Rodriguesand
Jones [35℄,Linsand Jones [24℄,Lang,QueinneandPiquer's [21℄,or
Hudson etal'sDistributedTrainGC [15℄.
9Conlusion
We have presented an algorithm for distributed referene ounting
and its proofof orretness, whihinvolvesbothsafety andliveness.
We used the Coq proof assistant to formalise this proof. This work
was a major undertaking, but gave valuable insights to the proof,
whihhad beenoverlookedintherstplae,duringthepaperproof.
A number of related issues are worth onsidering now. Support
formobileobjetsinonjuntionwithdistributedrefereneounting
wouldprovideanexellentspeiationthatouldbeusedtoertify
mobileagents. Extendingthereferene listingalgorithm with
times-tamps would make thealgorithm resilient to faults. Finally,proving
thehierarhialvariantofthealgorithmwouldbeausefulexerisein
ordertobuildorretmassivelydistributedomputingenvironments.
Aknowledgements. The oding of the proof in Coq was initiated during Lu
Moreau'sstayatEoleNormaleSuperieure,Lyon;LuMoreauwouldliketothank
LuBouge for making this visitpossible, as well as Eole Normale Superieure,
LyonandINRIARhones-Alpesfortheirsupport.ThankstoAndyGravellforhis
ommentsonthepaper.
Referenes
1. B. Barras, S. Boutin, C. Cornes, J. Courant, J.C. Filliatre, E. Gimenez,
H.Herbelin,G.Huet,C.Munoz,C.Murthy,C.Parent,C.Paulin,A.Sabi,
andB.Werner. TheCoqProofAssistantRefereneManual {VersionV6.1.
TehnialReport0203,INRIA,August1997.
2. MordehaiBen-Ari.Algorithmsforon-the-ygarbageolletion.ACM
Trans-ationsonProgrammingLanguagesandSystems, 6(3):333{344,July1984.
3. DavidI.Bevan.DistributedGarbageColletionusingRefereneCounting.In
PARLE ParallelArhiteturesandLanguagesEurope,volume259ofLeture
NotesinComputerSiene,pages176{187.Springer-Verlag,June1987.
4. AndrewBirrell,DavidEvers,GregNelson,SusanOwiki,andEdward
Wob-ber. DistributedGarbageColletion forNetworkObjets. TehnialReport
5. GeorgeE.Collins. AMethodforOverlappingandErasureofLists.
Commu-niationsoftheACM,3(12):655{657,Deember1960.
6. PeterDikman. OptimisingWeighted RefereneCountsfor Salable
Fault-TolerantDistributedObjet-Supportfsystems,1992.
7. EdsgerW.Dijkstra,LeslieLamport,A.J.Martin,C.S.Sholten,andE.F.M.
Steens. On-the-ygarbageolletion:Anexeriseinooperation.
Commu-niationsoftheACM,21(11):966{975,November1978.
8. DamienDoligezandGeorgesGonthier. Portable,UnobtrusiveGarbage
Col-letion for Multiproessor Systems. In ACM Conferene on Priniples of
ProgrammingLanguages,pages70{83,1994.
9. IanFoster.AMultiomputerGarbageColletorforaSingle-Assignment
Lan-guage. IntlJ.ofParallelProgramming,18(3):181{203,1989.
10. IanFoster,CarlKesselman,andStevenTueke.TheNexusApproahto
In-tegrating MultithreadingandCommuniation. Journal ofParalleland
Dis-tributed Computing,37:70{82, 1996.
11. Healfdene Goguen, Rihard Brooksby, and Rod Burstall. An
Abstrat Formulation of Memory Management. Available from
http://www.ds.ed.a.uk/ hhg/,Deember1998.
12. Benjamin Goldberg. Generational Referene Counting: A
Redued-CommuniationDistributedStorageRelamationSheme.InSIGPLAN
Pro-gramming Language Design and Implemantation PLDI'89, pages 313{320,
1989.
13. GeorgesGonthier. Verifying the Safety of aPratial Conurrent Garbage
Colletor.InR.AlurandT.Henzinger,editors,ComputerAidedVeriation
CAV'96,number1102inLetureNotesinComputerSiene,pages462{465,
1996.
14. K.HavelundandN. Shankar. Amehanizedrenementprooffor agarbage
olletor. Availablefromhttp://www.s.au.dk/ havelund/,1997.
15. R.L.Hudson,R.Morrison,J.E.B.Moss,andD.S.Munro.GarbageColleting
theWorld:OneCarataTime.InProeedingsofOOPSLA'97,Atlanta,USA,
1997.
16. G. Huet, Gilles Kahn, and Christine Paulin-Mohring. The Coq Proof
Assistant | Tutorial. Tehnial report, INRIA, 1999. Available from
oq.inria.fr.
17. PaulB.Jakson. Verifyingagarbageolletionalgorithm. InProeedingsof
11th International Conferene on TheoremProving in Higher Order Logis
TPHOLs'98,volume1479 ofLeture NotesinComputer Siene,pages225{
244.Springer-Verlag,September1998.
18. JavaRemoteMethodInvoationSpeiation,November1996.
19. RihardJonesandRafaelLins.GarbageColletion.AlgorithmsforAutomati
DynamiMemoryManagement. Wiley,1996.
20. LeslieLamport. Time, Cloks,and theOrderingofEventsinaDistributed
System.CommuniationsoftheACM,21(7):558{565,July1978.
21. BernardLang,ChristianQueinne,andJosePiquer.GarbageColletingthe
World. InProeedings of theNineteenth Annual ACM SIGACT-SIGPLAN
Symposium on Priniples of Programming Languages, pages 39{50,
Albu-querque,NewMexio,January1992.
22. Fabrie Le Fessant, Ian Piumarta, and Mar Shapiro. A
De-tetion Algorithm for Distributed Cyles of Garbage. In
