Local Domain. Other. Autonomous Domains. Local Host (DUT) ISP. Remote Host. Co Located Co Networked Bottleneck. Hosts. Network Elements M2 M3

(1)

Karthikeyan Bhargavan Universityof Pennsylvania [email protected] SatishChandra BellLaboratories [email protected] PeterJ. McCann Bell Laboratories [email protected] CarlA. Gunter Universityof Pennsylvania [email protected] Abstract

We consider the problem of monitoring an interactive

de-vice, suchas animplementation of anetworkprotocol, in

orderto check whether its execution is consistent with its

specication. Atrstglance,itappearsthatamonitorcould

simplyfollowtheinput-outputtraceofthedeviceandcheck

itagainstthespecication. However,ifthemonitorisable

toobserveinputsandoutputsonlyfromavantagepoint

ex-ternaltothe device|asis typicallythecase|theproblem

becomes surprisingly diÆcult. Thisis because eventsmay

bebuered,andevenlost,betweenthemonitorandthe

de-vice,inwhichcase,evenforacorrectlyrunningdevice,the

traceobservedatthemonitorcouldbeinconsistentwiththe

specication.

Inthispaper,weformulatetheproblemofexternal

monitor-ingasalanguagerecognitionproblem. Givenaspecication

thataccepts acertain languageofinput-outputsequences,

wedeneanotherlanguagethatcorrespondstoinput-output

sequencesobservableexternally. Wealsogiveanalgorithm

to check membership of a string in the derived language.

Itturnsoutthatwithoutanyassumptionsonthe

specica-tion,this algorithm may takeunboundedtime and space.

Toaddressthisproblem, wedene aseries ofpropertiesof

device specications or protocols that canbe exploited to

constructeÆcientlanguagerecognizersatthemonitor. We

characterizethesepropertiesandprovidecomplexitybounds

formonitoringineachcase.

Toillustrateourmethodology,wedescribepropertiesofthe

InternetTransmissionControlProtocol(TCP),andidentify

featuresoftheprotocolthatmakeitchallengingtomonitor

eÆciently.

1 Introduction

Computernetworkingprotocolshavealwaysbeenappealing

candidates for applications of automata theory. Not only

are protocols commonly specied as nite-stateautomata,

much of the current technology of implementing and

ver-ifying protocols relies on application of automata theory.

ExamplesincludeSpin/Promela[5],Verisoft[4 ],Esterel[1 ],

etc.

Weconsidertheproblemofmonitoringtheexecutionof

net-workprotocols. Suppose agivenpiece ofcomputer

equip-mentclaimstoimplementacertainnetworkprotocolthrough

oneofitscommunicationinterfaces. Weseektointroducea

passivemonitoroutsidethisinterfacethatcanwatchallthe

bitstravelingtoandfromthedeviceundertest,andcheck

themforsomeproperties. Suchamonitorcouldgive

impor-tantinformationabouttheproperrunningoftheprotocol.

Monitoring is complementary both to the implementation

andtothevericationofnetworkprotocols.

Apassivemonitorwouldessentiallymimictheactionsthat

thedeviceisexpectedtotakeinresponsetotheinputthatit

receives,exceptitwouldnotactuallyputanyoutputonthe

wire. Rather,itwouldpickuptheoutputgeneratedbythe

deviceundertest,andcomparetheoutputthatitcomputed

against theoutput itsniedfrom the wire. Inthis role, a

passivemonitorisalanguagerecognizer,wherethelanguage

thatitunderstandsisthe input-outputbehavior ofagiven

networkingprotocol.

Passivemonitorscanobservemanyusefulpropertiesof

real-world protocols. We give examples of such properties for

the InternetprotocolTCP inSection6. Sucha capability

canplay ausefulroleintestingnewimplementations, and

alsoinguardingagainstnetworkintrusionandothersecurity

violations.

Severalcomplications hamperourabilityto construct

pas-sive monitors accurately and eÆciently in the real world.

Themostimportant complicationisthedelityof

observa-tion of traÆc by the monitor. In general, our monitor is

notlocated exactlyatthedevice, inthesensethat itdoes

notsynchronouslyobserveinput andoutputactionsof the

automatoninsidethedevice. Therefore,themonitormight

failtoobservecertainpacketsthatthedevicesees,ormight

see certain packets that the device fails to see. Also,

be-causeofbuersattheinputandoutputportsofthedevice,

the monitor might observe a sequence of events in a

dif-ferent orderthanthe device. Anothercomplicationis that

of under-specicationof protocols. For example, theTCP

specicationallowscertainleewayastohowoftenareceiver

mustgenerateanacknowledgmenttoadatapacket. Because

of thesecomplications,theinput-outputlanguagethat the

monitormustrecognizecouldbesignicantlydierentfrom

thelanguagethatagivendeviceprocesses,orclaimsto

(2)

monitoringasalanguagerecognitionproblem,while

incor-poratingthesereal-worldcomplicationstotheextent

possi-ble. Furthermore,thealgorithmswepresentaresuitablefor

on-linemonitoringofprotocolexecution,thatis,wedonot

collecttraces andanalyzethemseparately. Thus, speedof

monitoringisalsoanimportantconcern.Mostpriorworkin

networkmonitoringeitherperformso-linemonitoring[11 ],

ordeals withreal-worldproblemsinad hoc ways[12 ]. We

believeanautomata-theoreticapproachhelpsusachieve

ro-bustalgorithmsandalsoletsusunderstandthekindof

prop-ertiesthatwecanorcannotmonitoreÆciently.

In the general case, we could perform on-line monitoring

using an expensive brute-force search algorithm, which is

essentiallyanineÆcient\parser"forthe\realworld"version

ofthelanguage. Insomecases,thereisnotevenanyapriori

upperboundontheamountofspacethisprocessmaytake.

However, for many kinds of properties that we may wish

tocheck,itispossibletocreatefasterlanguagerecognizers.

Inthis paper, wealso give asystematicexploration ofthe

spaceofpropertiesthatletsusconstructeÆcient monitors.

Wedrawananalogytotheproblemonconstructingparsers

for the syntax of programming languages. In general, a

particular syntaxcould be acontext-freegrammar.

How-ever, parsers for generalcontext-freegrammarsare

expen-sive: theymay take O(n 3

) time in the size of the input.

Thus, most real syntax specications require restrictions

on the grammar, such as the language should be LL(k),

LALR(1), etc. For grammars that obey suchrestrictions,

fastparsers canbe constructed. Wehavedened an

anal-ogous range of languages for monitoring. If the language

of the protocol obeyscertain properties, eÆcient

monitor-ing programs canbe created. These language

characteris-ticscouldbeusedduringthespecicationprocess,toselect

propertiesthatcanbeeasilymonitoredatrun-time,oreven

duringtheprotocoldesignphase,ifthedesignerisespecially

concernedthataprotocolbeamenabletoeÆcient

monitor-ingasmightbethecasewithcriticalsecurityprotocols.

Thecontributionsofourworkare thefollowing:

We deneautomatasuitablefor networkmonitoring,

taking into account real-world complications such as

bueringandpacketloss.

Weclassifypropertiesofnetworkprotocolsthatletus

constructeÆcientmonitors,andalsopointoutcasesin

whichmonitoringwouldnecessarily beanintractable

problem.

Foranimportantreal-worldprotocol,TCP,weidentify

properties that canbeeÆcientlymonitoredbasedon

thetheorythatwedevelop.

Therestofthepaperisorganizedasfollows: inSection2,we

giveadescriptionofthekindsofmismatchesthatcanbe

ex-pectedbetweenatracecollectedbyamonitorandthetrace

actuallyseenbyadeviceundertest. InSection3we

char-acterizethesedierencesformally,andinSection4,wegive

a brute-force approach to checking whether a given trace

collected at a monitor could have resulted from a correct

execution. Weproceed inSection5to give special

proper-tiesof specications that, whensatised, allow us to

con-structmoreeÆcienton-linepropertymonitors. InSection6

we show howour techniques could be appliedto

monitor-ingreal propertiesof TCP.Finally inSection7we discuss

connectionstorelatedworkandconclude.

Inthecontextofmonitoring,thetermdelityrefersto the

closenesswithwhichthesequenceofinputandoutputevents

seenbythedeviceundertestmatchesthesequenceofevents

observedby themonitor. Theextentof indelity is

deter-minedbytheperformanceofthemonitoranditsplacement

inthenetwork. Aperfectdelitymonitor,onethatsees

ex-actlytheinputsandoutputsofthedeviceundertest,canbe

obtainedbyinstrumentingtheprotocolstackonthedevice

itself. Suchamonitorisdepictedas M1inFigure1. Such

a co-located monitorobservesinput and outputactions of

thedevicesynchronouslywiththedevice|itencountersno

delityproblems.Suchmonitorsarehardtodeploybecause

oftheneedtoputnewsoftwareonthemonitoredsystem.

An alternative is to place the monitor somewhere in the

networkandobserveinputsandoutputsastheypassacross

the network links. While there are many possible points

at which a monitor could be placed (see Figure 1), a

co-networkedmonitorandabottleneckmonitorareparticularly

useful, as they are able to observe all traÆc between the

device and the remotehost. A co-networked monitor sits

outsidethedeviceonthephysicalnetworktowhichits

net-workinterfacecard(NIC)isattached,whereasabottleneck

monitorsitsoutsidetheNICof thegatewayofalocalarea

network. Ofthetwo,aco-networkedmonitorcanenjoy

bet-ter delity, as thereis nonetworkelement between it and

thedevice.

Wehaveconductedinitialexperimentstodeterminethe

ex-tent and nature of indelities for co-networked monitors.

Our experiment consistedof taking a PC (400 MHz

Pen-tium II) running Linux and using it to monitor input to

another similar Linux PC over 100 Mbps Ethernet. The

monitoracceptedtraÆcoanEthernethubthatwe

intro-ducedbetweenthedeviceandtherestofthenetwork. Our

experimentsshowthat withproperoperatingsystem

engi-neering,a co-networkedmonitorcouldeasily keepup with

inter-packetarrivaltimeofabout 20microseconds,

assum-ing noexpensivecomputationis performedperpacket. At

thesespeeds,it isfeasibleto monitortypicalTCP/IP

pro-tocoltraÆc over100 Mbps Ethernet,without themonitor

droppinganypackets. Moreover,thisisachievedusingonly

stockhardware(desktopPC'sandanEthernethub 1

);extra

hardwaresupportcouldhandleheavierloads. Consequently,

wecanassumetheexistenceofaco-networkedmonitorthat

doesnotdroppacketsforthiskindofhostandnetwork.

Theprimarydelitychallengesariseindealingwith

buer-ingonthedeviceitself. IfaprotocolS isbeing runbythe

device, thenthe goal of the monitorM is to determine if

S isproperlyimplemented. However,thismustbedoneby

observingbehavioronthenetwork,andthereareinputand

outputbuersbetweenthedeviceandthenetwork. The

sit-uationisdepictedinFigure2. Theinputandoutputbuers

may over-ll and cause packet losses between the device

and the network, thus introducing indelities between the

eventsobservedat the device, and theco-networked

mon-itor M. Notethat the over-llmay becaused by packets

fromsomeotherprotocolthatthedeviceisengagedin;the

monitormayneverevenlookat thesepackets. Our

experi-mentsshowedthat underheavynetworkloadsthiskindof

indelitywasquitepossible forinputbuers. However,the

1

In this sense it is not exactly co-networked asdened above,

butweassume thishubrepeatsalltraÆctoallports. Atruly

(3)

Other

Autonomous

Domains

Local

Host

(DUT)

Autonomous Domains

Hosts

Network Elements

Remote

Host

M2

M3

M4

M1

Local Domain

Co−Located

Co−Networked

Bottleneck

ISP

M6

M5

A

G

Figure1: MonitorPlacement: MonitorM1isco-locatedwiththedeviceundertest(DUT);itisunfeasibletodeploywithout

modifying DUTsoftware. Monitor M2 is co-networked and monitorM4 is at a bottleneck location. Monitor M3 will not

observetraÆcpassingthroughnetworkelementA. M5islocatedatanInternetserviceprovider(ISP),andM6islocatedin

anotherserviceprovider'snetwork. Because theInternetProtocolmaydrop,duplicate,or re-orderdatagrams,M5 andM6

canexperiencesignicantindelities.

S

M

S

M

d

c

NIC

Device Under Test (DUT)

Input Buffer

Output Buffer

NIC

Device Under Test (DUT)

Input Buffer

Output Buffer

e

c e

b

a

Figure2: Buers inthe DUT.M may observeinputsand

outputsinadierentorderthanS. Thesecondgureshows

onepossibleexecutionsequenceattheDUT.

kernelwill ordinarily throttlethe protocolimplementation

toavoidlossinoutputbuerssowewillassumeinthispaper

thatoutputsarenotlostatadevice'soutputbuer.

Asidefrominputbuerloss,themajorindelitybetweenthe

deviceandaco-networkedmonitorM arisesfromdierent

perceptions of packet arrival and dispatch caused by the

inputandoutputbuering. Wedescribethisphenomenain

somedetailinthenextsection. Toseetheissuebriey,note

thatitisimpossibleforMtotellinFigure2whetheroutput

dofS wascreatedbythedevicebeforeor afterthedevice

observed a or b, or even whether b was droppedbefore it

reachedS.

3 Model

Weassumethatthedeviceundertestisadeterministicand

reactive automaton, following a specication S. At each

step, eitherit consumesan input, or produces anoutput.

Outputsmaybeproducedinreactiontoinputs,orin

reac-tiontoeventssuchastimerexpirations,thatarenotvisible

to the outside. Ourmodel of S is similar to a

determin-istic I/O Automaton[9 ]; it is a machine with a (possibly

innite)state space carryingout parameterized input and

outputactions.

Letusdenoteinputofasymbolabythetokeniaandoutput

of asymbolaby thetoken oa. S recognizes certain nite

sequences of tokens, for example, ia ib od. Call the set of

suchnitesequencesLS.

Weintroduceaco-networkedmonitorM(S;m;n)withinput

andoutputbuersbetweenitandthedeviceundertest. The

systemnowcontainsthreecomponents:S;I,whichisan

in-putbuerofsizem;andO,whichisanoutputbuerofsize

n. Forconvenience,wewillnormallyabbreviateM(S;m;n)

toM.

Weintroducefournewtokens. iq

a

correspondstoenqueing

ofaninputsymbolainqueueI. idacorrespondstodeletion

ofafromheadofI andsimultaneousinputintodevice. oda

corresponds to output of a from device and simultaneous

enqueuing into O. Finally, oq

a

correspondsto outputof a

from the head of queueO. Note that M getsto see only

tokensiqandoq . Wealsoassumethatobservationofiq

a at

themonitorissimultaneouswithenqueingofaintoI,and

observationof oq

a

at themonitorissimultaneouswiththe

dequeingofafromO.

An execution of this system can be represented by a

se-quenceofsuchtokens. Considerthesequence

iq a iq b iq c idaid b od d iq e oq d

This sequencerepresents the following events: a, b, and c

gotenqueuedintheinputqueueI;thedeviceinputafrom

(4)

thequeueandthenproducedoutputd,whichwent onthe

outputqueueO;anothersymbolecameintoI;and,nally,

dleftO. Themonitorseesthesequence

iq a iq b iq c iq e oq d

and we will dene the language recognized by the

moni-torintermssuchsequences. Werstintroduceanotionof

admissibleexecution sequences underthebuering

restric-tions. We are interestedinthe admissible executionsthat

areallowedbyS.

Denition1(Admissibility) !,astringofiq,oq,idand

odtokens,issaidtobeanadmissibleexecutionsequencewith

respect toM(S;m;n),ifthefollowingare true:

FIFOInput: thesequenceofidtokensin!isthesameas

thesequenceof iqtokens,

FIFOOutput: thesequenceofoqtokens in! isthesame

asthesequence ofod tokens,

Causality: the k'th id tokencan onlyoccur afterthe k'th

iqtoken,andthek'thoqtokencanonlyoccurafterthe

k'thodtoken,

InputBuerLimit: in anyprex of !,the numberof iq

tokens minusthenumberofidtokens mustnot exceed

m,and

OutputBuerLimit: inany prex of !, the number of

odtokens minusthenumberofoqtokens mustnot

ex-ceedn.

Nowwehaveawayofdeningthelanguagerecognizedby

M. Essentially,astring belongstoLM,ifthereissome

ad-missiblesequence!whoseiq/oqprojectionis,andwhose

id/odprojectionbelongstoLS. ForeachstringsinLS,we

canconstruct several that mustbe in LM. Let us rst

constructanadmissible sequencewhoseid/odprojectionis

s. We take eachinput tokeni

a

ins and split it into two

consecutivetokensiq

a andid

a

. Wespliteachoutputtoken

o

a

intotwoconsecutivetokensod

a andoq

a

. Clearly,thisis

anadmissiblesequence(nobueringiscarriedout). Butwe

couldalsogenerateothersequencesoftokenscorresponding

to this execution. We can move eachiq token backwards,

skipping overanynumberof tokens, as long as we donot

violate relative orders of iq events, and we do not violate

inputbueringlimits. Likewise,wecanmoveeachoqtoken

forward, as long as we donot violate relative orders of oq

events,andwedonotviolateoutputbueringlimits.Every

suchsequence!yieldsastring inthelanguagerecognized

byM (LM),simplybyerasingouttheidandodtokens.

Example. ConsiderthefollowingstringinS:

ia i b oc i d oe i f ig

Wecanarriveatthefollowingadmissiblesequence(labeled

A). Weshallnormallyignoreparticular symbolsa,b,etc.,

and instead label the various tokens by order of their

ap-pearance inthe string, using the notationid k for the k'th occurrenceofid. A:iq 1 id 1 iq 2 id 2 od 1 oq 1 iq 3 id 3 od 2 oq 2 iq 4 id 4 iq 5 id 5

Now,letusallowaninputbuerof threeelementsand an

output buer of two elements. Here are some additional

admissiblestrings: B:iq 1 iq 2 id 1 id 2 od 1 oq 1 iq 3 id 3 od 2 oq 2 iq 4 id 4 iq 5 id 5 C:iq iq iq id id od oq id od oq iq id iq id D:iq 1 iq 2 iq 3 id 1 id 2 od 1 id 3 od 2 oq 1 oq 2 iq 4 id 4 iq 5 id 5 E:iq 1 iq 2 iq 3 id 1 id 2 od 1 iq 4 iq 5 id 3 od 2 oq 1 oq 2 id 4 id 5

The following string is not admissible because the input

queuecannotaccommodateiq

4 . F :iq 1 iq 2 iq 3 iq 4 id 1 id 2 od 1 iq 5 id 3 od 2 oq 1 oq 2 id 4 id 5

Foreachadmissiblestringgivenabove(A-E),wecanarrive

atastringinL

M

byerasingtheidandodtokens,asshown

below. Noteachadmissible stringgivesa uniquestringin

L

M

. Also, some non-admissible execution sequences can

yieldthesamestringinL

M

,asanadmissiblesequence(e.g.

E andF). A;B:iq 1 iq 2 oq 1 iq 3 oq 2 iq 4 iq 5 C ;D:iq 1 iq 2 iq 3 oq 1 oq 2 iq 4 iq 5 E;F :iq 1 iq 2 iq 3 iq 4 iq 5 oq 1 oq 2

3.1 EliminatingOutputBuering

Inthis section, wegive aconstructionthat letsus assume

that the monitor M does not need to consider anoutput

buer, by suitably adjusting the size of the input buer.

Thissimplicationisusefulinreasoningaboutpropertiesof

protocolsthatpermiteÆcientmonitoring.

Theorem 1 L M(S;m;n) L M(S;m+c U n;0) , where cU is a

constant dependent on S: cU is the maximum number of

input symbols withoutan intervening output symbolinany

string inLS.

We claim that a monitor similar to M, but that has an

associated input buer of size m+cUn and no output

buer,canadmit all theobservablebehaviorsofM. That

is,if asequenceoftokens observedbyM canbederived

constructively for M usingthe procedure describedin the

previous section froma strings 2 LS, then canalso be

derivedconstructively for this newmonitorfromthe same

s. WedenotethisnewmonitorM(S;m+c

U

n;0)byM 0

.

Inthemonitoringprocess,wewillmakeuseofaconsequence

ofthistheoremthat 2=L

M 0

) 2=LM. Byconstruction

of LM, 2= LM implies that the device is not following

S. Thus, by inferring 2= L

M 0

, M 0

can infer that a

cer-taininput-outputsequenceit|orM|observesisdenitely

inconsistentwithS.

Sketch of Proof. The theorem doesnot triviallyhold,

be-cause a stringinLM may bea result ofoutput buering,

whereasthereisnooutputbuerinM 0

.

First assume that every input to S generates an output

(c

U

= 1). Let 2 L

M

. By construction, is a result

oferasingidandodtokensfromsomeadmissibleexecution

! with respectto M. ! contains a uniqueexecution s in

LS. Wenowshowatransformationon!thatmakesit

ad-missiblewithrespecttoM 0

,whilestillcontainingthesame

s(id/odprojection)andthesame (iq/oqprojection)init.

Then, 2 L

M 0

, because we can obtain from s by rst

constructing the transformed !, which is admissible with

respecttoM 0

,andthenerasingidandodtokensfromit.

We performthe following transformation on !. We start

(5)

od , the k'th occurrence of od, if od is not immediately

followedbyoq k

,wemoveitforwardsomenumberofstepsto

makeitso. Indoingthis,wemightskipoverotheriqandoq

tokens. Anyidtokensinbetweenod k andoq k aremovedin orderafteroq k

. Noodtokenscanoccurinbetweenbecause

they must have already been moved to their appropriate

positions. Byskipping overall theintermediateoq tokens,

we reduce the output buering requirement to zero. For

every iqtoken that some id token skipsover, we increase

theinputbueringrequirementbyone. Wearguethatthe

maximumincreaseinthe inputbueringrequirement isn,

thesizeoftheoutputbuerO.

Considerthe largest numberofiqtokensbetween od k

and

oq k

. Supposethatatthepositionaftertheod k

,therearep

elementsinthe inputbuer. There isatleastoneelement

intheoutputbuer (thecontent of od k

). Therstm p

iqtokenslluptheinputbuer. Everyiqtokenafterthese

mustbeprecededbysomeidtoken,whichproducesanod.

Therefore,n 1suchiqtokenswillcausetheoutputbuer

tobelledup. Onemoreidandiqcanalsoappear|inthat

order|forwhomthe outputhasnotyetbeenproduced by

thedevice. No moreiq'sareallowedbeforeoq k

. Afterthe

last suchiq,the numberof iqtokensminus thenumberof

idtokenscanbeatmost(m p+n)+p,sincewepushed

all theintermediateid tokensout. This willbeadmissible

inamonitorwithinputbuersizem+n.

Ingeneral,Sneednotexpectanoutputforeveryinput;the

above transformation no longer works. However, suppose

it is given that S cannotaccept any morethan cU inputs

beforeitmustseeanoutput. IfwesupplyM 0

withaninput

buerofsizem+cUn,thetheoremstillholds. Toseethis,

we need to modify our previous counting argument, as to

howmanyiqtokensmightoccurbetweenod k

andoq k

. The

rstm piqtokenslluptheinputbuer. Thenextc

U n

iq'smustbeprecededbyc

U

nid's,thatproducen 1od's

llinguptheoutputbuer. Nomoreiq'sarepossible.

Example. Letm=2andn=2. Considerthe following!,

wherecU=2. ThisstringisadmissiblewithrespecttoM.

iq 1 iq 2 od 1 id 1 iq 3 id 2 iq 4 od 2 id 3 iq 5 id 4 iq 6 oq 1 oq 2

Aftermovingtheodtokens,wearriveatthefollowing

trans-formed!. Notethat therelativeordering ofid, od tokens

must be maintained, otherwise we change the underlying

inputtoS. iq 1 iq 2 iq 3 iq 4 iq 5 iq 6 od 1 oq 1 id 1 id 2 od 2 oq 2 id 3 id 4

Thistransformationincreasesthemaximuminputbuering

requirementto 6,whichisadmissible toM 0

,whichhasan

inputbuerofsize6(2+22).

3.2 Dealing withLoss

Wenowintroduceintothismodelthepossibilityoflosinga

packetbetweentheco-locatedmonitorandthedevice,i.e.,

the monitor observes some input packets that the device

does not. The model is as follows: we assume that the

outputfromtheinputqueueI couldeitherbeabsorbedby

alossunitL,nevertobeseenagain,orgoesintothedevice

as before. We use token il to denote the loss event that

consumestheheadoftheinputqueue.

Example. Considerthefollowingsequence:

iq a ilaiq b iq c id b od d ilcoq d

Inthissequence,Sexecutesthefollowing string:

i

b o

d

Whereas,themonitorobservesthefollowing tokens:

iq a iq b iq c oq d

Inputsaandcarelostinthelossunit.

NotethattheMtoM 0

conversionoftheprevioussubsection

wouldneedtoknowaboutthemaximumeectivenumberof

inputsseenbythemonitorbeforeanoutputappears. This

implies that we must impose a limit on the number of il

tokens that canappear ina sequencewithoutanid token.

If therecan be onlylessthan c

L

iltokens betweentwo id

tokens,thenM 0

needsaninputbuerofsizem+c

L c

U n

tobeabletoeliminatethesizenoutputbuer.

Corollary 1 If S cannot accept more that c

U

id tokens

withoutaninterveningodtoken,andtheremustbelessthan

cLiltokens betweentwoidtokens,then,

L M(S;m;n) L M(S;m+c U c L n;0)

Fortheremainderofthepaper,weassumeamonitorofthe

formM 0

,appropriatelyparameterizedwiththevaluesofm,

n, c

U and c

L

. Furthermore,we shalluse B torefer to the

inputbueringrequirement(m+c

U c

L

n). Theadmissible

strings!maynowalsocontainiltokens(inadditiontoiq,

id, od and oq), and their placement will follow conditions

mentionedhere. Wemodify the denitionof admissibility

to take iltokens into account: the FIFO input, causality,

and input buer limit clauses are updated appropriately,

treatingiltokensjustlikeidtokens. Inaddition,wehavean

InputLossLimitcondition:inanyprexof!,thenumberof

consecutiveil'swithoutaninterveningidtokenislessthan

cL.

Corollary 2 Considerastring admissibletoM 0

. Ignoring

any intervening id or il tokens, the maximum number of

iq tokens without an intervening od oq pair is bounded by

B+cUcL.

ThecorollaryholdsbecausecUcL idandiqpairscan

ap-pear inadditionto B iq's,withoutincreasingthe eective

buering requirement. (Anotherid mustforceanod.) The

importanceofthiscorollary willbecomeapparentinthe

al-gorithmspresentedinSection5.

4 AlgorithmforCo-networkedMonitoring

Ingeneral,wewouldliketocheckthatagivendeviceunder

testisaproperimplementationofaspecicationS,givena

tracecollected fromaco-networkedpassive monitor. That

is,givenatrace ofiqandoqeventsobservedatamonitor

thatisseparatedfromthedevicebyalossunit(L),input(I)

andoutputbuer(O)asspeciedinSection3,wewouldlike

todetermine whetherthedevice isbehavinginaccordance

with S. The device is not incorrect as long as 2 L

M 0:

wecanexhibitasequence!whichisderivedfrom bythe

additionofid,il,andodeventsandwhichisconsistentwith

thefollowingsetofconditions:

!isadmissiblewithrespecttoM 0

(6)

(denoted[!]

id,od

) isconsistent withthespecication

ofS.

AssumewehaveafunctiongthatchecksS onasequence

ofidand odtokens andtells uswhetherthe sequenceis in

LS;gcouldbeuseddirectlyas aco-locatedmonitorforS.

Wewritethequerytogintheform2g.Weassumethatg

isasafety property: itisprex-closed,andcanbechecked

over nitetraces. Our problem is to construct a function

F(g;),thatgivenatraceofiqandoqeventscollectedbya

co-networkedmonitorandafunctiong,tellsuswhetherthe

tracecorrespondstosomeproperexecutionwithrespectto

S. Anon-deterministicalgorithmforF isstraight-forward.

Given a , guess a sequence ! admissible with respect to

M 0 ,suchthat[!] iq,oq =. If[!] id,od satisesg, isOK.

Otherwise,reportfailure.

A naive determinization of the above algorithm is

brute-forcesearch:simplyconstructallpossible!'sfrom,

check-ing each admissible! againstg,until amatchisfound or

all possibilities are exhausted. Additionally,wewould like

F to be computableon-line, meaning thatit should make

onlyonepassoverthe input. Wegivesuchabrute-force

breadth-rst-searchalgorithm inFigure3. We callthis

al-gorithmBF(g;).

DataType. isasetofpairs(admissiblestring,string

ofinputsymbols). Initially,=f(;)g

Event Handlers. Onreceiving

iq x : 1. 8(!;b)2: delete(!;b)from; check-add(!::iq x ;b::iq x )to.

2. iterateuntilnomoreadditionsto:

8(!;iq y ::b)2: check-add(!::id y ;b)to; check-add(!::il y ;b)to oq x : 1. 8(!;b)2: delete(!;b)from; check-add(!::od x ::oq x ;b)to.

8(!;iq y ::b)2: check-add(!::id y ;b)to; check-add(!::il y ;b)to

where check-addadds(!;b)toifandonlyif:

jbj<B,and

[!]

id,od 2g

inany prex of !, the numberof consecutive

il'swithoutaninterveningidtokenislessthan

c

L .

If = after executing eitherevent handler, ag

anerror.

Figure3: Brute-Force MonitoringAlgorithm

strings!that satisfythefollowing:

C1 [!] iq;oq =. C2 [!] id;od 2g.

C3 !isadmissible(withrespecttoM 0

).

TherearetwosourcesofineÆcienciesinBF(g;). First,we

mightmaintainalargenumberofplausiblesequences.

Sec-ond, we needto examinethe suitability of eachcandidate

extensionofeachsequencewithrespecttotheabstract

spec-ication. Ifthereisnoadditionalinformationaboutg,then

thereisnoboundontheamountofcomputationrequiredat

eachevent. SupposethatN

iq

inputandNoqoutputevents

have taken place. Thenthe amount of workdone at each

eventisessentiallythesizeofatthatpoint. Thesizeof

isexponentialinN

iq +N

oq

,evenintheabsenceofloss. To

seethis,noticethatthenumberofplacementsofmatching

idtokensrelativetoasequenceofiqtokensisitself

exponen-tialinN

iq

;possibilityofloss factorsinanotherN

iq binary

choicesbetweenidandil. Therefore,evenintheabsenceof

loss, the computation that needs to be performed at each

eventgrowswiththenumberofeventsthathaveoccurred.

Foraneectiveonlineprocedure,wemustarriveatamore

eÆcient monitor process that bounds this per-event

com-putation. Weshallexploitproperties ofgthat allowus to

achievethisoptimization.

5 PropertyBasedOptimization

The algorithm given in Figure 3 exhibits space and time

complexitythatisexponentialinthenumberofinputsand

outputs in the trace. Clearly, for a long trace , the

al-gorithmwillquicklybecomeintractable. However,some

as-sumptionsallowustoprunelargesectionsofthebrute-force

search. Inthefollowingsubsections, weexamineproperties

ofg,andassumptionsontheinput-outputtracesthatallow

eectiveonlinemonitoring.

Anotable featurethatiscommontomany(butnotall)of

the algorithms presented in this section is that they

sim-ply buer iq tokens as they appear, and take interesting

actiononlywhenoqtokensareobserved.Thisisincontrast

to the brute-force algorithm, in which aniteration is

per-formed after eachiq event to consume the pending buer

inall possibleways. If this iteration is notperformed, up

toB+cUcLiqtokensmustbebueredbetweenoutputs

intheworstcase, followingCorollary 2. However,afteran

output,themaximumnumberof buerediqtokenscannot

bemorethanB.

5.1 NoLoss

Each of the following sub-sections describes a class of g's

that canbeeectively monitoredifweassume thereis no

loss betweenthemonitorand thedevice,i.e. cL=1. This

isanassumptionofthenetworkbehavior,whichwhentrue,

allowsustouseveryeÆcientalgorithmsformonitoring.

5.1.1 P1: CountingProperties

Averybasickindofpropertyis asimpleconstraintonthe

(7)

betweenc

min andc

max

inputswithoutplacinganyadditional

constraintsontheallowedsequences,thengsatises:

8;;odx;ody:2g ^ (=_endsinsomeod)

^hasonlyid's::

od y 2g () c min jjc max

Thatis,astringisingifandonlyifitisconstructedfrom

anotherstringthatisalsoing,whichisitselfemptyorends

inanoutputevent,byaddingbetweenc

min andc

max input

events.

Analgorithmforcheckingsuchg'sinthepresenceofbuers

isshowninFigure4. Thisalgorithmmaintainstwointegers,

Constants. c

max ;c

min

areintegers.

DataType. buf

min andbuf max areintegers. e isaboolean. Initially,buf min =buf max =0ande=false.

iq x : buf max =buf max +1,buf min =buf min +1 if(buf min >B+c max )thene=true elsebuf max =min(buf max ;B+cmax) oq x : if(buf max <c min ) thene=true elsebuf max =min(B;buf max c min ); buf min =max(0;buf min c max )

Ifeistrueafterexecutingeithereventhandler,ag

anerror.

Figure4:AlgorithmforcheckingP1

buf

min andbuf

max

,representingtheminimumandmaximum

numberof inputs that are currentlybuered between the

monitorandthedeviceundertest. Ifbuf

min

evergrowstoo

large,itindicatesthattheparticulariq;oq stringseensofar

couldnotbeavalidexecutionwithoutadditional buering

betweenthe monitorand thedevice. Thatis,toofew

out-puts have been seen to account for all the inputsseen so

far. Similarly,ifbuf

max

everbecomestoosmall,itindicates

thattheparticulariq;oq stringseensofarcouldnotreect

avalidexecutionbecauseevenifeachoutputhasconsumed

the minimum numberof inputs, there have not been

suf-cient inputs to account for every outputgiven that each

outputmustconsumeat leastcmin inputs. Ineachcase an

errorageisset.

Toprovethecorrectness ofthisalgorithm,wecandoa

re-duction from the brute-force algorithm of Figure 3 to the

oneinFigure4.Thereductionproceedsbydeninga

map-ping between the two state spacesand showing that it is

maintainedwheneachalgorithmtakesastepinresponseto

thesameevent. Theproofisgivenintheappendix;weomit

proofsoftheremainingproperties.

5.1.2 P2: IndependentInputsandOutputs

If g checks a composition of two independent properties,

one of the input trace and the other of the output trace,

Formally,wesaythat:

([!] id 2g^[!] od 2g),[!] id,od 2g

Forexample,supposewewanttocheckthataslongasthere

isnolossandalldataisacked,theTCPsenderweare

mon-itoringwillkeeppumpingoutnewdata. Firstwecheckthat

everyoutputdata segment hasasequencenumberstrictly

greaterthanthatofthelastoutput.However,thisneedsto

betrueonlyaslongas alldataisackedbythereceiver. So

inconjunction,weneedtocheckthateveryinputackhasa

sequencenumberstrictlygreaterthanthatofthe last

out-put.Aslongasbothofthesearetrue,thesenderisbehaving

correctly. If thesecond property fails, ourassumptionhas

beenviolated,andthesenderisassumedcorrectbydefault.

ThemonitoringalgorithmforP2simplyneedstocheckthat

thesequenceofidxeventscorrespondingtotheiq

x

events,as

wellasthesequenceofodxeventscorrespondingtotheoq

x

events,isacceptableaccording tog. Inaddition,ifwewish

to monitor more complexproperties, theycan be checked

independently, since P2does notplaceany restrictionson

therelativeorderings ofod'sandid's. Theprocedurefeeds

onetokentogateveryeventandneedstomaintainnostate

extraneoustog. ThealgorithmisasshowninFigure5.

Data Type. !iisastringofid's.

!oisastringofod 's.

eisaboolean.

Initially,!i=!o=ande=false.

EventHandlers. Onreceiving

iq x : ! i =! i ::id x ; if! i 62gthene=true; oq x : ! o =! o ::od x ; if! o 62gthene=true;

anerror.

Figure5: AlgorithmforcheckingP2

5.1.3 P3: PeriodicOutputs

Supposethespecicationrequires thatthedeviceproduces

anoutputexactlyeveryP inputs,wecanhaveaneÆcient

algorithm to parseatrace; nootherassumptions ongare

required. Theconditioncanbegivenasfollows:

8;:consistsonlyofinputs^jj>0^

(=_endsinsomeod)::

(ody 2g))jj=P

Many protocols have a periodic output behavior. For

in-stance,ICMPechoprotocolhasanoutputperiod(P)of1:

everyinputmustberespondedtobyanoutput.SomeTCP

implementationsmaintainaP of2. P3canbethoughtofa

restrictedcountingproperty(cmin=cmax=P)in

(8)

seen by the monitor, it feedsexactly P buered inputsto

g,andthenchecksthattheoutputisenabledatthispoint.

ThealgorithmisasshowninFigure6.

Constants. P is the number of inputs consumed by g

beforeeachoutput.

DataType. !isastringofid'sandod 's.

bisastringofiq's.

e isaboolean.

Initially,!=b=ande=false.

iq x : b=b::iq x ; ifjbj>B+P thene=true; oq x : ifjbj<P thene=true; else 1. iterateP times:

deletetherstelementiq

y fromb; !=!::id y ; 2. !=!::odx; if!62gthene=true;

anerror.

5.1.4 P4: DeterministicPlacementofOutputs

Suppose ghasthe propertythat theplacement ofoutputs

inasequenceofinputshasnoleeway: thereisexactly one

positionatwhicheachoutputcouldbeplacedsuchthatthe

resultingstringis ing. Formally, thedeterministic output

placementpropertycanbegivenasfollows:

8;:consistsonlyofinputs ^jj>0::

od

k

2g) od

k 62g

Inpractice, weonlyneedthat od

k

62gfor jj<B. A

propertyinP4ischeckedbymaintainingabuerof

uncon-sumedinputs. Ongettinganoutput,wefeedinputsfromthe

buertoguntiltherstpointwheretheoutputisenabled,

andthenwefeedtheoutputtog. Thisworks, becauseP4

guaranteesthattheoutputcouldnotoccur afteranyother

input in the future. This algorithm maintains buer b of

inputsof size (B+cU). At each output we feed the rst

jbj Binputstog,followedbyatmostB strings(oflength

atmostB).

Ingeneral,nite-statemachinesdonotsatisfyP4. However,

somedo,and specicinstancesof otherkindsof automata

(PDAetc.) couldsatisfyP4too. Forexample,thefollowing

grammarobeysP4: G! G!GG G!HoSTOP H!iA H!i LPAREN Hi RPAREN

Sometimes,thereisarangeofpositions inabueredinput

streamwhere onecould reasonably placeanoutput.

Con-sider the following two restrictions: the range ofpositions

overwhichtheoutputisenablediscontiguous,inthesense

thattheinputisenabledateverypointintheinputstream

betweentherst andlast positions itisenabled. Also, we

require that the output commute with each of the inputs

inthis contiguouswindow, suchthat if theoutput is

con-sumedandthentheinput,wearriveat astatewhichis

in-distinguishablefromconsumingtheinputrstandthenthe

output. Thesetwoconditionsareembodiedinthefollowing

formula:

8;:consistsonlyofinputs::

(odx2g)^(odx 2g)) (8Æ1;Æ2:Æ1 Æ2=:: (odxgÆ1 odx Æ2)) where g (8:2g()2g)

Anexampleof suchaspecication isTCP owcontrol. If

aTCPackodx isallowedjustbeforedatasegmentid k

and

just after id k+p

, then it must be allowed at all points in

between,since none ofthemmatter to the ack. Moreover,

the receiver'swindow isthe same whether this ack occurs

before or after any of these data segmentsin the window

id k

:::id k+p

.

ForginP5,wecanworkwithacreditscheme. Wealways

placeoutputat therstplaceitis enabled,butremember

the credit we get for notplacing it at a laterplace. If at

some point inthe future, the buer overows because we

consumed too few inputs, we can then use up this credit,

i.e. eatupthatmanyinputtokens.

ThealgorithmforP5(Figure7)maintainsabuerofinputs

ofsize (B+cU),and oneintegerindicatingthecredit: the

range of positions where the last output could have taken

place. Ateachoutputwefeedthe rstjbj B inputstog,

followedbyatmostB strings(oflengthatmostB).

5.1.6 P6: Output-checkpointedAutomata

Wesaythatgisanoutput-checkpointedautomatonif,

start-ing fromastate,for eachoutputodx,thereisat mostone

next state that g can be in. In terms of strings, if two

stringsandareequivalentwithrespecttog,andifthey

areconcatenatedwithdierentstringsofinputsfollowedby

thesameoutputod

x

,thenthetwostringsendinginod

x are

stillequivalentwithrespecttog. Formally,

8;:

(g ))

(8odx;Æ1;Æ2:Æ1;Æ2consistonlyofinputs::

((Æ 1 od x 2g)^( Æ 2 od x 2g))) (Æ 1 od x g Æ 2 od x ))

Aninstance ofP6isaprotocolinwhicheachoutputgives

completeinformationaboutthestateinwhichitisenabled.

For example,some transportlayerprotocols haveanotion

of selective acknowledgments (sack's) where datareceivers

send informationabout alldata received upto thatpoint.

(9)

DataType. !;! 0

arestrings.

b;b 0

arestringsofinputs.

creditisaninteger.

e isaboolean.

Initially,!=b=,credit=0ande=false.

iq x : b=b::iq x ;

if(jbj>(B+cU)^credit>0)then

y ofb; !=!::id y ; credit=credit 1; if(jbj>(B+c U

)^credit=0)thene=true;

oq

x :

1. repeatuntil

((jbjB)^(!::odx2g_b=)):

y ofb; !=!::id y ; 2. ifb6=then b 0 =b;

y ofb 0 ; ! 0 =!::idy; credit=0; repeatuntil! 0 ::odx62gorb 0 =: credit=credit+1;

y ofb 0 ; ! 0 =! 0 ::idy; 3. !=!::od x ; if!62gthene=true;

anerror.

hashappened,thestateatthereceiveriscompletelyknown

tous. Sowe cancarry outouranalysis outputto output,

forgettingeverythingthatoccurredbeforethelastoutput.

ThealgorithmformonitoringP6maintainsjustone

admis-sible string ! because after an output occurs, all strings

allowedbygmustbeequivalent;wejustneedtokeeponeof

them,andwekeeptheonewiththerstpossibleplacement

ofthatoutput. Inaddition,wemaintainabuerbofinputs

ofsize (B+c

U

), andB bits indicatingpositionsat which

thelastoutputcouldalsohavehappened.

Ateachoutput,weneedtoconcatenateeverysub-stringof

bueredinputs,startingfromthesepositions,to!andcheck

iftheoutputisallowed afterthisstring. Theremaybeup

toB 2

suchinputstrings(oflengthatmost(B+cU))that

needtobechecked. Thisgivesusaboundfortheper-event

computation,polynomialinthelengthofthebuers.

5.1.7 P7: FiniteStateMachines

Ifgisknowntobeanitestatemachinewithasetofstates

,thenthereisaboundtowhichthebrute-forcesearchset

cangrow. Supposeg=(; ;Æ;s0;Err),whereconsists

ofall id,odsymbols, isthenitesetofstates,Æisthe

de-0

isthesetoferrorstates. Each(!;b)incanberepresented

as(s;b)wheres=Æ

(s

0

;!). Now,bmustcontainsome

suf-xof theiq elementsin!,oflengthat mostB. However,

every!inmustcontainthesamesub-sequenceofiq'sand

oq's. Therefore,everyb inmustbe asuÆxof the same

B-elementstring. Thisgivesusaboundofj jB elements

for. Thismeansthatwemayhavetofeeduptoj jB 2

elementsto gat eachoutput. Theboundispolynomialin

thenumberofstatesandthesize ofthebuers. Moreover,

it givesusaway ofexecutingthe generalbrute-force

algo-rithmwithaneÆcientrepresentationofthestring!asthe

states. ThealgorithmisasshowninFigure8. Noticethat,

becauseweperformaniterationafteraniqevent,asinthe

brute-forcealgorithm,wecapthebueringrequirementby

B.

Constants. g=(; ;Æ;s0;Err ).

Data Type. isasetofpairs(state,listofinput

sym-bols). Initially,=f(s

0 ;[])g

iq x : 1. 8(s;b)2: delete(s;b)from; check-add(s;b::iq x )to.

8(s;[iq y ::b])2: check-add(Æ(s;id y );b)to; oq x : 1. 8(s;b)2: delete(s;b)from; check-add(Æ(s;od x );b)to.

8(s;[iq

y

::b])2:

check-add(Æ(s;idy);b)to;

wherecheck-addadds(s;b)toifandonlyif:

bobeysthebueringconstraints(jbj<B),and

sisnotanerrorstateofg(s62Err).

If = after executing eithereventhandler, ag

anerror.

Figure8: Algorithmfor checkingP7

5.2 Dealingwith Loss

Allowingloss ofinput eventsbetweenthe monitorand the

deviceintroducesadditional complexity tooursearch

algo-rithm. However,asbefore,wecanderiveeÆcientalgorithms

for someclassesofg. Ineachofthefollowing sub-sections,

wedescribe apropertyofgthatallowsustomonitorit

ef-fectivelyeveninthe presenceofloss. Wewill seethat the

classes of g that wereeÆciently monitorable withoutloss,

(10)

Asmentionedbefore,propertiesbasedsolelyonthenumber

ofinputs and outputsconsumed are often used to

charac-terizeprotocols. Forsuchg, monitoringinthepresenceof

loss is very similar to the loss-less case. Suppose that we

knowthatnomorethanc

L

inputscanbelostinsuccession.

Theninordertocheckthateachoutputmustconsume

be-tweenc

min andc

max

inputs,wefollowthesameprocedureof

maintainingbuf

max andbuf

min

,exceptthatweallowbuf

max andbuf min togrowuptom+c max c L (n+1)duringiq

processing. Weleavetheoq processingunchanged. Thereis

clearlyaconstantamountofworktobedoneateachevent.

5.2.2 P2o: IndependentOutputProperties

Propertiesabouttheoutputstreamcanbecheckedevenin

the presence of loss because by assumption outputs never

getlost. Thecheckingproceduresimplyinvolvescheckingg

againstthetraceseenatM. Nobueringisneeded.

AnexampleofaP2opropertyisoutputmonotonicity: every

outputcontains asequencenumberthat is strictlygreater

thanthatof thelast output. Thisjustinvolvesstoring an

outputand comparingit withthe nextone. We feed gat

mostonetokenateveryoutput.

5.2.3 P8: Insert-closedCommutativeOutputs

Wedescribea classofmonitorable properties that are

im-pervioustothepresenceofinputloss. Essentially,ifastring

endinginodxisacceptable,soisthestringwithanarbitrary

stringofinputsÆ insertedbeforeodx. Moreover,odx

com-muteswith every input inÆ. In eect, this saysthat ifg

enablesodx at the endofthe string, it remainsenabled

forever, anditcanbeplaced atany pointafterwiththe

sameeect. (8;odx:odx2g) ((8id y :id y od x 2g)^ (8Æ 1 ;Æ 2

consistingonlyofinputs:

od x Æ 1 Æ 2 g Æ 1 od x Æ 2 )))

NotethatP8isreallyaspecialcaseofP5. Thefactthat

out-putsonceenabledareenabledforeverimpliesthe

contiguously-enabledproperty,inadditiontowhichwealreadyhave

com-mutativity ofoutputs. Aninstance ofP8is acontainment

property onoutputs. For example, a property could state

thateveryoutputcontainsaeldbytes-receivedthatis less

thanthesumofthesizesofinputsreceived.

The algorithm for checkingP8 is simple. Always assume

thatnoinputshavebeenlostandplacetheoutputafterthe

rst input where it is enabled. To keep track of the

suc-ceedingpositionswheretheoutputcouldhaveoccurred,we

maintain a credit as in the algorithm for P5. Suppose in

reality, someof the inputsbetweentwooutputs werelost,

thenwehavegeneratedastringthathassomeextrainputs

insertedbetweentheseoutputs. However, P8tells us that

insertingtheseinputsstillkeepsthestringing. Next,

sup-pose that an output that we placed after aninput id k

in

theinputstream,didnotinrealityoccuruntilid k+p

. Here

again,P8assuresusthattheoutputcommuteswiththe

in-puts id k ;id k+1 :::;id k+p

. Finally, note that the string we

constructisanadmissiblestring: oneinwhichnolosses

oc-cur. Therefore,aslongasthisstringbelongstog,wemust

gorithmfeedsgwithatmost(B+c

U c

L

)inputtokensat

eachoutput.

Data Type. !isastring.

bisastringsofinputs.

creditisaninteger.

eisaboolean.

Initially,!=b=,credit=0ande=false.

iq x : b=b::iq x ; if(jbj>(B+c U c L )^credit>0)then

y ofb; !=!::id y ; credit=credit 1; if(jbj>(B+cUcL)^credit=0)then e=true; oq x : 1. repeatuntil ((jbjB)^(!::odx2g_b=)):

y ofb; !=!::id y ; 2. !=!::odx; if!62gthene=true; 3. credit=jbj;

anerror.

5.2.4 FiniteStateMachines(P7revisited)

If g is an FSM, we can bound the amount of state that

we need in order to model loss in the brute-force search

strategy. Infact, theboundonisexactlythesameasin

theabsenceof loss. There canbe atmostj jB elements

in. Asbefore, thisboundmeansthat we caneectively

execute the brute forcealgorithm to monitor g. However,

inthis case we have to consider all 2 B

lossy substrings of

thebuerateachevent. Therefore,wemayneedtofeedas

manyasj jB 2

2 B 1

tokenstogateachevent.

5.2.5 P9: DeterministicStatelessTransducers

Letusassumethatalltheinputsthatthemonitorseeswill

be distinct. Thisrestriction forbidsthe environment from

producingduplicate inputsfor thelifetimeof themonitor.

Underthisassumption,wecandescribeagthatspeciesa

statelesstransducingbehavior. Supposethatanoutputody

canoccurafteraninputidx. Then,ineverystringallowed

bygthatcontainsody,itmustoccurimmediatelyafteridx.

Moreover,anystringendinginanidxodypairisequivalent

to thestringidx ody. Thismeansthatgis stateless: all it

(11)

8;idx;ody:2g)

(idx odygidxody)^

(idx ody2g)

(8idz6=idx:idz ody62g)^

(ody62g)

Theconditionthat eachinputis distinctisaratherstrong

one. Inpractice,whatweneedtoimposeisthatnoinputis

repeatedwithinaspaceof(B+cUcL)iqtokens,whichis

themaximumnumberof inputsthatthe monitorneedsto

analyzeatatime.

Anexample of P9 is the specication of theICMP (ping)

protocol. Inapingsession, allinputs(ICMPRequests)are

uniquebecausetheyhavemonotonicallyincreasingsequence

numbers. Givenan ICMPReply,we canmap it uniquely

toanICMPRequestonthebasis ofthesequencenumber.

Moreover, once we have mapped the Reply to a request,

thereisnothingthatweneedtorememberaboutthe

anal-ysisuptothatpoint.

P9 canbe checkedeÆciently even inthe presence of loss.

Essentially,eachoutputodypointstoatmostoneinputidx

that triggered it and we do not need to check any other

input. Ifidxisnotinthebuer,andodycannotoccur

with-outaninput, thenthereis anerror, otherwise, everything

upto idx canbedroppedfrom thebuer. Thisalgorithm

feedsgatmost(B+cUcL)elementsateveryoutput. The

algorithmisasshowninFigure10.

DataType. !isastring.

bisastringofinputs.

e isaboolean.

Initially,!=b=ande=false.

iq x : b=b::iq x ; if(jbj>(B+c U c L ))thene=true; oq x : ifb=thene=true; else

y ofb;

repeatuntil

((jbjB)^(idy::odx2g_b=)):

y ofb;

!=!::idy::odx;

if!62gthene=true;

anerror.

5.2.6 P10: Output-checkpointedStatefulTransducers

In P9, the statelessness assumption allowed us to forget

about the history of the monitoring, once an output had

beenplacedintheinputstream. Wecangeneralizethis

no-tion,by takinga hint fromoutput-checkpointed automata

(P6). Insteadofsayingthatthestate ofgafterconsuming

y x

id

y od

x

,wesaythatstartingfromastate,ifod

x

isallowed

aftersomeinputsthenthereisauniquestatethatgcanbe

inafterconsumingod

x .

As before, all inputsin mustbe distinct. Formally, g is

expressed as a conjunction of P6 with a generalization of

P9: 8;: ( g )) (8od x ;Æ 1 ;Æ 2 :Æ 1 ;Æ 2

consistonlyofinputs::

((Æ 1 od x 2g)^( Æ 2 od x 2g))) (Æ 1 od x g Æ 2 od x ))^ 8;id x ;od y ;Æ 1 ;Æ 2 :Æ 1 ;Æ 2

consistonlyofinputs::

(Æ 1 id x od y 2g))

(8idz 6=idx:Æ2idz ody62g)^

(ody62g)

Anexampleofsuchagisaprotocolinwhichtherearetwo

kinds of inputs: data inputs containing integers and

ack-request inputs that contain unique magic numbers. The

protocolissupposedtosendanoutputwhenitreceivesan

ack-request, andthe outputcontainsthe magic numberof

therequestandasumofallinputsseensincethelastoutput.

Clearly, everyoutputhasauniqueinput (theack-request)

afterwhichitcanoccur. Moreover,itcontainsinformation

that is compatibleonlywithsome sub-sequencesofinputs

sincethelastoutput.

TheP6partofP10allowsustojustmaintainonestate

af-tereachoutput. Moreover,theP9parttellsusthatwecan

uniquelyplacetheoutputintheinputstream. In

conjunc-tion,thisgivesusasimplealgorithm forcheckingP10. At

eachoutput,weplaceitinthesequenceofbueredinputs

as inP9. Once we have found aposition, wemustsearch

for a lossysub-sequence of theinputsbeforethat position

that makesthe stringacceptable to g. If boththese tests

succeed,wecanproceedwithasinglenextstateandasingle

buertomonitortherestofthestring.

Thealgorithm maintainsainput buerbofsize(B+cU

cL). Thenateachoutput,wemayneedtocheckgagainst

2 (B+c U c L )

input sequences. Themaximumnumberof

to-kenstobefedtogis(B+c U c L )2 (B+c U c L ) 1 .

5.2.7 Output-checkpointedAutomata (P6revisited)

P6 can be handled inthe presence of loss by doing some

further exploration of buer strings. However, no

explo-ration ofg's statespace needs tobedone. Wemaintaina

set of positions in the buer where the last output could

havehappened. Thenfor thenextoutput,weneedtond

all sub-sequencesstartingfrom thesepositions thatenable

theoutput.Weareguaranteedthatallthesesub-sequences

will endinthesame state,so we will justneedto

remem-ber the positions where we can place the output and the

common nextstate. If there is no position where we can

place the output,there isan error. This algorithm would

requireustocheckamaximumof2 (B+c U c L ) inputstrings

against g at each output, where (B+cUcL) is the size

of the input buer. We may have to feed a maximum of

(B+cUcL)2 (B+c U c L ) 1 tokenstog.

(12)

Inthissection,wedescribetherelationshipsbetweenallthe

classesofgthatwehavediscussedinthissection. InTable1,

wesummarizethecomplexitiesofthealgorithmsdescribed

forthevariouspropertyclasses. Weassumethatghasbeen

written ina way that allows us to feed it input and

out-puttokensonebyone, andit will takeaconstant amount

oftimetoanalyzeeachtoken. Thenthecomplexityofthe

monitoringisthenumberoftokensthatneedtobefedtog

atanystep. Notethatmostofthecomplexitiesareinterms

ofthesize ofthebuerB. ThevalueofB isdependenton

thesizesofthe input(m)andoutput(n)buers, protocol

entitieslikethemaximumnumberofinputsallowedbetween

twooutputs(cU),andenvironmentalentitieslikethe

max-imumnumberofinputsthatcanbelost insuccession(cL),

betweenthe monitorand thesystem. Wedenethebuer

size inthe absence of loss as B =m+cUn, and inthe

presenceoflossasB=m+cUcLn.

Table 1: MonitoringComplexitiesforPropertyClassesofg

Property Withoutloss Withloss

All Unbounded Unbounded

P7 B 2 B 2 2 B P6 B 3 B2 B P5 B 2 Unbounded P4 B 2 Unbounded P8 B 2 B 2 P3 P Unbounded P10 B B2 B P9 B B P2 1 Unbounded P2o 1 1 P1 1(justcount) 1

AsTable 1shows, most ofthe propertiesdescribedinthis

sectionbelongtodierentcomplexityclassesformonitoring.

Many oftheproperties areinfactinclusionsof eachother.

TheinclusionrelationshipsareasshowninFigure11.

How-ever,thereare someotherinteresting relationshipsaswell.

Ingoingfromtheno-losstothelossycase,propertiesP3,P4

and P5 becomevery ineÆcient to monitor. Therefore, we

needtopicknersubsetsofP5: P8,P9 andP10,thatwill

stilllendthemselvestoeÆcient monitoringinthe presence

ofloss. ThesesubsetscollapsetosimilarcomplexitiesasP3,

P4andP5intheno-losscase.

Properties P5, P6, and P7 are three orthogonal ways of

reigningintheunboundedsizeofinthebrute-forcesearch.

Itis possibletocombine someofthesepropertiesto arrive

ataproperty thatrelatestoareal-worldprotocol. For

ex-ample,wecombineP6and P7inthenextsection toarrive

at a TCP property. It may also be possible to combine

oneof moreoftheseproperties withsomeprotocol-specic

knowledge toreduce thesizeof. Therefore,ifaprotocol

property thatonewishes tomonitordoesnotfallinoneof

P5,P6,orP7,itdoesnotimmediatelyruleoutaneÆcient

procedure,contrarytowhatseemstobethecasebylooking

atFigure11(\All"category). Whetherthereareother

use-fulprotocol-independent properties that are orthogonal to

thethreeabove,isanopenquestion,onewehopetoaddress

infuturework.

P5

P6

P7

P2

P8

P4

P10

P3

P2o

P9

P1

All

Figure11: RelationshipsbetweenPropertiesofg. Anarrow

frompropertyQtopropertyRmeansthatQimpliesR.

6 Monitoring TCP

We now show how to use the techniques described in the

previous sections inorderto monitorTCP [15]

implemen-tations. Forreasonsofspace,weconsideronlythe

receiver-sidebehavior.

TCPimplementsa\slidingwindow"protocoltoimplement

reliable,in-orderdelivery. Totransferalargebuerofdata,

aTCPsenderbreaksitintosmallxed-sizedsegmentsand

sends each numbered segment in aseparate message.

Ini-tially,thesenderassumesthatthe receiverisreadyfor the

rst segment, and that the receiver can accept a certain

count of segments inits \window". It may send some of

thesesegmentsoutonthenetworktothereceiver. The

re-ceiveracknowledges(acks)thesemessages,withasequence

numberof thesegmentimmediately afterthe contiguously

receivedpartofthebuer. Thus,ifsegments1,2,4,and5

havebeenreceived,wheresegment3wasdelayedorlostin

transit,TCPreceivermaygenerateacks2,3,3,and3. The

senderformsajudgmentofwhichsegments(e.g. 3)gotlost

in transit, and resends them. If the receiver now receives

segment3,itgeneratesanackwithsequencenumber6,

be-causesegments1-5havebeenallreceived. Occasionally,the

receiveralso gives an indication of newly available

capac-ityinitswindow,oncesomeprexoftheearliercontiguous

packetsareconsumedbythereceivingapplication.

TheTCPspecicationprescribesthesequencenumberthat

mustbe containedinanack, basedonthestate ofthe

re-ceiverwindowasdescribedabove. It,however,allowsa

lee-way that receivers may generateanack at leasteverytwo

messages,soanimplementationmaydecidetonotackevery

singlemessage.

Typically,wemaywishtomonitorTCPpropertiessuchas

thefollowing:

1. Theimplementationisgenerating anack for at least

everyothermessageitreceives.

2. Acksequencenumbersarenon-decreasing.

3. Acksalwaysacknowledgeexactlythecontiguously

(13)

entailsdierentcomplexitiesinthemonitoringprocess.

Con-sider data segments as input symbols and acks as output

symbols.

TCPproperty1describesagthatisacounting

prop-erty, g 2 P1, with cmin = 1 and cmax = 2. This

propertyis easytomonitorbothinthepresenceand

absenceofloss.

TCP property 2 describes a g in which inputs and

outputs areindependent. Sinceg2P2o, eveninthe

presenceofloss thispropertyis easyto check.

More-over, property2canbe checkedindependentlyofthe

otherproperties.

TCP property 3describes ag that falls inclass P5.

Intheabsenceoflossbetweenthemonitoranddevice,

wecancheckthiseÆcientlyusingthealgorithmin

Fig-ure7. Interestingly,wecaneÆcientlycheckproperty1

inconjunctionwithproperty3,becausethealgorithms

for P1andP5 compose well. Bothdenerangesover

whichthe last outputcould havetakenplace, the

al-gorithmfor the conjunctionsimplymaintains the

in-tersectionfor theseranges.

Thealgorithmdescribedabovedoesnotworkifthere

islossbetweenthemonitoranddevice. However,

prop-erty3canalsobeexpressedasthecompositionofan

output-checkpointedproperty (P6)and a nitestate

property (P7). In order to see this, note that the

TCP monitor's state couldbeseen as a tripleof the

form(unacked-data,contig-recd,recv-window),where

unacked-data is the numberof unacknowledged data

packetssincethelastack,contig-recdisthelargest

se-quencenumberinthecontiguouslyreceiveddata,and

recv-windowisthedatathathasbeenreceivedinthe

windowbutismissingsomesegments. Here

(unacked-data,contig-recd) is output-checkpointed: when you

see anack for sequencenumbers+1, unacked-data

mustbe 0, andcontig-recdmustbe s. Onthe other

hand, the recv-window is nite-state, and its state

space can beas large as 2 W 1

,where W isthe

win-dow size. So to monitor property 3 (together with

property1)wecancomposethealgorithmsforP6and

P7. Thecomplexityofsuchanalgorithmisthesum

ofthecomplexitiesofthetwoalgorithmsas shownin

Table 1,whichisessentiallythecomplexityoftheP7

part(2 W B 2 2 B ).

7 Conclusionsand RelatedWork

Thispaperis mostclosely related toworkonpassive

test-ing[7 ]. Theyalsouseon-linemonitoringtotestprotocol

im-plementations,butdonotaddresstheproblemsofindelity

oftraces. Toour knowledgethispaperis therstattempt

tocreateanabstractmodeloftheindelitiesthatcanarise

whenmonitoringaremotedeviceundertest,andtoformally

specifyconditionsunderwhicheÆcientcorrectness-checking

algorithmscanbedeployed.

Paxson[13 ]givesagoodempirical overviewofsourcesand

manifestations ofindelity,with afocus onwide-area

net-works. Our workconcentratesonindelity inalocal-area

networksettingandisamoreformaltreatment.

Mostoftheotherliteraturedevotedtonetworkmonitoring

istargetedtowardnetworksecurityconcerns. Paxson'sBro

examplesofpracticalmonitoringtoolsthatmustdealwith

remotemonitoringoftrustedanduntrusteddevices. These

tools oer high performance and customizability with the

addition of new specication scripts, but such scripts are

developedinanad-hocmannerandmusttakeindelityinto

accountbyhand.

Testing of reactive systems from abstract specications is

alsoanareaofactiveresearch. Bhargavanetal.[3 ]givean

overviewofnetworking-relatedanalysistechniques. Formal

specicationandautomaticvericationhavealsobeenused

in the context of telecommunications systems [6 , 4 ]. Our

work diers from these in that we are not attempting to

generatetestcasesbutrathertomonitorthecorrectnessof

animplementationduringdeployment. O'Malleyetal.[10 ]

discuss the creation of property checking automata from

a graphical specication notation. Event specications [8]

have beenused for run-timeproperty verication,and the

Verisimtool[2] applies this workto properties of network

protocols. However,allofthesesystemsassumeaco-located

styleof specication|thatis,theyassume thatthe device

undertestcanbedirectlyinstrumentedandpropertiescan

be checkedina synchronousmanner. If the device under

testcannotbedirectlyinstrumentedthenthespecications

used must be transformed to take the resulting indelity

intoaccount. Thispaperhasbeenasystematicexploration

ofsuchtransformations.

References

[1] G.BerryandG.Gonthier. TheEsterelsynchronous

programminglanguage: Design,semantics,

implemen-tation. Science of Computer Programming, 19(2):87{

152, November1992.

[2] Karthikeyan Bhargavan, Carl A. Gunter, Moonjoo

Kim,InsupLee,DavorObradovic,OlegSokolsky,and

MaheshViswanathan. Verisim: Formalanalysisof

net-worksimulations,August2000. Toappearin:

Interna-tional SymposiumonSoftwareTestingandAnalysis.

[3] Karthikeyan Bhargavan, Carl A. Gunter, and Davor

Obradovic. A taxonomy of logical network analysis

techniques. Technical Report MS-CIS-00-14,

Univer-sityofPennsylvania,2000.

[4] Patrice Godefroid. Model checking for programming

languages using VeriSoft. In Proceedings of the 24th

ACM Symposiumon Principles of Programming

Lan-guages,pages174{186,January1997.

[5] Gerard J. Holzmann. Designand Validation of

Com-puter Protocols. PrenticeHall,1991.

[6] L. J. Jagadeesan, A. Porter, C. Puchol, J. C.

Ram-ming,andL.G.Votta. Specication-basedtestingof

re-activesoftware: Toolsandexperiments.InProceedings

of theInternationalConference onSoftware

Engineer-ing,May1997.

[7] D. Lee, K. Sabnani, A. Netravali, B. Sugla, and

A. John. Passive testing and itsapplications to

net-workmanagement. InProceedingsof theInternational

Conference onNetworkProtocols,1997.

[8] I. Lee, S. Kannan, M. Kim, O. Sokolsky, and

(14)

onParallelandDistributed Processing Techniquesand

Applications,1999.

[9] NancyA.LynchandMarkR.Tuttle. Anintroduction

to input/output automata. CWI Quaterly, 2(3):219{

246,1989.

[10] T.O.O'Malley, D.J.Richardson, andL.K.Dillon.

Ef-cientspecication-basedtestoracles. InSecond

Cali-forniaSoftwareSymposium(CSS'96),April1996.

[11] V. Paxson. Automated packettrace analysis of TCP

implementations. Computer Communication Review,

27(4),October1997.

[12] V.Paxson.Bro:Asystemfordetectingnetwork

intrud-ersinreal-time. Computer Networks,31(23-24):2435{

2463,December1999.

[13] V. Paxson. End-to-end internet packet dynamics.

IEEE/ACM Transactions on Networking, 7(3):277{

292,June1999.

[14] M.J.Ranum,K.Landeld,M.Stolarchuk,

M.Sienkiewicz,A.Lambeth,andE.Wall.

Implement-ingageneralizedtoolfor networkmonitoring. In

Pro-ceedings ofthe Eleventh Systems Administration

Con-ference (LISAXI),pages1{8,1997.

[15] W. Richard Stevens. TCP/IP Illustrated, Volume

1: The Protocols. Addison-Wesley, Reading,

Mas-sachusetts,1994.

Appendix: Prooffor P1

Denition2(iqtail) Letiqtail(!)equalthesequenceofiq

elementsin!thatarepriortothelastoqelementbutwhich

were notconsumedbefore theoq,thatis,whichdonot have

acorresponding id element before the oq, plusalliq'sthat

appear after the last oq element, regardless of whether the

corresponding id is in !. If ! does not contain any oq's,

theniqtail (!)isjustthesequence ofiq'sin!.

Denition3(idtail) Letidtail (!)equalthesequenceofid

elementsin!thatappearafterthelastodelement. If!does

not contain any od's, thenidtail (!) isjust thesequence of

id'sin!.

StateSpace Mapping. Themappingisgivenbythe

fol-lowinginductivehypothesis:

:e=)(8i::buf min ibuf max () (9(!;b)2::jiqtail (!)j=i)) ^ e()=

Base Case. Initially,containsonlyoneelement,which

has azero length !and azero lengthb. The initial

con-ditionsforbuf

max ,buf

min

,andeare all consistentwiththe

statespace mapping.

Inductive Step. Considertheinput actionsineach

algo-rithmcorrespondingto iqevents. InFigure 3,this action

deleteseachmemberofandaddsanewmemberwiththe

iqactionaddedtoboth!andb,as longas!isadmissible

anditsprojection[!]

id,od

ising.InFigure4,aninputevent

min max

tion,andthenadjustsbuf

max

ifnecessary.Weneedtoshow

thatthestatespace mappingstillholds. First,notethatif

any(!;b)2hasan!suchthatjiqtail (!)j>=B+c

max ,it

willbedeletedfrombecausetheresultofaddingtheiqto

!willbeinadmissibleaccordingtoM 0

,whichrecallisequal

toM(S;m+cn;0). Inthiscasec=c

max

andB=m+cn.

!willbeinadmissiblebecauseatmostc

max

id'scanappear

afterthelast od;oq bywayofpropertyP1. Theremaining

B iq's ll up the buerand adding onemore causes it to

overow,removingtheresultingstringfromconsideration.

Ifbuf

min

>=B+c

max

beforetheinputaction,this implies

(by the inductive hypothesis) that every (!;b) 2 has a

iqtail (!)whosesizeisatleastB+c

max

.Whentheiqevent

is added, all will be deleted and will be empty. This

corresponds to setting e to true inFigure 4. If buf

min <

B+cmax beforetheinputaction,thenthereisatleastone

(!;b)2suchthatjiqtail (!)j=iforeveryibetweenbuf

min

and buf

max

. Each will be removed and replaced with an

(!;b)pairwithexactlyoneiqeventaddedtoeachof!and

b, except for thosethat becomeinadmissible with respect

toM 0

. Thiscorrespondsexactlytobuf

min andbuf

max being

incrementedbyone, andbuf

max

being cappedatB+cmax

inFigure4. Theiteration step2inFigure3 hasnoeect

on the mapping because it only adds id events to ! and

doesnotmodifyiqtail (!). Note,however,thatthisiteration

guarantees that every pattern of idtail (!) such that 0

jidtail (!)j min(c

max ;buf

max

) will be generated. This is

because each element of the iqtail(!) will be converted to

an id and check-added to . Those with morethan c

max

id'swillbediscarded,sincetheyarenoting,butallothers

willbeacceptedduetopropertyP1. Thelongestiqtailhas

lengthbuf

max

,leadingtotheupperbound.

Next, considerthe outputactions ineachalgorithm

corre-sponding to the oq events. In Figure 3, this action rst

deleteseveryelementof,replacingeachwithonethathas

the od;oq event pair added. Thenthe same iteration

dis-cussed previouslyis repeated. Note that if any (!;b) has

anidtail (!)<c

min

,itwill bedeletedbecause theresultof

addingtheodwillnotbeing,accordingtoP1. InFigure4,

if buf

max

< cmin, then all idtail (!) musthave length less

thancmin, because idtail (!) <iqtail(!)< buf

max

. thus

becomesemptyaftertherststep. Thiscorrespondsto

set-tinge totrueinFigure4. Otherwise, Figure4decrements

buf

max

bycmin,butcapsbuf

max

at B because eventhough

iqtailcangrowlargerthanB,thenewiqtailwillbeatmost

B because itconsistsof onlythe un-consumediqeventsin

!, of which there can be at most B. The algorithm also

decrementsbuf

min

bycmax,settingittozero ifitgoes

neg-ative. Thiscorrespondstothe factthatpairs (!;b)will be

deletedfromiftheyviolatethec

min andc

max

limitsgiven

by g, whichmeans that the new minimum and maximum

valuesofiqtail (!)willbegivenwhenthemaximumnumber

of inputs are consumed from the minimum previous iqtail

andthemaximumnumberofinputsareconsumedfromthe

maximumpreviousiqtail. These consumptionsarejustied

because we have a guarantee that everypattern idtail (!),

suchthat0jidtail (!)jmin(cmax;buf

max

),willhavebeen