Norm approximation for imperfect monitors

Norm Approximation for Imperfect Monitors

Natasha Alechina

University of Nottingham Nottingham NG8 1BB, UK

[email protected]

Mehdi Dastani

Computing Sciences Universiteit Utrecht

The Netherlands

[email protected]

Brian Logan

University of Nottingham Nottingham NG8 1BB, UK

[email protected]

ABSTRACT

In this paper, we consider the runtime monitoring of norms with imperfect monitors. A monitor is imperfect for a norm if it has insufficient observational capabilities to determine if a given exe-cution trace of a multi-agent system complies with or violates the norm. One approach to the problem of imperfect monitors is to en-hance the observational capabilities of the normative organisation. However this may be costly or in some cases impossible. Instead we show how to synthesise an approximation of an ‘ideal’ norm that can be perfectly monitored given a monitor, and which is opti-mal in the sense that any other approximation would fail to detect at least as many violations of the ideal norm. We give a logical analy-sis of (im)perfect monitors. We state the computational complexity of the norm approximation problem, and give an optimal algorithm for generating optimal approximations of norms given a monitor.

Categories and Subject Descriptors

I.2.11 [Artificial Intelligence]: Distributed Artificial Intelligence—

Multi-agent systems

Keywords

1.

INTRODUCTION

Norms have been widely proposed as a means of coordinating and regulating the behaviours of agents within a multi-agent system (MAS). In a normative multiagent system, the interaction between agents and their environment is governed by anormative organ-isationspecified by a set of norms (obligations and prohibitions). Individual agents update the state of the environment by performing actions. The role of the normative organisation is to continuously evaluate the state updates resulting from agent actions with respect to the norms to 1) determine any new obligations to be fulfilled or prohibitions that should not be violated (termed detached obliga-tions and prohibiobliga-tions), 2) check if any previously detached norms are obeyed or violated in the current state, and 3) impose sanctions when norms are violated. This continuous process is often imple-mented by a so-called control cycle [9].

For a system of norms to have their intended effect, it must be possible for the normative organisation to monitor the environment

This is author’s accepted manuscript version of an AAMAS 2014 paper

updates to determine when norms are detached, and when a de-tached norm is obeyed or violated. In general, previous work on normative organisations has implicitly assumed that the environ-ment is fully observable, and hence can be perfectly monitored. For example, in determining whether a set of norms will have their in-tended effect, [3] assumes the environment can be perfectly moni-tored. In practice, this assumption may not hold. Norms specify the ideal behaviours of an agent or agents independent of the realisation of these behaviours in a particular MAS. However the environment of a particular multi-agent system may not be fully observable, or some facts may be observable only at great cost. For example, con-sider a university norm ‘feedback should be returned to a student by the deadline, otherwise a reminder is sent to the lecturer’. It is dif-ficult to automatically monitor whether proper feedback has been sent, but it is possible to check that a message contains a grade, a minimal amount of text for each assessed component etc. As an-other example, consider a long motorway where the norms to be enforced are driving regulations, such as speed limit etc. It is too expensive to position monitoring equipment so that the norms are monitored along the entire motorway. In such environments, it is impossible or infeasible for the normative organisation to monitor (and so enforce) the ideal norms specified by the designer of the normative system.

One way of solving this problem is to change the ‘ideal’ norms such that all violations of the new norm become detectable. This raises further problems: which of the possible modifications or ap-proximations of the original ‘ideal’ norm is ‘closest’ to the intent of the designer of the normative system. Continuing with the mo-torway example, one possible choice is between using sparse but accurate speed cameras (and essentially approximating the speed limit norm to mean that speeding is not permitted in the neigh-bourhood of the speed camera) or cheaper, more numerous but less accurate cameras (which means that the norm is approximated to prohibit speeding in more places, but adjusting the precise value of the speed limit).

Such approximate norms are important for several reasons. Firstly, norm-aware agents, e.g., [2], need to know which norms are actu-ally in force in a normative MAS in order to make a rational de-cision whether to comply with a norm. Secondly, the designer of a normative MAS needs to know which approximations of norms correspond to the behaviours that can actually be monitored by the monitoring capabilities of a particular normative organisation in or-der to determine whether the realisable approximation of a set of ideal norms will have an effect that is sufficiently close to the ideal system behaviour specified by the ideal norms.

observ-able properties and norms, we show how to automatically synthe-sise optimal (in the sense of minimising undetected violations) ap-proximations of the ideal norms defined in terms of the observable properties. We show that the procedure is also optimal in the sense that its complexity matches the complexity of the norm approxima-tion problem.

The structure of the paper is as follows. In section 2 we define conditional norms and (perfect) monitors. In section 3 we introduce an example. In section 4 we introduce imperfect monitors which do not have perfect observational capabilities. In section 5 we provide an optimal procedure for norm approximation. We survey related work and conclude in section 6.

2.

CONDITIONAL NORMS

We focus on conditional normswith deadlines and sanctions [16], as these represent a reasonable compromise between expres-sive power and ease of reasoning about norms and monitoring.

DEFINITION1 (NORMS). Letcond,φ,dandsanbe boolean combinations of propositional variables from some propositional languageLN. Aconditional obligationis represented by a tuple

(cond, O(φ), d, san)and aconditional prohibitionis represented by a tuple(cond, P(φ), d, san). A norm setN is a set of condi-tional obligations and condicondi-tional prohibitions.

We assume that the possible behaviours of the agents are rep-resented by a transition system. A transition systemM is a tuple (S, R, V, sI)whereSis a set of states,sI ∈Sis the initial state,

Ris the transition relation (intuitively, corresponding to possible actions the agents can perform) andV is a labelling of states with propositional variables. Conditional norms are evaluated on runs of the transition system. A conditional normn = (cond, Z(φ), d, san), where Z isOorP, isdetachedin a state satisfying its conditioncond. Detached norms persist as long as they are not obeyed or violated, even if the triggering condition of the corre-sponding conditional norm does not hold any longer. A detached obligation(cond, O(φ), d, san)isobeyedif no state satisfyingdis encountered before execution reaches a state satisfyingφ, and vio-latedif a state satisfyingdis encountered before execution reaches a state satisfyingφ. Conversely, a detached prohibition(cond, P(φ), d, san)is obeyed if no state satisfyingφis encountered be-fore execution reaches a state satisfyingd, and violated if a state satisfyingφis encountered before execution reaches a state satis-fyingd. If a detached norm is violated in a states, the sanction corresponding to the norm is applied (becomes true) ins. We say a detached norm is annulled in a states0immediately after a state sin which the norm is obeyed or violated, unless the same norm is detached again ins0. As our focus in this paper is on the mon-itoring of norms, in the interests of readability we abstract from sanctions and omit thesancomponent from the representation of norms below.

In what follows, we focus on monitoring for violations of condi-tional norms: that is, determining if a particular run of the system violates a conditional norm. Algorithm 1 illustrates how a condi-tional obligation can be monitored on a finite pathρ. The function VIOLreturns trueif the conditional norm(cond, O(φ), d)is vi-olated in a stateρ[i]at positionionρandfalseotherwise. The algorithm for monitoring for violations of conditional prohibitions is similar and is illustrated in Algorithm 2.

A (perfect) monitor for a conditional norm is a device which takes a finite path and a conditional norm and returns true if the path violates the norm and false if it does not violate the norm, in other words it implements something like the algorithms above. Note

Algorithm 1Violation of a Conditional Obligation

bool det←false

functionVIOL((cond, O(φ), d), ρ)

fori←1toρ.lengthdo ifρ[i]|=condthen

det←true end if ifdetthen

ifρ[i]|=d∧ρ[i]6|=φthen returntrue

else ifs|=φthen det←false end if

end if end for returnfalse end function

Algorithm 2Violation of a Conditional Prohibition

bool det←false

functionVIOL((cond, O(φ), d), ρ)

fori←1toρ.lengthdo ifρ[i]|=condthen

det←true end if ifdetthen

ifρ[i]|=φ∧ρ[i]6|=dthen returntrue

else ifs|=dthen det←false end if

end if end for returnfalse end function

that in order to be able to implement the algorithms, the monitor needs to be able to perform the boolean queriescond,φanddon the states. This means that the states must be perfectly observable as far as these queries are concerned.

Another way to give precise meaning to violations of conditional norms is to express them asPLTLformulas. First we need to in-troduce some background onPLTL. We follow the semantics of PLTLon finite traces adopted in [12]. Given a transition system M = (S, R, V, sI)with an initial statesI ∈ S, apaththrough

M is a sequence s0, s1, s3, . . . of states such that siRsi+1 for i= 0,1, . . .and the first state iss0 =sI. In this paper, we

con-sider finite paths. We denote paths byρ, ρ0, . . . ,and the state at positionionρbyρ[i]. The syntax ofPLTLformulas is defined relative to a set of propositional variablesΠas follows:

p∈Π| ¬φ|φ∧ψ| Yφ|φSψ

whereYmeans previous state unless we are in the first state of the sequence, in which case it means the current state, andSstands for since. The truth definition for formulas is given relative toM, a pathρand the state at positionionρ:

M, ρ, i|=piffp∈V(ρ[i])

M, ρ, i|=¬φiffM, ρ, i6|=φ

M, ρ, i|=φ∧ψiffM, ρ, i|=φandM, ρ, i|=ψ

M, ρ, i|=Yφiff eitheri >0andM, ρ, i−1|=φ, ori= 0and M, ρ, i|=φ.

Now we are ready to give a translation inPLTLof the violation conditions of norms (essentially the same translation as given in [3]):

DEFINITION2 (VIOLATIONFORMULA). Aviolation formula

for a conditional obligation(c, O(φ), d)is

d∧ ¬φ∧((Y(¬φ∧ ¬d)S(c∧ ¬φ∧ ¬d))∨c)

Aviolation formulafor a conditional prohibition(c, P(φ), d)is

¬d∧φ∧((Y(¬φ∧ ¬d)S(c∧ ¬φ∧ ¬d))∨c)

We will denote a violation formula for a conditional normnby v(n). A normnis violated on a pathρif, and only if,v(n)holds in some state onρ. An obligation(c, O(φ), d)is violated onρif in some state the deadline descriptiondis true, and it holds that some time in the past the condition of the norm became true and since then the obligation descriptionφhas not been true (and is not true in the current state). A prohibition(c, P(φ), d)is violated onρif somewhere onρthere exists a state where the prohibition descriptionφis true, and some time in the past the condition of the norm became true and since then the deadline description has not become true.

THEOREM 1. A conditional norm n is violated on a path ρ

in modelM if and only if there exists a stateρ[i]onρsuch that

M, ρ, i|=v(n).

PROOF. Obvious from the conditions of norm violation given by Algorithms 1 and 2.

Clearly, conditional norms are not as expressive as arbitraryPLTL formulas.

THEOREM 2. The language of violation formulas is strictly less expressive than the fullPLTLlanguage.

PROOF. In order to prove this, we exhibit an operation on runs which preserves violation formulas but does not preserve arbitrary PLTLformulas. Consider a runρand an operationdupwhich duplicates states onρ. Clearly, if a formula of the formv(n)is true onρ, then it is true ondup(ρ). However, arbitraryPLTLformulas are not preserved under this operation.

Given that conditional norms are less expressive thanPLTL for-mulas, one might ask why not usePLTLfor expressing norms? In addition to the simplicity of Algorithms 1 and 2 and intuitiveness of the meaning of the norms, there is another consideration which is important for the topic of this paper: conditional norms are easier to approximate under restricted observability of states than arbitrary PLTLformulas.1

3.

EXAMPLE

In this section, we introduce a running example to illustrate the notions introduced in the paper. Consider the transition system shown in Figure 1 which represents possible behaviours of a car, where the propositional variables have the following meanings:

• pi: the car is identified at positioni.

• sx: the speed of the car isxmph.

1_{Monitoring for violations of conditional norms has slightly lower}

complexity than monitoringPLTLformulas. Detecting violations of a conditional norm using Algorithms 1 and 2 requires time linear inρand constant space (since monitoring of a conditional norm requires considering only the current state and a single boolean flag detindicating whether an instance of the norm has been detached in a previous state), while evaluating a past timePLTLformulaψ on a finite path requires time linear inρand space linear inψ[12]. However such differences are unlikely to be significant in practice.

p₂, s₆₀

p₃

p₁ p₂, s₇₀

s₃ s₁

p₂, s₈₀

s₅ s₂

s₄

Figure 1: Example transition systemMt.

Consider the norm prescribing that a car should have a maximum speed of 60 mph between positions1and3on a ringway road. As-suming that the only possible values for the speed are 60, 70 and 80 mph, this conditional norm can be represented as a conditional prohibitionn1= (p1, P(p2∧(s70∨s80)), p3)or as a conditional obligationn2 = (p1, O(p2 ∧s60), p3). Clearly, both norms are violated on the pathss1, s3, s5ands1, s4, s5. Ons1, s3, s5, prohi-bitionn1is detached ins1and violated ins3. On the same path, obligationn2is detached ins1and violated ins5.

4.

IMPERFECT MONITORS

In previous sections, we considered monitoring norms under the condition of perfect observability. We could always evaluate boolean queries used in the norms in each state. However, in many situa-tions, such perfect observability may not be possible. For example, we may only have a speed camera that can distinguish between speeds of 70 mph and less and speeds over 70 mph. To study mon-itoring under partial observability, we generalise the notion of a monitor to include a set of queries which the monitor can ask of a state. ‘Standard’ monitors then become a special case of our moni-tors, where the set of possible queries includes all possible queries definable in the language of conditional norms.

We start with a languageLwhich is given as a set of formulas; we don’t assume anything about the formulas apart from that they can be evaluated in the states of the system to true or false. Intu-itively, the formulas inLdefine the set of possible observations or queries with a yes or no answer. For example, a formula may say that an object in front is a car, or that it is moving at a speed in excess of 100 mph, or that it is a red Lamborghini, or that there is no person in the driving seat, etc.

DEFINITION3 (MONITOR). A monitormΦ(whereΦ⊆Lis

a finite set of formulas) is a function which given a finite pathρand a normn, returns true, false or undefined.

usemΦto perfectly monitor for violations of norms ifΦdoes not contain all the formulas required to express the norms. To char-acterise the relation between (imperfect) monitors and norms, we need some technical definitions, which characterise the expressive power ofΦwith respect to different norms and a transition system (S, R, V, sI).

The monitor queriesΦ ={φ1, . . . , φk}define a partition of the

setS of states of a transition system in the following way. Two states are equivalent or indistinguishable (are in the same equiva-lence class in the partition) with respect to the monitormΦ, denoted bys∼Φs0, if they give the same answers to all the queries inΦ2. Each equivalence class[s]Φ, s∈ S in the partition defined byΦ is definable by a conjunction∼ φ1∧. . .∧ ∼φkwhere∼φiis

φiifsanswers yes to (satisfies)φi, and is¬φiifsanswers no to

(does not satisfy)φi. We will call the∼φ1∧. . .∧ ∼φkthe ‘types

definable by the monitormΦ’ or just ‘types ofmΦ’,T(m). Note that a monitormΦcan partition a setSof states into at most2|Φ| types because it is possible that there are no states inSof a type defined bymΦ.

We lift the partition of states of a transition system to a partition of paths. LetΓbe the set of all possible finite paths of a transition system. We say that two pathsρ, ρ0∈Γare equivalent with respect to a monitormΦ, denoted asρ ∼Φ ρ0, if the paths are state-wise equivalent with respect toΦ, i.e.,

ρ∼Φρ0iff∀i: ρ[i]∼Φρ0[i]

whereρ[i]is thei-th state on pathρ. A monitormΦthus defines a partition of the set of pathsΓ. We write[ρ]Φ, ρ∈Γto denote the equivalence class containing the pathρ, andΓ/∼Φfor the partition ofΓ(the set of equivalence classes).

Coming back to our running example in Figure 1, suppose we have a monitor with the set of queries:Φ ={p1, p2, p3,s60∨s70} so that the monitor can identify the car’s position and whether its speed is strictly above 70 mph. These queries specify four equiv-alence classes on the states:{{s1},{s5},{s2, s3},{s4}}. These equivalence classes are respectively characterised by the monitor types (p1 ∧ ¬p2 ∧ ¬p3 ∧ ¬s60∧ ¬s70∧ ¬s80), (¬p1∧ ¬p2∧ p3∧ ¬s60∧ ¬s70∧ ¬s80), (¬p1∧p2∧ ¬p3∧s60∧s70∧ ¬s80), and (¬p1∧p2∧ ¬p3∧ ¬s60∧ ¬s70∧s80). Note that the equiv-alence classes characterised by all other monitor types are empty given a proper domain theory (which states that the car is always in exactly one position, has at most one speed, etc.). This moni-tor cannot distinguish, for example, the pathsρ1 =s1, s2, s5and ρ2=s1, s3, s5:ρ1∼Φρ2.

Given a transition systemM, a conditional normnwithPLTL violation formulav(n), a monitormΦ, and a pathρ∈Γ, we say thatmΦobservesρobeyingn, denoted asobey(mΦ, ρ, n), ifv(n) is false in all states on all paths in[ρ]Φ, i.e.,

obey(mΦ, ρ, n) ⇔ ∀ρ0∈[ρ]Φ∀i:M, ρ0, i6|=v(n)

Note that if we haveobey(mΦ, ρ, n), then we also have∀ρ0 ∈ [ρ]Φ : obey(mΦ, ρ0, n). In other words, the equivalence class [ρ]Φ consists of paths obeying conditional normn. The set of equivalence classes that contain paths obeyingn is denoted by O(mΦ, n) ={[ρ]Φ | obey(mΦ, ρ, n)}.

We say that the monitormΦobserves the pathρviolating the conditional normn, denoted asviol(mΦ, ρ, n), ifv(n)is true in

2

Indistinguishability by queries inΦis clearly a special case of an epistemic indistinguishability relation. The monitormΦis guaran-teed to ‘know whetherψ’ (i.e.,KmΦψ∨KmΦ¬ψ) ifψis definable

using formulas inΦ, and is not guaranteed to know whetherψ oth-erwise.

some state on all paths in[ρ]Φ, i.e.,

viol(mΦ, ρ, n) ⇔ ∀ρ

0

∈[ρ]Φ∃i:M, ρ

0

, i|=v(n)

The set of equivalence classes that contain paths violatingnis denoted byV(mΦ, n) ={[ρ]Φ | viol(mΦ, ρ, n)}.

Finally, we say that the monitormΦcannot observe whetherρ violates or obeys the conditional normn, denoted asviob(mΦ, ρ, n), and define it as follows:

viob(mΦ, ρ, n) ⇔ ∃ρ0∈[ρ]Φ∀i:M, ρ0, i6|=v(n) & ∃ρ0∈[ρ]Φ∃i:M, ρ0, i|=v(n)

The set of equivalence classes that contain paths some obey-ing and some violatobey-ingnis denoted byOV(mΦ, n) = Γ/∼Φ\ (V(mΦ, n)∪O(mΦ, n)).

DEFINITION4 (MONITOR BEHAVIOUR). Given a set of mon-itor queriesΦ, the monitor functionmΦ(ρ, n)is defined as follows:

mΦ(ρ, n) =







true ifviol(mΦ, ρ, n) false ifobey(mΦ, ρ, n) undefined ifviob(mΦ, ρ, n)

Note that a perfect monitor never returnsundefined.

Coming back to the running example, the transition systemMt

shown in Figure 1 generates paths(s1(s2|s3|s4)s5)∗. For the obli-gationn2 = (p1, O(p2 ∧s60), p3) we have O(mΦ, n2) = ∅, V(mΦ, n2) ={[ρ]Φ | ∃i Mt, ρ, i|=s80}, andOV(mΦ, n2) = {[ρ]Φ| ∃i Mt, ρ, i|=s60∨s70}contains all partitions including paths some obeying and some violatingn2. This means thatmΦis not a perfect monitor forn2onMt.

DEFINITION5 (PERFECT/IMPERFECTMONITORS). LetΓbe the set of all possible paths of a transition system,nbe a condi-tional norm, andmΦbe a monitor defining a partition ofΓ. We say

thatmΦis aperfect monitorfornif and only ifOV(mΦ, n) =∅,

andmΦis animperfect monitorfornif and only ifOV(mΦ, n)6= ∅.

In other words, a perfect monitor for a conditional norm is able to identify all paths as either obeying or violating the norm while an imperfect monitor is unable to identify some paths as obeying or violating the norm. It is important to emphasise that the distinc-tion between perfect and imperfect monitors is a relative distincdistinc-tion which depends on the set of paths and the norm under considera-tion. A monitor can be imperfect for a set of paths and a norm while it can be perfect for a subset of the paths (or a different set of paths) and the norm. It is however clear that ifΦcontains all the queries which are needed to define formulasc,φanddin a norm n= (c, Z(φ), d), thenmΦis a perfect monitor forn:

PROPOSITION 1. Letn= (c, Z(φ), d)be a norm, andΦa set of queries such that there exist a boolean combination of queries in

Φwhich can be used to equivalently expressc,φandd. ThenmΦ

is a perfect monitor forn.

PROOF. Ifc,φanddcan be equivalently rewritten in terms of queries inΦ, thens∼Φs0implies that:s|=ciffs0|=c,s|=φ iffs0|=φ, ands|=diffs0|=d.

¬d ¬!

d ¬! c ¬d

¬! j

¬d ¬!

k1 km i

and for a prohibition

¬d ¬!

¬d

!

c ¬d ¬!

j

¬d ¬!

k1 km i

For everyρ0such thatρ∼Φ ρ0, the same pattern will exist onρ0, sinceρ[j] ∼Φ ρ0[j], ρ[i] ∼Φ ρ0[i]and for everyk in between, ρ[k] ∼Φ ρ0[k]. Hence,ρ0, i|=v(n). So,OV(mΦ, n)is empty: each equivalence class either contains only the paths violating the norm, or the paths which do not.

The distinction between perfect and imperfect monitors can be refined by introducing the notionmonitor sensitivity. Until now we have assumed a set of state formulaeΦ = {φ1, . . . , φn}, which

uniquely specifies a monitor. This assumed set of state formulae determines the sensitivity of the monitor with respect to a set of paths and a norm as it defines the monitor’s ability to identify paths as obeying or violating the given norm. When the monitormΦis perfect for a set of pathsΓand a normn(i.e.,Γ/∼Φ\(V(mΦ, n)∪ O(mΦ, n)) = ∅), we say thatmΦis fully sensitive forΓandn. However, when the monitor is imperfect, there exist partitions that contain paths some of which obey and some others violaten.

ExtendingΦwith additional state formulae may make a monitor more sensitive in the sense that the set ofviobpaths shrinks while restrictingΦmay make the monitor less sensitive in the sense that the set ofviobpaths may grow.

PROPOSITION 2. LetΦ1andΦ2be two sets of state formulae, Γbe a set of paths,nbe a conditional norm, andmΦ1andmΦ2be

two monitors defining two partitions ofΓ. Then, we have

Φ1 ⊆Φ2 ⇔

[

OV(mΦ2, n)⊆

[

OV(mΦ1, n)

PROOF. Straightforward.

Adding extra queries certainly allows us to implement better monitors for norms. However, sometimes providing extra queries may be too costly or impossible. A complementary approach is, given a set of queriesN and a fixed set of queriesΦ, come up with a set of approximations ofNfor whichmΦis a perfect moni-tor. Intuitively, this corresponds to asking, if we cannot enforce the normsN, which norms can we enforce? This is the topic of the next section.

5.

APPROXIMATING NORMS

As we have seen in the previous section,mΦis a perfect monitor for a normnwith condition c, obligation or prohibition φ, and deadlinedifc,φanddcan be defined in terms of formulas inΦ. This, however, is not always the case. Sometimes the designer of a normative system has a given set of possible queries and has to modify the norms so that a (perfect) monitor can be implemented for the new norms, while minimising the difference in terms of missed violations between the old and the new norms.

Given a set of queriesΦand a set of ‘ideal’ normsN, thenorm approximationtask is to produce a set of normsN0for whichmΦ is a perfect monitor and which is the best approximation forN. By ‘best’ we mean that the set of violations of the approximated norm is included in the set of violations of the original norm, and the difference between the two sets is minimal.

Let us denote the language in which the conditions, deadlines, obligations and prohibitions ofN are formulatedLN. We assume

that we are given a propositional theoryTin the language ofΦand LNwhich contains some axioms relatingΦandLN. For example,

T may contain a statement thats80→ ¬(s60∨s70)wheres80 ∈ LNands60∨s70∈Φwhich says that the speed of 80 mph is not a speed of 60 or 70 mph.Tmay also be empty, in which case we only have classical propositional logic to reason about the relationship of queries definable usingΦandLN. For example, ifp∈LNand

p∨q∈Φ(butp6∈Φ) we still know thatp→p∨q.

In what follows, we assume that all transition systemsM are models ofT(that is, axioms ofTare true in all states ofM).

The set of paths violatingnis defined relative to some transition systemM.

DEFINITION6 (VIOLATIONS). Given a systemM = (S, R, V, sI)

and a normn, a set of violations ofnonMViol(M, n)is

{ρ| ∃i M, ρ, i|=v(n)}

A norm n can be approximated (for detecting violations) by weakeningnto a normn0such that all paths violatingn0also vio-laten. This means that each violation detected forn0is a violation ofn, but if we monitor for violations ofn0, we may miss some vio-lations ofn; that is, we get some false negatives for violations ofn, but never false positives. This kind of approximation makes sense in a context where agents are penalised for violating norms, and if we do not have a perfect monitor forn, we want to weakennso we have a perfect monitor for its approximation. This way, agents will never be ‘punished unfairly’.

DEFINITION7 (APPROXIMATION). A conditional normn0is aapproximation ofnwith respect toΦand a background theoryT

(a(Φ, T)-approximation ofn) iffn0is formulated in terms of the queries inΦand the set of violations ofn0is a subset of the set of violations ofnin all possible modelsM ofT:

∀M(Viol(M, n0)⊆Viol(M, n))

DEFINITION8 (OPTIMAL APPROXIMATION). A norm n0 is

the optimal approximationofnwith respect toΦand a background theoryT(an optimal(Φ, T)-approximation ofn) iff it is a(Φ, T) -approximation ofnand for all other(Φ, T)-approximationsn00of

n,

∀M(Viol(M, n00)⊆Viol(M, n0))

The reason we define approximation (and optimality) with re-spect to all possible models and not with rere-spect to a single model is because the latter notion is not robust. An optimal approximation with respect to a specificM may be exploiting some ‘accidental’ features ofMwhich may become true if the environment model is slightly adjusted or the agents are given one extra possible transi-tion which changes the set of paths. For example, it is possible that inMon every path where some normnis detached, the state be-fore the conditioncof the norm becomes true satisfies somep∈Φ which has no connection withcin theoryT. We can obtain an opti-mal approximation fornonMby replacingcwithp. Clearly, this approximation may break down if it becomes possible to reach ac state without going through apstate.

Next we state how to define an optimal approximation with re-spect toΦandTof a set of normsNwhich is stated in the language LN. We assumeLNandΦare finite.

We can construct all possible state descriptions givenT, which is a finite (although in the worst case exponential in|Φ|+|LN|)

with respect toTand is a conjunction of all possible formulas inΦ andLNor their negations. Let us denote the set of all possible state

descriptions byS(Φ, LN, T). For each formulaφin the combined

languageLN∪Φ, we can define a functionval(φ)which mapsΦ

to the state descriptions inS(Φ, LN, T)whereφis true:

val(φ) ={α∈S(Φ, LN, T)|α|=φ}

DEFINITION9 ((BEST)INNER APPROXIMATION). For two propositional formulasφandφ0in the language ofLN∪Φ, we will

say thatφ0is aninner approximationofφwith respect toΦandT

((Φ, T)- inner approximation ofφ) iffφis written using boolean combinations of queries inΦandval(φ0)⊆val(φ).

φ0is thebest inner approximationofφwith respect toΦandT

(best(Φ, T)-inner approximation ofφ) iffφ0is a(Φ, T)-inner ap-proximation ofφand for every other(Φ, T)-inner approximation ofφ,φ00,val(φ00)⊆val(φ0).

DEFINITION10 ((BEST)OUTER APPROXIMATION). For two propositional formulasφandφ0in the language ofLN∪Φ, we will

say thatφ0is anouter approximationofφwith respect toΦandT

((Φ, T)- outer approximation ofφ) iffφis written using boolean combinations of queries inΦand

val(φ)⊆val(φ0)

φ0 is thebest outer approximationofφwith respect toΦandT

(best(Φ, T)-outer approximation ofφ) iffφ0is a(Φ, T)-outer ap-proximation ofφand for every other(Φ, T)-outer approximation ofφ,φ00,

val(φ0)⊆val(φ00)

Next we show how to construct best inner and outer (Φ, T )-approximations of propositional formulas.

Observe that any formulaφ0which is definable in terms of queries fromΦcan be equivalently written asW

θi, where eachθi∈ T(mφ),

that is, is of the form

(∼φ1∧. . .∧ ∼φk)

where∼φjis eitherφjor its negation. For each of the2|Φ|

possi-bleθi, we compute the subsetval(θi)ofS(Φ, LN, T)whereθiis

true. We define

φ−(Φ,T)=

_

{θi∈T(mΦ)|val(θi)⊆val(φ)}

θi

φ+_(Φ,T)= _

{θi∈T(mΦ)|val(θi)∩val(φ)6=∅}

θi

In what follows, we will omit the subscript(Φ, T)inφ−_(Φ,T)and φ+_(Φ,T)for readability.

Clearly,T|=φ−→φandT |=φ→φ+ .

Coming back to the running example in Figure 1, observe that the languageLNcontains{p1, p2, p3, s60, s70, s80}, andΦ ={p1,p2, p3,s60∨s70}. The domain theoryTwill contain statements about the speed of the car and position of the car being unique, and that the car always has some position, etc. The setS(Φ, LN, T)of all

possible state descriptions contains, for example, a description of s4:α4=¬p1∧p2∧ ¬p3∧ ¬s60∧ ¬s70∧s80, similarly fors2and s3. Note that the set of monitor types or state descriptions in terms ofΦis smaller. It only contains two rather than three possibilities for the car being in positionp2:

θ1=¬p1∧p2∧ ¬p3∧(s60∨s70)

θ2=¬p1∧p2∧ ¬p3∧ ¬(s60∨s70)

For the formulaφ= (p2∧s60),φ−=⊥andφ+=θ2. Before we prove thatφ−and φ+

are the best inner and outer approximations ofφ, we need a lemma which will also be useful in later proofs.

LEMMA 1. If a state descriptionα∈ S(Φ, LN, T)satisfies a

formulaφ0written using boolean combinations of formulas from

Φ, and does not satisfyφ−, thenφ0is not a(Φ, T)-inner approxi-mation ofφ.

If a state descriptionα ∈ S(Φ, LN, T)satisfiesφ+and does

not satisfy a formulaφ0written using boolean combinations of for-mulas fromΦ, thenφ0is not a(Φ, T)-outer approximation ofφ.

PROOF. Straightforward.

PROPOSITION 3. φ−_(Φ,T)andφ+_(Φ,T)are the best inner and outer

(Φ, T)-approximations ofφ.

PROOF. Clearly,val(φ−) ⊆ val(φ)andval(φ) ⊆ val(φ+). To show thatφ−is the best inner approximation ofφ, assume that there is another(Φ, T)-inner approximationφ0 of φand a state descriptionα∈val(φ)such thatα∈val(φ0)andα6∈val(φ−). By Lemma 1,φ0is not an inner approximation ofφ.

The proof thatφ+_{is the best outer approximation ofφis} simi-lar.

Next we state the method to produce the optimal(Φ, T )-approxi-mations for violations given the original set of norms, and a theo-rem which states that any other approximation will miss at least as many violations as the one computed by the method.

Given an obligation(c, O(φ), d), we need to produce an obliga-tion(c0, O(φ0), d0)wherec0, φ0, d0are definable in terms of queries fromΦ. We do this as follows:

• c0=c−(c0is the best inner approximation ofc, fewer states satisfyc0thanc)

• φ0=φ+_(φ0

is the best outer approximation ofφ, more states satisfy the obligation descriptionφ0)

• d0=d−(d0is the best inner approximation ofd, fewer states satisfyd0)

These conditions are intended to guarantee that the approximation of an obligation is violated only if the original obligation is vio-lated:

LEMMA 2. For any conditional obligationn = (c, O(φ), d), its approximationn0= (c−, O(φ+), d−), modelM, pathρinM, if∃i M, ρ, i|=v(n0), then∃i M, ρ, i|=v(n).

PROOF. AssumeM, ρ, i |= v(n0), that is, M, ρ, i |= d−∧ ¬φ+_∧((Y(¬φ+_{∧ ¬d}−

)S(c−∧ ¬φ+_{∧ ¬d}−

))∨c−). This means that there is a pattern onρ:

¬d

-¬!+

d

-¬!+

c-¬d

-¬!+

j

¬d

-¬!+

k1 km i

where possiblyj=i(in which casem= 0).

So, ifM, ρ, j |=c−∧ ¬φ+

, then alsoM, ρ, j |=c∧ ¬φ, and ifM, ρ, i |=d−∧ ¬φ+, then alsoM, ρ, i |= d∧ ¬φ. However M, ρ, k|=¬φ+∧ ¬d−does not implyM, ρ, k|=¬φ∧ ¬d (be-cause¬d−is easier to satisfy). Basically, a violation of nmay occur in a different state onρ(beforei), however if there is a vio-lation ofn0(some indexksatisfies¬φ+) then there is a violation ofn, son0is an approximation.

For a prohibition(c, P(φ), d), wherec,φ, we need to produce a prohibition(c0, P(φ0), d0)wherec0, φ0, d0are definable in terms of queries fromΦ. We define the approximation as(c−, P(φ−), d+). This definition is intended to guarantee that the approximation of a prohibition is violated only if the original prohibition is violated:

LEMMA 3. For any conditional prohibitionn= (c, P(φ), d), its approximationn0= (c−, P(φ−), d+₎

, modelM, pathρinM, if∃i M, ρ, i|=v(n0), then∃i M, ρ, i|=v(n).

PROOF. The proof is similar to the proof of Lemma 2.

Now we can prove correctness of our construction of the set of optimal norm approximations.

THEOREM 3. GivenNandΦ, the following setN0is the set of optimal approximations of the norms inN:

N0 = {(c−, O(φ+), d−)|(c, O(φ), d)∈N} ∪ {(c−, P(φ−), d+)|(c, P(φ), d)∈N}

PROOF. By Proposition 1, norms inN0can be perfectly moni-tored bymΦsince they are defined in terms ofΦ. We need to show for everyn∈N, that its approximationn0is an approximation of nfor violations, formally on everyM,

Viol(M, n0)⊆Viol(M, n)

and that it is optimal, i.e., for every other approximationn00,

∀M(Viol(M, n00)⊆Viol(M, n0))

To prove thatn0is an approximation ofnfor violations, suppose that for someM,ρ∈Viol(M, n0), that is, for somei,M, ρ, i|= v(n0). From Lemmas 2 and 3 it follows that for everyM,ρif for somei,M, ρ, i|=v(n0)then for somej,M, ρ, j |=v(n). This means thatρ ∈ Viol(M, n). So, for everyM,Viol(M, n0) ⊆ Viol(M, n).

To prove thatn0is an optimal approximation ofnfor violations, suppose by contradiction that for some other normn00definable in terms ofΦwhich is an approximation ofn, there is some modelM ofT and a pathρsuch that:

1. for somei,M, ρ, i|=v(n)(there is a violation ofn)

2. for somei0,M, ρ, i0|=v(n00)(it is also a violation ofn00)

3. for alli,M, ρ, i6|=v(n0)(it is not a violation ofn0, so mon-itoring forn0this violation ofnwill be missed)

Let us consider the case whennis a prohibition,n= (c, P(φ), d). Sincenis violated on someρ, there exists a pattern of states onρ

¬ d ¬!

¬ d

!

c ¬ d ¬!

j

¬ d ¬!

k1 km i

Sincen00is violated onρ, there exists a similar pattern of states onρforc00andφ00. Let us call the state wherec00is trues1, and the state whereφis trues2. Note that it is possible thats1 =s2.

Observe also that eithers1 6|=c0, ors26|=φ0, or some state between s1ands2onρsatisfiesd0but notd00. It can be shown in all these cases that then we can construct a modelM0and a pathρ0inM0 such thatρ0contains a violation ofn00but not ofn, son00is not an approximation ofn.

Consider the case whens1 6|= c0ands1 6=s2. By Lemma 1, c00 is not an inner approximation ofc, hence there exists a state descriptionαsuch thatα∈val(c00)andα6∈ val(c). Consider a states0which conforms to this description, and consider a model M0which consists of a single step path froms0tos2. This path will be a violation ofn00sinces0 satisfiesc00ands2 satisfiesφ00, but it will not be a violation ofnsincenis not detached ins0. Let us consider the case whens1 =s2. Suppose by contradiction that it is impossible to construct a states0satisfyingc00and andφ00(a violation ofn00) but not satisfyingcorφ(hence not a violation of n). This would only be the case ifT |= (c00∧φ00)≡(c∧φ)and T |= (c≡φ). The latter entailsT |= (c0 ≡φ0). Since we know thats1 =s2does not violaten0,c0is not equivalent toc00andφ00. Sincec0 andc00are in the language ofΦ, there is aθ0 such that val(θ0) ⊆val(c00)such thatval(θ0) 6⊆ val(c), so there exists a state which satisfiesc00but does not satisfycandφ. Other cases for a conditional prohibition (whens26|=φ0or one of the intermediate states satisfiesd0) are similar.

The case for obligations is analogous.

The stated method of finding an optimal norm approximation is computationally expensive (exponential in the size of the original set of norms). However we can show that it is still optimal since the problem of norm approximation itself is computationally hard. Since we know from Theorem 3 that components of conditional norms have to be weakened in a particular way to obtain an optimal approximation, the norm approximation problem can be re-stated as follows:

DEFINITION 11. Norm approximation problem:

Input: A theoryT(which may be empty), a set of queriesΦand a norm(c, Z(φ), d)whereZ∈ {O, P}

Output: For each formulaψ ∈ {c, φ, d}, findψ1 andψ2 such

that:

• T |=ψ1→ψ

• for allψ0in the language ofΦsuch thatT |= ψ0 →ψ, it holds thatT|= (ψ0→ψ1)

• T |=ψ→ψ2

• for allψ0in the language ofΦsuch thatT |= ψ →ψ0, it holds thatT|=ψ2→ψ0.

To establish the complexity of the norm approximation problem, we compare it to the followinginterpolation problem:

DEFINITION 12. Interpolation problem:

Input: A classical propositional tautologyφ→ψ

Output: A formula χ(an interpolant, [8]) in the common lan-guage ofφandψsuch thatφ → χandχ → ψare tau-tologies.

THEOREM4 (COMPLEXITY OFNORMAPPROXIMATION). The norm approximation problem is at least as hard as the interpolation problem.

PROOF. We give a polynomial reduction from the interpolation problem to the norm approximation problem. Take a tautology φ→ ψ. Make two norms,(>, O(φ),⊥)and(>, P(ψ),⊥), and setΦto be the set of propositional variables in the common lan-guage ofφandψ. Then we can prove that the interpolant isφ2 defined in Definition 11, hence a solution to norm approximation problem gives us a solution to the interpolation problem.

By the definition of norm approximation problem,|=φ →φ2. There is (by the Craig interpolation theorem [8]) an interpolantρ in the same language asφ2such thatφ→ρandρ→ψ. Sinceφ2 is the strongest formula for whichφ→φ2holds,φ2→ρ. Hence φ2→ψ.

6.

DISCUSSION

There is a growing body of literature on logical analysis of norms in multi-agent systems, for example [1, 9]. Most approaches to nor-mative multi-agent organisation assume perfect monitoring mech-anisms, i.e., that monitors are able to detect all violations or com-pliances of a given norm (see e.g., [10, 14]). An exception to these perfect monitor approaches is the recent work by Bulling et al. [7]. Our approach is similar to [7] in that we also define a monitor in terms of (in)distinguishable execution traces. However, in contrast to [7], we define a monitor based on indistinguishable states and then lift it to an indistinguishability relation on execution traces. Moreover, while [7] investigates the problem of combining moni-tors to construct ideal monimoni-tors, we investigate how norms can be modified such that a given imperfect monitor can function as a per-fect monitor for a norm approximation. Another important dis-tinction is our use of conditional norms with deadlines evaluated on finite execution traces rather than using arbitrary LTL formulae as norms evaluated on infinite traces, as proposed in [7]. Although the latter approach allows more expressive norms and monitors, our approach has more potential applications. Typical applications are systems that require runtime monitoring, for example, monitoring web services, policy monitoring, debugging, fault monitoring, and runtime repair and recovery of system executions (see [4, 5, 11]).

Our approach is also related to runtime verification of computer programs, where the current execution of a system is monitored and evaluated with respect to some desirable properties [6]. Run-time verification is often used to detect the violation of correct-ness properties expressed in some temporal language such as LTL. Like runtime verification our approach is based on finite execu-tion traces, but unlike these approaches we focus on approximaexecu-tion of properties that can be observed and evaluated by an imperfect monitor. Our approach is also tailored to normative multi-agent organisations, where conditional norms with deadlines are used to coordinate the (inter)action of individual agents.

7.

REFERENCES

[1] T. Ågotnes, W. van der Hoek, and M. Wooldridge. Robust normative systems and a logic of norm compliance.Logic Journal of the IGPL, 18(1):4–30, 2010.

[2] N. Alechina, M. Dastani, and B. Logan. Programming norm-aware agents. InProceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2012), pages 1057–1064. IFAAMAS, 2012. [3] N. Alechina, M. Dastani, and B. Logan. Reasoning about

normative update. InProceedings of the Twenty Third

International Joint Conference on Artificial Intelligence (IJCAI 2013), pages 20–26. AAAI Press, 2013. [4] F. Barbon, P. Traverso, M. Pistore, and M. Trainotti.

Run-time monitoring of instances and classes of web service compositions.19th International Conference on Web Services (ICWS’06), pages 63–71, 2006.

[5] L. Baresi, S. Guinea, and L. Pasquale. Towards a unified framework for the monitoring and recovery of BPEL processes. In T. Bultan and T. Xie, editors,Workshop on Testing, Analysis, and Verification of Web Services and Applications (TAV-WEB), pages 15–19. ACM, 2008. [6] A. Bauer, M. Leucker, and C. Schallhart. Runtime

verification for LTL and TLTL.ACM Transactions on Software Engineering and Methodology, 20(4):1–68, 2011. [7] N. Bulling, M. Dastani, and M. Knobbout. Monitoring norm

violations in multi-agent systems. InTwelfth International conference on Autonomous Agents and Multi-Agent Systems (AAMAS’13), pages 491–498, 2013.

[8] W. Craig. Three uses of the Herbrand-Gentzen theorem in relating model theory and proof theory.J. Symb. Log., 22(3):269–285, 1957.

[9] M. Dastani, D. Grossi, J.-J. C. Meyer, and N. Tinnemeier. Normative multi-agent programs and their logics. InProc. Workshop on Knowledge Representation for Agents and Multi-Agent Systems, LNCS 5605, pages 16–31, 2009. [10] M. Dastani, J.-J. C. Meyer, and D. Grossi. A logic for

normative multi-agent programs.Journal of Logic and Computation, 23(2):335–354, 2013.

[11] N. Delgado, A. Q. Gates, and S. Roach. A taxonomy and catalog of runtime software-fault monitoring tools.IEEE Transactions on Software Engineering, 30(12):859–872, 2004.

[12] K. Havelund and G. Rosu. Synthesizing monitors for safety properties. In J.-P. Katoen and P. Stevens, editors,Tools and Algorithms for the Construction and Analysis of Systems, 8th International Conference, TACAS 2002, volume 2280 of

LNCS, pages 342–356. Springer, 2002.

[13] J. Krajícek. Interpolation theorems, lower bounds for proof systems, and independence results for bounded arithmetic.J. Symb. Log., 62(2):457–486, 1997.

[14] S. Modgil, N. Faci, F. R. Meneguzzi, N. Oren, S. Miles, and M. Luck. A framework for monitoring agent-based

normative systems. InEighth International Joint Conference on Autonomous Agents and Multiagent Systems

(AAMAS’09), pages 153–160, 2009.

[15] D. Mundici. A lower bound for the complexity of Craig’s interpolants in sentential logic.Arch. Math. Logic, 23:27–36, 1983.