Formal Methods for Event Processing

(1)

Formal Methods for Event Processing

Alexander Artikis¹ and Opher Etzion²

1NCSR “Demokritos”, Greece

2IBM Research, Isreal

[email protected], [email protected]

(2)

An example to kick-off this talk:

Act: Passenger has been re- routed to another destination – send the luggage

Act: Bag has reached the wrong aircraft

Notify: Bag has been checked but did not reach the ULD within 20 minutes

Notify: Bag has been checked but did not reach the connecting flight

THE LUGGAGE PERSPECTIVE:

Across the 24 largest airlines more than 5.6 million bags went missing in 2006, this is an average of 15.7 bags per 1,000 travelers. 15%

of the bags are never found.

BBC News, April 4, 2007

Some of these are

reactive and some are proactive

(3)

What is an event-driven application?

I want to know about it immediately and react in the best possible way

Detect Derive Decide Do

A suspicious account is detected whenever there are at least three big cash deposits in the last 15 days followed by transfers abroad

At least three big cash deposits in the last 15 days followed by transfers a broad

A suspicious account Assign risk score Open investigation

(4)

What is an event-driven application?

Awareness

Awareness Reaction Reaction

Situation Situation

Detect Derive Decide Do

I want to know about it immediately and

react in the best possible way

(5)

So, what are characteristics of event-driven scenarios?

Events trigger action

Events influence logic for

the results There may be multiple events whose combined content influences the results

Temporal contexts (15 days) influence the results

The next section provides an introduction to event processing

A suspicious account is detected whenever there

are at least three big cash deposits in the last 15

days followed by transfers abroad

(6)

Tutorial Structure

I Introduction

I Temporal reasoning systems

I Event processing under uncertainty

I Ontology- and user-oriented modeling of event processing

I Open issues

(7)

Tutorial Resources

I Alexander Artikis, Anastasios Skarlatidis, Franois Portet, Georgios Paliouras: Logic-based event recognition. Knowledge Eng. Review 27(4): 469-506 (2012).

I Alexander Artikis, Opher Etzion, Zohar Feldman, Fabiana Fournier: Event processing under uncertainty. DEBS 2012:

32-43.

I Opher Etzion, Peter Niblett: Event Processing in Action.

Manning Publications Company 2010, isbn 978-1-935182-21-4, pp. I-XXIV, 1-360.

I Slides on slideshare.

(8)

I Introduction

I Open issues

(9)

Event processing history

First start-ups Descendents of academic projects

Apama acquisition By Progress

Around 2000

2007

TIBCO and Oracle announce

products

Streambase Coral8 2005

EPTS Established Hitting

the analysts hype…

2008

IBM

Joins 2012

New players:

SAS. Yahoo, Twitter

2013

EP at the height of BIG DATA hype

Cycle

M&A:

TIBCO/Streambase Software AG/Apama

(10)

Event processing in 2013

Internet of Things

There is now an accelerated development of new event-based systems –

many of the current trends in computing are event-driven

(11)

Event Processing is being used for various reasons

EP Solution Segments – Business Value

BUSINESS

IT

CONSUMERS

(12)

Healthcare scenario: Personalized Health Monitoring–

Diagnosis + Information Dissemination

Patient is hooked up to multiple monitors (in hospital or at

home) - the physician can set up rules based on multiple measurements and patient’s history when to send an alert and to whom:

Patient is hooked up to multiple monitors (in hospital or at

home) - the physician can set up rules based on multiple measurements and patient’s history when to send an alert and to whom:

Example: If fever is more than 103F, and blood pressure is up from yesterday – alert the nurse

Example: If blood pressure is up

constantly within 2 days total of more than 10%

alert the physician Derive

Derive Detect Detect

Decide Decide Do Do Derive Detect

Decide

Do

(13)

Retail BAM Example: Smart Shelf

Individual shelf Alerts:

1. More than 25 percent of those who touched a product item in the last 2 hours – returned it back to the shelf.

2. More than 30 percent of the product items on the shelf has been removed in the last 15 minutes

3. A product item that does not belong on the shelf has been put there

Business Level Alerts:

1. A Certain product was not been taken from any shelf in the last 2. hour A certain item has been taken

from the shelf, but did not reach the checkout within 1 hour

Derive Derive Detect Detect

Decide Decide Do Do Derive Detect

Decide

Do

(14)

Big data hype cycle 2012

(15)

What’s new?

The analog: moving from files to DBMS

In recent years – architectures, abstractions, and dedicated commercial products emerge to support functionality that was traditionally carried out within regular programming.

For some applications it is an improvement in TCO; for others is

breaking the cost-effectiveness barrier.

(16)

What is an event – three views

An event is anything that happens, or is contemplated as happening.

The happening view

The state change view

An event is a state change of some entity or actor

The detectable condition view

An event is a detectable condition that can trigger a notification

(17)

In daily life we often react to events..

(18)

Many times we react to combination of events within a context

The house sensor detects that the child did not arrive home within 2 hours from the scheduled end of classes for the day

I want to be notified when my own investment

portfolio is down 5% since the start of the trading

Day; have an agent call me when I am available,

send SMS when I am in a meeting, and Email when

I am out of office.

(19)

Event Patterns

Pattern detection is one of the notable functions of

event processing

(20)

What we actually want to react to are – situations

TOLL

VILOATOR

Sometimes the situation is determined by detecting that some pattern occurred in the flowing events.

Toll violation

Frustrated customer

Sometimes the events can approximate or indicate with some certainty that the

situation has occurred

(21)

More on Business Velocity

Increase in business velocity

happens constantly and

makes tremendous impact

on business and life

(22)

Growth in sensing sources

RFID

GPS

Instrumentation

Observation Business Process State

Changes The Web Program emitting Events

Need to

fuse all

these…

(23)

25%

18%

11%

7%

25%

0% 5% 10% 15% 20% 25%

Banking/Financial Energy/Utilities Aerospace/Defense Education Telecom Transportation/Logistics

Other

EPTS use cases survey - industries

Computer Software Healthcare

Manufacturing

Retail and distribution etc

<4% each

(24)

I Introduction

I Open issues

(25)

Running Example

City Transport Management

Control Centre IRM Demonstrator

SENSOR & GEO DATA PROCESSING

EVENT RECOGNITION

INFORMATION EXTRACTION SENSOR

NETWORK

RESOURCE DATA

& DIGITAL MAPS

Data Communication High-Level

Events Low-level Events

Training/

Debriefing Actual

Operation

Operator Operator

Driver Driver

(26)

Event Processing for City Transport Management

I Input: Low-Level Events (LLE) coming from GPS,

accelerometers, internal thermometers, microphones, internal cameras.

I Output: High-Level Events (HLE) concerning punctuality, passenger and driver safety, passenger and driver comfort, passenger satisfaction, traffic congestions, etc.

(27)

Event Processing for City Transport Management

Input Output

200 scheduled stop enter 215 scheduled stop leave [215, 400] abrupt acceleration [500, 600] very sharp turn 700 late stop enter 705 passenger density

change to high 715 scheduled stop leave 820 scheduled stop enter 815 passenger density

change to low . . .

(28)

Event Processing for City Transport Management

Input Output

200 scheduled stop enter

215 scheduled stop leave 215 punctual

[215, 400] abrupt acceleration [215, 400] uncomfortable driving [500, 600] very sharp turn [500, 600] unsafe driving

700 late stop enter 705 passenger density

change to high 715 scheduled stop leave 820 scheduled stop enter 815 passenger density

change to low . . .

(29)

Event Processing for City Transport Management

Input Output

200 scheduled stop enter

215 scheduled stop leave 215 punctual

[215, 400] abrupt acceleration [215, 400] uncomfortable driving [500, 600] very sharp turn [500, 600] unsafe driving

700 late stop enter 700 non-punctual

705 passenger density since(705) reducing passenger

change to high comfort

715 scheduled stop leave 820 scheduled stop enter

815 passenger density [705, 815] reducing passenger

change to low comfort

. . .

(30)

Event Definitions

punctual (Id, VehicleType)

stop enter (Id, VehicleType,

StopCode, late)

V

V (Id, VehicleType, ^{stop leave} StopCode, early) non-punctual

(Id, VehicleType)

stop leave (Id, VehicleType, StopCode, scheduled)

stop enter (Id, VehicleType, StopCode, early)

stop enter (Id, VehicleType, StopCode, scheduled)

V

stop leave (Id, VehicleType,

StopCode, late)

(31)

Event Definitions

uncomfortable driving

(Id, VehicleType) V

sharp turn (Id, VehicleType, sharp)

V

sharp turn (Id, VehicleType,

very_sharp)

unsafe driving (Id, VehicleType)

abrupt acceleration (Id, VehicleType, abrupt)

abrupt deceleration (Id, VehicleType, abrupt)

abrupt acceleration (Id, VehicleType,

very_abrupt)

abrupt deceleration (Id, VehicleType,

very_abrupt)

(32)

Event Definitions

compromising passenger safety

violence (Id, VehicleType)

emergency stop (Id, VehicleType) unsafe driving (Id, VehicleType)

vehicle accident (Id, VehicleType)

(33)

High Level Event as Chronicle

A HLE can be defined as a set of events interlinked by time constraints and whose occurrence may depend on the context.

I This is the definition of a chronicle.

Chronicle recognition systems have been used in many applications:

I Cardiac monitoring system.

I Intrusion detection in computer networks.

I Distributed diagnosis of web services.

(34)

Chronicle Representation Language

Predicate Meaning

event(E, T) Event E takes place at time T event(F:(?V1,?V2),T) An event takes place at

time T changing the value of property F from ?V1 to ?V2 noevent(E, (T1,T2)) Event E does not take place

between [T1,T2)

noevent(F:(?V1,?V2), No event takes place between (T1,T2)) [T1,T2) that changes the value

of property F from ?V1 to ?V2 hold(F:?V, (T1,T2)) The value of property F is ?V

between [T1,T2)

occurs(N,M,E,(T1,T2)) Event E takes place at least N times and at most M times between [T1,T2)

(35)

Chronicle Representation Language

chronicle punctual[?id, ?vehicle](T1) {

event( stop enter[?id, ?vehicle, ?stopCode, scheduled], T0 ) event( stop leave[?id, ?vehicle, ?stopCode, scheduled], T1 ) T1 > T0

end - start in [1, 2000]

}

chronicle non punctual[?id, ?vehicle]() {

event( stop enter[?Id, ?vehicle, *, late], T0 ) }

chronicle punctuality change[?id, ?vehicle, non punctual](T1) { event( punctual[?id, ?vehicle], T0 )

event( non punctual[?id, ?vehicle], T1 ) T1 > T0

noevent( punctual[?id, ?vehicle], ( T0+1, T1 ) ) noevent( non punctual[?id, ?vehicle], ( T0+1, T1 ) ) end - start in [1, 20000]

}

(36)

Chronicle Representation Language

compromising passenger safety

violence (Id, VehicleType)

emergency stop (Id, VehicleType) unsafe driving (Id, VehicleType)

vehicle accident (Id, VehicleType)

I Mathematical operators in the atemporal constraints of the language are not allowed:

I cannot express that passenger safety is compromised more when a vehicle accident takes placefar from a hospital or a police station.

(37)

Chronicle Representation Language

I Universal quantification is not allowed:

I cannot express that a route is punctual ifall vehicles of the route are punctual.

CRS is a purely temporal reasoning system.

It is also a very efficient and scalable system.

(38)

Chronicle Recognition System: Semantics

Each HLE definition is represented as a Temporal Constraint Network. Eg:

stop enter [?id,?vehicle,

?stopcode, scheduled]

stop leave [?id,?vehicle,

?stopcode, scheduled]

[1,2000]

(39)

Chronicle Recognition System: Consistency Checking

Compilation stage:

I Constraint propagation in the Temporal Constraint Network.

I Consistency checking.

[2, 5]

A C

A

B [1, 6]

[0, 10]

[2, 5]

A C

A

B [1, 6]

[2, 10]

[3, 10]

A C A

B

[0, 8]

(40)

Chronicle Recognition System: Forecasting

Recognition stage:

I Partial HLE instance evolution.

I Forward (predictive) recognition.

[2, 5]

A A C

B [1, 6]

[2, 10]

[3, 10]

[0, 8]

6 8 9 11 16 6 10 11 16

A C B A

A A B

C C@10

→

(41)

Chronicle Recognition System: Partial instances

HLE definition: Reduce tram endurance

A ^[1,3] B ^[0,3] C

A: enter tram intersection B: abrupt deceleration C: abrupt acceleration

time

time A@1

A@1 B[2,4]

A@3

A@1 B[4,4]

A@3 B[4,6]

B@5

killed instance

A@1 B@5 C[5,8]

A@1 B[5,6]

duplicated

(42)

Chronicle Recognition System: Partial instances

A ^[1,3] B ^[0,3] C

time A@1

A@1 B[2,4]

A@3

A@1 B[4,4]

A@3 B[4,6]

B@5

killed instance

A@1 B@5 C[5,8]

A@1 B[5,6]

duplicated

(43)

Chronicle Recognition System: Partial instances

A ^[1,3] B ^[0,3] C

time A@1

A@1 B[2,4]

time A@1

A@1 B[2,4]

A@3

A@1 B[4,4]

A@3 B[4,6]

B@5

killed instance

A@1 B@5 C[5,8]

A@1 B[5,6]

duplicated

(44)

Chronicle Recognition System: Partial instances

A ^[1,3] B ^[0,3] C

time A@1

A@1 B[2,4]

A@3

time A@1

A@1 B[2,4]

A@3

A@1 B[4,4]

A@3 B[4,6]

B@5

killed instance

A@1 B@5 C[5,8]

A@1 B[5,6]

duplicated

(45)

Chronicle Recognition System: Partial instances

A ^[1,3] B ^[0,3] C

time A@1

A@1 B[2,4]

A@3

A@1 B[4,4]

A@3 B[4,6]

time A@1

A@1 B[2,4]

A@3

A@1 B[4,4]

A@3 B[4,6]

B@5

killed instance

A@1 B@5 C[5,8]

A@1 B[5,6]

duplicated

(46)

Chronicle Recognition System: Partial instances

A ^[1,3] B ^[0,3] C

time A@1

A@1 B[2,4]

A@3

A@1 B[4,4]

A@3 B[4,6]

B@5

time A@1

A@1 B[2,4]

A@3

A@1 B[4,4]

A@3 B[4,6]

B@5

killed instance

A@1 B@5 C[5,8]

A@1 B[5,6]

duplicated

(47)

Chronicle Recognition System: Partial instances

A ^[1,3] B ^[0,3] C

time A@1

A@1 B[2,4]

A@3

A@1 B[4,4]

A@3 B[4,6]

B@5

time A@1

A@1 B[2,4]

A@3

A@1 B[4,4]

A@3 B[4,6]

B@5

killed instance

A@1 B@5 C[5,8]

A@1 B[5,6]

duplicated

(48)

Chronicle Recognition System: Partial instances

A ^[1,3] B ^[0,3] C

time A@1

A@1 B[2,4]

A@3

A@1 B[4,4]

A@3 B[4,6]

B@5

killed instance

A@1 B@5 C[5,8]

time A@1

A@1 B[2,4]

A@3

A@1 B[4,4]

A@3 B[4,6]

B@5

killed instance

A@1 B@5 C[5,8]

A@1 B[5,6]

duplicated

(49)

Chronicle Recognition System: Partial instances

A ^[1,3] B ^[0,3] C

time A@1

A@1 B[2,4]

A@3

A@1 B[4,4]

A@3 B[4,6]

B@5

killed instance

A@1 B@5 C[5,8]

A@1 B[5,6]

duplicated

(50)

Chronicle Recognition System

Recognition stage — partial HLE instance management:

I In order to manage all the partial HLE instances, CRS stores them in trees, one for each HLE definition.

I Each event occurrence and each clock tick traverses these trees in order to kill some HLE instances (tree nodes) or to develop some HLE instances.

I The performance of CRS depends directly on the number of partial HLE instances

I each tick or event O(Kn²) with K number of instances, n size of models.

(51)

Chronicle Recognition System: Optimisation

Several techniques have been recently developed for improving efficiency. Eg, ‘temporal focusing’:

I Distinguish between very rare events and frequent events based on a priori knowledge of the monitored application.

I Focus on the rare events: If, according to a HLE definition, a rare event should take place after the frequent event, store the incoming frequent events, and start recognition only upon the arrival of the rare event.

I In this way the number of partial HLE instances is significantly reduced.

I Example: Reduce tram endurance

A ^[1,3] B ^[0,3] C

(52)

Chronicle Recognition System: Summary

I One of the earliest and most successful formal event processing systems.

I Very efficient and scalable event processing.

I But:

I It is a purely temporal reasoning system.

I It does not handle uncertainty.

(53)

Event Calculus

I A logic programming language for representing and reasoning about events and their effects.

I Key components:

I event (typically instantaneous).

I fluent: a property that may have different values at different points in time.

I Built-in representation of inertia:

I F = V holds at a particular time-point if F = V has been initiated by an event at some earlier time-point, and not terminated by another event in the meantime.

(54)

HLE Definitions in the Event Calculus

HLE definition:

punctuality (ID) = non punctual initiated iff enter stop(ID, StopCode, late) happens or leave stop(ID, StopCode, early ) happens

punctuality (ID) = non punctual terminatedAt T iff enter stop(ID, StopCode, scheduled ) happensAt T⁰, leave stop(ID, StopCode, scheduled ) happensAt T

HLE recognition:

I punctuality (ID) = non punctual holdsFor I

(55)

HLE Definitions in the Event Calculus

HLE definition:

driving quality (ID) = low iff

punctuality (ID) = non punctual or driving style(ID) = unsafe

Compiled HLE definition:

driving quality (ID) = low holdsFor I₁ ∪ I₂ iff punctuality (ID) = non punctual holdsFor I1, driving style(ID) = unsafe holdsFor I₂

(56)

HLE Definitions in the Event Calculus

HLE definition:

driving quality (ID) = medium iff punctuality (ID) = punctual , driving style(ID) = uncomfortable

driving quality (ID) = medium holdsFor I₁ ∩ I₂ iff punctuality (ID) = punctual holdsFor I1, driving style(ID) = uncomfortable holdsFor I₂

(57)

HLE Definitions in the Event Calculus

HLE definition:

driving quality (ID) = high iff punctuality (ID) = punctual , driving style(ID) 6= unsafe, driving style(ID) 6= uncomfortable

driving quality (ID) = high holdsFor I1 \ I₂ ∪ I₃ iff punctuality (ID) = punctual holdsFor I1, driving style(ID) = unsafe holdsFor I₂, driving style(ID) = uncomfortable holdsFor I3

(58)

Run-Time Event Processing

Real-time decision-support in the presence of:

I Very large LLE streams.

I Non-sorted LLE streams.

I LLE revision.

I Very large HLE numbers.

(59)

Event Calculus: Run-Time Event Processing

time

Q178 Q179 Q180 Q181 Q182

Q177

Working Memory

(60)

Event Calculus: Run-Time Event Processing

time

Q178 Q179 Q180 Q181 Q182

Q177

Working Memory

time

Q178 Q179 Q180 Q181 Q182

Q177

Working Memory

(61)

Event Calculus: Run-Time Event Processing

time

Q178 Q179 Q180 Q181 Q182

Q177

Working Memory

time

Q178 Q179 Q180 Q181 Q182

Q177

Working Memory

time

Q178 Q179 Q180 Q181 Q182

Q177

Working Memory

(62)

Event Calculus: Summary

I Complex temporal representation.

I Succinct representation → code maintenance.

I Intuitive representation → facilitates interaction with domain experts unfamiliar with programming.

I Complex atemporal representation.

I The full power of logic programmign is available.

I Very efficient reasoning.

I Even in the absence of input data filtering.

I Even when input data arrive with a delay and are revised.

I But:

I The Event Calculus has to deal with uncertainty.

(63)

I Introduction

I Open issues

(64)

Common Problems of Event Processing

I Limited dictionary of LLE and context variables

I Incomplete LLE stream

I Erroneous LLE detection

I Inconsistent HLE annotation

I Inconsistent LLE annotation

Therefore, an adequate treatment of uncertainty is required.

(65)

Logic-based models & Probabilistic models

I Logic-based models:

I Very expressive with formal declarative semantics

I Directly exploit background knowledge

I Trouble with uncertainty

I Probabilistic graphical models:

I Handle uncertainty

I Lack of a formal representation language

I Difficult to model complex events

I Difficult to integrate background knowledge

(66)

Can these approaches combined?

Research communities that try combine these approaches:

I Probabilistic (Inductive) Logic Programming

I Statistical Relational Learning How?

I Logic-based approaches incorporate statistical methods

I Probabilistic approaches learn logic-based models

(67)

ProbLog

I A Probabilistic Logic Programming language.

I Allows for independent ‘probabilistic facts’ prob::fact.

I Prob indicates the probability that fact is part of a possible world.

I Rules are written as in classic Prolog.

I The probability of a query q imposed on a ProbLog database (success probability) is computed by the following formula:

P_s(q) = P( _

e∈Proofs(q)

^

f_i∈e

f_i )

(68)

Event Processing using ProbLog

Input Output

200 0.45::scheduled stop enter

215 0.80::scheduled stop leave 215 0.42::punctual 700 0.89::late stop enter

705 0.82::passenger density change to high

715 0.67::scheduled stop leave 820 0.53::scheduled stop enter 815 0.33::passenger density

change to low . . .

(69)

Event Calculus in ProbLog

HLE Probability

1.0

0.5

0.2

Time

Init Init

10

… Init …

30 Term

0 40

Term … Term … 20

(70)

Markov Logic Networks (MLN)

Grounding

Markov Logic Networks Markov Network

(probabilistic inference)

Input Output

LLE Recognised

HLE HLE

Knowledge Base

P(HLE =True | LLE) (evidence)

I Syntax: weighted first-order logic formulas (F_i, w_i).

I Semantics: (Fi, wi) represents a probability distribution over possible worlds.

I A world violating formulas becomes less probable, but not impossible.

(71)

Markov Logic: Representation

Example definition of HLE ‘uncomfortable driving’ :

w₁

abrupt movement(Id , Vehicle, T ) ← abrupt acceleration(Id , Vehicle, T ) ∨ abrupt deceleration(Id , Vehicle, T ) ∨ sharp turn(Id , Vehicle, T )

w2

uncomfortable driving (Id , Vehicle, T2) ← approach intersection(Id , Vehicle, T1) ∧ abrupt movement(Id , Vehicle, T₂) ∧ before(T1, T2)

(72)

Markov Logic: Representation

I Weight: is a real-valued number

I Higher weight −→ Stronger constraint

I Hard constraints

I Infinite weight values

I Background knowledge

I Axioms

I Soft constraints

I Strong weight values: almost always true

I Weak weight values: describe exceptions

(73)

Markov Logic: Network Construction

I Formulas are translated into clausal form

I Weights are divided equally among clauses

1

3w1 ¬abrupt acceleration(Id, Vehicle, T ) ∨ abrupt movement(Id, Vehicle, T )

1

3w1 ¬abrupt deceleration(Id, Vehicle, T ) ∨ abrupt movement(Id, Vehicle, T )

1

3w1 ¬sharp turn(Id, Vehicle, T ) ∨ abrupt movement(Id, Vehicle, T )

w2 ¬approach intersection(Id, Vehicle, T₁) ∨ ¬abrupt movement(Id , Vehicle, T2) ∨

¬before(T₁, T2) ∨ uncomfortable driving (Id , Vehicle, T2)

(74)

Markov Logic: Network Construction

Template that produces ground Markov network:

I Given a set of constants: detected LLE

I Ground all clauses

I Boolean nodes: ground predicates

I Each ground clause:

I Forms a clique in the network

I Associated with w_i and a Boolean feature P(X = x ) =_Z¹exp (P

iw_in_i(x )) Z =P

x ∈Xexp(P(X = x ))

(75)

Markov Logic: Network Construction

1

3w1 ¬abrupt acceleration(Id, Vehicle, T ) ∨ abrupt movement(Id, Vehicle, T )

1

3w1 ¬abrupt deceleration(Id, Vehicle, T ) ∨ abrupt movement(Id, Vehicle, T )

1

3w1 ¬sharp turn(Id, Vehicle, T ) ∨ abrupt movement(Id, Vehicle, T )

LLE:

abrupt acceleration(tr0, tram, 101) approach intersection(tr0, tram, 100) before(100, 101)

Constants:

T = {100, 101}

Id = {tr0} Vehicle = {tram}

(76)

Markov Logic: Network Construction

For example, the clause:

produces the following groundings:

w2 ¬approach intersection(tr₀, tram, 100 ) ∨ ¬abrupt movement(tr0, tram, 100 ) ∨

¬before(100 , 100 ) ∨ uncomfortable driving (tr₀, tram, 100 )

w2 ¬approach intersection(tr0, tram, 100 ) ∨ ¬abrupt movement(tr0, tram, 101 ) ∨

¬before(100 , 101 ) ∨ uncomfortable driving (tr0, tram, 101 )

w2 ¬approach intersection(tr₀, tram, 101 ) ∨ ¬abrupt movement(tr0, tram, 100 ) ∨

w2 ¬approach intersection(tr0, tram, 101 ) ∨ ¬abrupt movement(tr0, tram, 101 ) ∨

(77)

Markov Logic: Network Construction

before (100,100)

before (101,101) uncomfortable

_driving (tr0, tram,101)

abrupt_

movement (tr0,tram,101)

approach_

intersection (tr0,tram,100)

before (100,101)

abrupt_

deceleration (tr0,tram,101) sharp_turn

(tr0,tram,101)

abrupt_

acceleration (tr0,tram,101)

uncomfortable _driving (tr0, tram,100)

abrupt_

approach_

before (101,100) abrupt_

deceleration (tr0,tram,100)

sharp_turn (tr0,tram,100) abrupt_

(78)

Markov Logic: World state discrimination

before (100,100)

abrupt_

approach_

before (100,101)

abrupt_

(tr0,tram,101)

abrupt_

approach_

P(X = x1) =_Z¹exp(¹₃w1· 2 +¹₃w1· 2 +¹₃w1· 2 + w2· 4) = _Z¹e^2w¹^+4w²

(79)

Markov Logic: World state discrimination

before (100,100)

abrupt_

approach_

before (100,101)

abrupt_

(tr0,tram,101)

abrupt_

approach_

(80)

Markov Logic: Inference

I Event recognition involves querying about HLE

I Having a ground Markov network

I Apply standard probabilistic inference methods

I Large network with complex structure

I Infeasible inference

I MLN combine logical and probabilistic inference methods

(81)

Markov Logic: Conditional inference

Query: The trams that are driven in an uncomfortable manner given a LLE stream

I Query variables Q: HLE

I Evidence variables E : LLE

I Hidden variables H

P(Q | E = e) =P(Q, E = e, H) P(E = e, H)

abrupt_

approach_

before (100,101)

abrupt_

(tr0,tram,101)

abrupt_

(82)

Markov Logic: Conditional inference

P(Q | E = e) =P(Q, E = e, H) P(E = e, H)

abrupt_

approach_

before (100,101)

abrupt_

(tr0,tram,101)

abrupt_

(83)

Markov Logic: Conditional inference

P(Q | E = e) =P(Q, E = e, H) P(E = e, H)

abrupt_

approach_

before (100,101)

abrupt_

(tr0,tram,101)

abrupt_

(84)

Markov Logic: Conditional inference

I Efficiently approximated with sampling

I Markov Chain Monte Carlo (MCMC): e.g Gibbs sampling

I Random walks in state space

I Reject all states where E = e does not hold

(85)

Markov Logic: MCMC

uncomfortable _driving (tr0, tram,101) abrupt_

approach_

before (100,101)

abrupt_

(tr0,tram,101) abrupt_

approach_

before (100,101)

abrupt_

approach_

before (100,101)

abrupt_

approach_

before (100,101)

abrupt_

... ...

(86)

Markov Logic: Deterministic dependencies

I MCMC is a pure statistical method

I MLN combine logic and probabilistic models

I Hard constrained formulas:

I deterministic dependences

I isolated regions in state space

I Strong constrained formulas:

I near-deterministic dependencies

I difficult to cross regions

I Combination of satisfiability testing with MCMC

P(X=x)

xi xj x

P(X=x)

xi xj x

(87)

Event Calculus

0 3 10 20 Time

0.0 0.5 1.0

Initiated Initiated Terminated

(88)

Event Calculus in MLN

0 3 10 20 Time

0.0 0.5 1.0

Hard-constrained inertia rules:

2.3 HLE initiatedAt T if [Conditions]

¬(HLE holdsAt T ) iff

¬(HLE holdsAt T −1),

¬(HLE initiatedAt T −1)

2.5 HLE terminatedAt T if [Conditions]

HLE holdsAt T iff HLE holdsAt T −1,

¬(HLE terminatedAt T −1)

(89)

Event Calculus in MLN

0 3 10 20 Time

0.0 0.5 1.0

Soft-constrained initiation inertia rules:

2.8 ¬(HLE holdsAt T ) iff

HLE holdsAt T iff HLE holdsAt T −1,

(90)

Event Calculus in MLN

0 3 10 20 Time

0.0 0.5 1.0

Soft-constrained termination inertia rules:

2.8 HLE holdsAt T iff HLE holdsAt T −1,

(91)

Event Calculus in MLN

0 3 10 20 Time

0.0 0.5 1.0

Soft-constrained termination inertia rules:

0.8 HLE holdsAt T iff HLE holdsAt T −1,

(92)

Event Processing Under Uncertainty: Summary

There are ways to deal with:

I Uncertainty in the input.

I Uncertainty in the HLE definitions.

But:

I We are still missing a framework for real-time event processing under uncertainty.

(93)

I Introduction

I Open issues

(94)

Event processing: one of the major gaps is that all products in this area are geared towards IT developers

Comprehensive user survey show that 84%

of the users wish that Event rules will be defined By business users

There is a gap

Current models:

Implementation oriented

Business analysts oriented

Modeling

(95)

Driver Ranking

The delivery Controller assigns delivery to a

driver

The driver drives to pick up the package

The driver delivers and the customer confirms

the delivery

New driver initiation

A simple example: Generic Delivery Process

Based on events:

Compute Driver Ranking

A delivery company uses independent drivers For its deliveries. Drivers are ranked based on their record in meeting delivery deadlines.

The example is a dynamic event-driven updates

of the drivers’ ranking.

(96)

How is driver ranking computed based on events?

(a) New driver (initiation) gets ranking of 10 points (b) For every 20 assignments, if no deadline is missed, driver gets a ranking increase of 1 point

(c) For every 20 assignments, if more than 5 deadlines

are missed, driver gets a ranking decrease of 1 point.

(97)

Current practice is to embed Driver Ranking in the Delivery Process Model (unnecessarily complex, tied to proprietary software, not understandable by knowledge workers)

Can we do it simpler?

Driver Ranking

The delivery Controller assigns delivery to a driver

The driver drives to pick up the

package

The driver delivers and the customer confirms The delivery

New driver

initiation

(98)

The vision: lift the center of gravity up in event based systems from the programmer/code level to the knowledge worker/model level

Center of gravity of the life-cycle is in the level of the execution code:

Development, modification

Center of gravity of the life-cycle is in the level of

the computing independent

model: Development, modification

TODAY TOMORROW

CODE LEVEL

Controlled by IT people

MODEL LEVEL

Controlled by the

business people

(99)