Towards Efficient Semantically Enriched Complex Event Processing and Pattern Matching

Complex Event Processing and Pattern

Matching

Syed Gillani1,2 Gauthier Picard1 Fr´ed´erique Laforest2

Antoine Zimmermann1

Institute Henri Fayol, EMSE, Saint-Etienne, France1

Overview

Introduction

Traditional Vs Real-Time Data Processing Event Processing Vs Time Axis

Complex Event Processing

Semantic Complex Event Processing

Proposed Approach

Traditional Vs Real-Time Data Processing

Database

One Shot Database Queries

E2 E4 E1 E3 E5 En Continuous Event Query Processing

Event Arrival Time

Time-Past Time-Future

Incoming Events

Time-Current

Event Processing Vs Time Axis

Before Event Arrival

At Event Arrival Some Time

After the Event

After Considerable Time e.g. 2 Hours, 1 Day, 3 Months

Time Axis Proactive Actions Pr edictive A nal ysis (Based on Historical A_nal ysis ) Real-Time Complex Event Pr ocessing and Patter n Matching Late Reaction Historical Events Post Processing and Historical Analysis

Complex Event Processing

I _{Aggregation, derivation of Primitive Events}

I _{Occurrence and non-occurrence of certain events}

I _{Imposing Temporal Constraints (application of certain}

rules )

I _{For Instance}

I _{Detection of state changes based on observations (If total}

consumed electricity > 10MWatt)

I _{Matching sequence of events that describes a scenario (If}

A<10 AND B>40 OR B<80 AND C>90)

Primitive Events Primitive Events Primitive Events Complex Events Event Source 1 Event Source 2 Event Source n

Overview

Introduction

Semantic Complex Event Processing SCEP

State-of-the-art SCEP

Foundational Challenges for SCEP

Proposed Approach

SCEP

I _{Complex Event Processing +Stream Reasoning+ Semantic}

Technologies (rules & ontologies) + Heterogeneous Data Handling?

I _{Incoming Stream Reasoning + Background Knowledge}

I _{Distributed into TWO flavours}

I _{Stream Reasoning (Real Time + Background Information +}

Aggregation through Windows) (C-SPARQL, CQELS....)

I _{Pattern Matching (Sequence, Optional, Negation)}

State-of-the-art SCEP

*Streaming the Web: Reasoning over Dynamic Data: Alessandro Margara, Jacopo Urbani, Frank van Harmelen, Henri Bal

State-of-the-art SCEP

I _{Complex Pattern Matching (Approaches)}

I _{Relational Community}

I _{NFA, EDG, RETE algorithm, Rule based system}

I _{Semantic Web Community}

I _{RETE algorithm, Logical Rule based system}

I _{How about NFA and EDG in SCEP context?}

I _{NFA and EDG are proven to be the most efficient for}

Pattern Matching in relational community

*Non-Deterministic Finite Automata *Event Detection Graphs

Foundational Challenges for SCEP

I _{Distributed Event Processing (per Query): Moving from}

centralised push based event processing

I _{Distributed Temporal Pattern Matching: Dedicated language}

for Pattern Matching (Implementation of Kleene Closure, Negation in distributed manner)

I _{Historical Management of Events: Storing and Partitioning of}

events

I _{Defining Event Boundaries: Triple based to Graph based}

streaming, preserving graph model to implement Event boundaries

I _{Predictive Event Processing: A new paradigm for SCEP}

Overview

Introduction

Semantic Complex Event Processing

Proposed Approach

Event and Stream Data Model

Query Model and Language Specification

Event and Stream Data Model

I _{Considering RDF as first class citizen (even for temporal}

reasoning, instead relying on external engines)

I _{Temporally Annotated RDF Named Graph}

(<NG, [ts, te] >)

< h t t p :// www . s t r e a m i n g i n f o . com / ElecGen > [ st1 , et1 ]

: g e n 1 : h a s N a m e ‘ PowGen - Sect1 ’.

: g e n 1 : h a s L o c a t i o n ‘ St - Etienne ’. : g e n 1 : h a s C u r r e n t P o w e r ‘60 ’.

Proposed Data Model

I _{Data Partitioning ==> Optimises query time}

I _{Summarisation ==> Merging of similar NG}

I _{Event Boundaries ==> With NG}

I _{Access Control ==> With NG}

I _{Provenance Tracking ==> With NG}

Query Model and Language Specification

I _{Former Query Models}

I _{Reliance on Triple-Based Data Model}

I _{Uses black-box approach (delegation to external Engines)}

I _{Overhead in query and data translation}

I _{Query Semantics not suitable for distributed processing per}

Proposed Query Model

Sub-Query 1 (Event Pattern A)

Sub-Query 2 (Event Pattern B)

Sub-Query 3 (Event Pattern C)

(a) (b) Stream Source Selection, Temporal Operators Pattern Duration Temporal Pattern Description Rewritten Subqueries (Stream Processing) KB Integration

System Overview

∆ =P1⇒True _{∆ =}P1&P2⇒True

∆ =P1&P2⇒True

∆ =P2&P3⇒True

Stream

1 Stream 2 Stream 3 Stream 4

Stage 1: Stream Selection Stage 2: Continuous Query Processing and Inference Stage 3: Rule or Pattern Mapping Stage 4: Distributed and Parallel Pattern Matching

(a) EDG (b) NFA

Storage of Archived Streams

Archived Streams

Streami : Incoming Streams Sk : Select Operators Ji : Join Operations Gt : Generated Events

Proposed Model

I _{Supports Triple based and NG based data model}

I _{Offers event source based Filtering}

I _{Historical management of events through summarisation}

(Facts Assignments)

I _{Provide dedicated design for SCEP (No Data or Query}

Translation unlike EP-SPARQL and other systems)

I _{Distributed and parallel sub-query processing with query}

Proposed Model

I _{Integrating stream processing and CEP}

I _{Offers various new operators including, Sequencing,}

Kleene Closure and Negation for RDF Graph patterns

I _{Allows NFA and EDG to be used in the context of SCEP}

through query rewriting (from Rule based to State based system)

Overview

Introduction

Semantic Complex Event Processing

Proposed Approach

Conclusion

I _{Annotated RDF NG enables temporal reasoning at RDF}

level

I _{Our data/query model and query rewriting allows}

I _{Annotated NG based event data model} I _{Historical management of stream data}

I _{Integration of various new operators for RDF Graphs}

(Kleene Closure, Negation )

I _{Integration of NFA and EDG in the context of SCEP} I _{Parallel and distributed event processing (per query)}