Semantic and Data Mining Technologies. Simon See, Ph.D.,

Semantic and Data Mining Technologies

Introduction

to Semantic Web and

Business Use Cases

NAR 2009 – over 1170 databases

Lots of

Scientific

Resources



Paul writes workflows for identifying biological

pathways implicated in resistance to Trypanosomiasis

in cattle



Paul meets Jo. Jo is investigating Whipworm in

mouse.



Jo reuses one of Paul’s workflow

without change

.



Jo identifies the biological pathways involved in sex

dependence in the mouse model, believed to be

involved in the ability of mice to expel the parasite.



Previously a manual

two year study by Jo

had failed

to do this.

•

Access to distributed

and local resources

•

Automation of data flow

•

Iteration over data sets

•

Interactive

•

Agile software

development

•

Experimental protocols

•

Declarative mashups?

Workflows

1) The Playground Panel with A) Data Objects that feed into BioMoby objects, B) BioMoby Services which can be executed through a right-click option and C) the ‘record’ button for recording user

activity. 2) The Taverna tree of

Web Services which can be dragged into the

Playground Panel and connected to Data Objects.

3) Semantic Discovery panel that shows

compatible services based on input or output.

5) The Data Viewer panel for viewing data that features the Taverna data rendering

capabilities.

Andrew Gibson

The Data

Playground

4) The Data Editor panel for inputting Data.

Semantic Technology Stack

http://www.w3.org/2007/03/ layerCake.svg

7

•

Basic Technologies

•

URI

• Uniform Resource Identifier

•

RDF

• Resource Description Framework

•

RDFS

• RDF Schema

•

OWL

• Web ontology language

•

SPARQL

• Protocol and Query Language

Transform & Edit Tools •RDF/OWL Data Management •SQL & SPARQL Query •Inferencing •Semantic Rules •Security •Semantic Indexing •Versioning • BI Analytics • Graph Visualization • Social Network Analysis • Metadata Registry • Faceted Search • SPARQL Endpoint Load, Query & Inference Applications & Analysis Tools

Extraction, Modeling, Reasoning & Discovery

Workflow

Other Data Formats RSS, email Transaction Systems Data Sources Unstructured

Content Entity Extraction & Transform Ontology Engineering Categorization Custom Scripting Partner Tools Partner/Oracle Tools

Why Organizations use Oracle RDF Database

• Oracle database is the leading

commercial database with native RDF/ OWL data management

• Scalable & secure platform for wide-range of semantic applications

• Readily scales to ultra-large repositories (10s billions of triples) • _{Choice of SQL or SPARQL query}

• Leverages Oracle Partitioning. RAC supported

• Growing ecosystem of 3rd _{party tools}

partners

• Native RDF graph data store • Manages billions of triples • Fast batch, bulk and incremental load

• SQL: SEM_Match

• SPARQL: via Jena plug-in • Ontology assisted query of RDBMS data

• Forward chaining model • OWLprime, OWL 2 RL, RDFS • User defined rule base

Key Capabilities: Load / Storage Query Reasoning 9

3

rd

-P

art

y C

all

ou

ts

R ea so ne rs : P ell et N LP In fo . E xtr ac to r: C ala is , G A T E

Java API support

SPARQL: Jena / Sesame

JDBC Java Programs

SQL

Interface

S Q Lp lu s P L/ S Q L S Q Ld ev .

P

ro

gr

am

m

in

g

In

te

rfa

ce

SPARQL Endpoints

Joseki / Sesame

Tools

V is ua liz er (c yto sc op e)

Architectural Overview

Enterprise (Relational) data RDF/OWL data and ontologies Rulebases: OWL, RDF/S, user-defined Inferred RDF/OWL data

R

D

F

/O

W

L

O

ra

cle

D

B

Security: fine-grained Versioning: Workspaces S_em

an tic In de xe s Ontology-assisted Query of Enterprise Data Query RDF/ OWL data and ontologies INFER L O A D R D F /S U se r-de f. O W L su bs et s B ul k-Lo ad In cr . D M L

C

or

e

fu

n

cti

on

ali

ty

QUERY (SQL-based SPARQL)

Topbraid Composer

Core Entities in Oracle Database Semantic Store

OWL subset RDF / RDFS Rule base m

…

Rulebases & Vocabularies

X1 X2 Xp Entailments A₁ A₂ A_n

…

R R R h er m an sc o tt sc o tt Application Tables

Semantic Network (MDSYS)

• _{Sem. Network  Dictionary and}

data tables. New entities: models,

rule bases, and rules indexes

(entailments). OWL and RDFS rule

bases are preloaded.

• Model  A model holds an RDF graph (set of S-P-O triples) and is associated with an sdo_rdf_triple_s

column in an application table.

• Application Table  Contains col

of object type sdo_rdf_triple_s to allow loading and accessing RDF triples, and storing ancillary values.

• SDO_RDF_TRIPLE_S  A new

object type for RDF.

• Rulebase  A rulebase is a set of rules used for inferencing.

• Entailments  An entailment stores triples derived via inferencing.

M1 M2 Mn

…

Models Values Triples Oracle Database

Load / Query / Inference

OWL subset RDF / RDFS Rule base m

…

Rulebases & Vocabularies

X1 X2 Xp Entailments A₁ A₂ A_n

…

R R R h er m an sc o tt sc o tt Application Tables

Semantic Network (MDSYS)

•

Load



– Bulk load

– Incremental load

• Query 

• SPARQL (direct or SQL-based)

• simple data access

• Inference 

• using OWL 2 RL, RDFS, etc.

• using User-defined rules

M1 M2 Mn

…

Models Values Triples Oracle Database

Data Integration Platform in Health Informatics

Run-Time Metadata

Design-Time Metadata

Enterprise Information Consumers (EICs)

Business Intelligence _AnalyticsClinical Patient

Care ManagementWorkforce

Model

Physical

Model

Virtual

Relate

Deploy

Access

Access

Integration Server

(Semantic Knowledge base) HTB CIS LIS HIS 13

Oracle Database 11g Ontology repository

Oracle Text 11g For text based search

Oracle RDF 11g For graph based search

Semantic Portal

BACK FRONT

ITM Features

Web User Interfaces Multilingual

Connectors to text mining Collaborative maintenance Import / export

Scalability

API & Web Services Java – J2EE – LDAP

Ontology model (OWL) Terminology – Taxonomy (RDF SKOS) Knowledge representation Catalog - Yellow pages (RDF)

Intelligent Topic Manager (ITM) Manage Edit Maintain Search Control Import Export Audit

Vocabulary Management

Entity Extraction Engine Feature/ term/relation Extraction, categorization (Insight, Lymba,

Calais, Gate)

National Intelligence: Text Mining

Blogs, open source, newsfeed Signal Intelligence, message traffic Analyst Reports (Content Mgmt) Analyst Browsing, Presentation, Reporting, Visualization, Query Tools (e.g. i2, Centrifuge, Visual Analytics)

Explore 1. Unstructured Data (Text) 2. Model Knowledge Base (RDF Store) 3. Structured Data XML/OWL/N3

“Oracle’sfounderLarry Ellison wins 2010 America’s Cup Race …”

organization

SPARQL/SQL

founder person

Triple Structure: Subj – Pred - Obj

Larry Ellison fo un de r Oracle Ontologies + Rules 10’s of billions of triples

Copyright 2010 Oracle Corporation

Oracle Data Mining Algorithms

Classification Association Rules Clustering Attribute Importance

Problem Algorithm Applicability

Classical statistical technique Popular / Rules / transparency Embedded app

Wide / narrow data / text

Minimum Description Length (MDL)

Attribute reduction Identify useful data Reduce data noise

Hierarchical K-Means Hierarchical O-Cluster

Product grouping Text mining

Gene and protein analysis Apriori Market basket analysisLink analysis Multiple Regression (GLM)

Support Vector Machine Classical statistical technique_{Wide / narrow data / text}

Regression

Feature Extraction

NMF Text analysis_{Feature reduction} Logistic Regression (GLM)

Decision Trees Naïve Bayes

Support Vector Machine

One Class SVM Lack examples

Anomaly Detection

A1 A2 A3 A4 A5 A6 A7

Copyright 2010 Oracle Corporation

Traditional Analytics

Hours, Days or Weeks

In-Database Data Mining

Data Extraction Data Prep & Transformation Data Mining Model Building Data Mining Model “Scoring” Data Preparation and Transformation Data Import Source
 Data SAS
Work


Area
 SAS
 Process ing Process
 Output
 Target
 Results

• Faster time for

“Data” to “Insights”

• Lower TCO—Eliminates • Data Movement

• Data Duplication • Maintains Security

Data remains in the Database SQL—Most powerful language for data preparation and

Embedded data preparation Cutting edge machine

learning algorithms inside the SQL kernel of Database

Model “Scoring”

Data remains in the Database

Savings

Secs, Mins or Hours

Model “Scoring” Embedded Data Prep

Data Preparation Model Building

Oracle Data Mining

S A S S A S S A S

Copyright 2010 Oracle Corporation

11g Statistics & SQL Analytics

• Ranking functions

• rank, dense_rank, cume_dist, percent_rank, ntile

• _{Window Aggregate functions (moving}

and cumulative)

• Avg, sum, min, max, count, variance, stddev, first_value, last_value

• LAG/LEAD functions

• Direct inter-row reference using offsets

• Reporting Aggregate functions

• Sum, avg, min, max, variance, stddev, count, ratio_to_report

• Statistical Aggregates

• Correlation, linear regression family, covariance

• Linear regression

• Fitting of an ordinary-least-squares regression line to a set of number pairs. • Frequently combined with the

COVAR_POP, COVAR_SAMP, and CORR functions

Descriptive Statistics

• DBMS_STAT_FUNCS: summarizes numerical columns of a table and returns count, min, max, range, mean, median, stats_mode, variance, standard deviation, quantile values, +/- n sigma values, top/bottom 5 values

• Correlations

• Pearson’s correlation coefficients, Spearman's and Kendall's (both nonparametric).

• Cross Tabs

• Enhanced with % statistics: chi squared, phi coefficient, Cramer's V, contingency coefficient, Cohen's kappa

• Hypothesis Testing

• Student t-test , F-test, Binomial test, Wilcoxon Signed Ranks test, Chi-square, Mann Whitney test, Kolmogorov-Smirnov test, One-way ANOVA

• Distribution Fitting

• Kolmogorov-Smirnov Test, Anderson-Darling Test, Chi-Squared Test, Normal, Uniform, Weibull, Exponential

Note: Statistics and SQL Analytics are included in Oracle Database Standard Edition

Copyright 2010 Oracle Corporation

Integration with Oracle BI EE

ODM provides likelihood _{of expense reporting fraud}

….and other important questions.

Oracle’s Partners for Semantic Technologies

Integrated Tools and Solution Providers:

Ontology Engineering

Query Tool Interfaces Standards

Reasoners Applications SI / Consulting Sesame NLP Entity Extractors

Joseki

21

Some Oracle Database Semantics Customers

Hutchinson 3G Austria

Life Sciences

Clinical Medicine & Research

Publishing

Thomson Reuters

Education

Telecomm &

Networking

Defense/

Intelligence

Summary: Oracle Database 11g Release 2

Oracle database is the leading commercial database with native support for

W3C standards compliant RDF/OWL data store w/ comprehensive

capabilities for ….

Reasoning and Discovery

supporting std. ontologies persistent, native &

3

rd

party inference, and user-defined rules

Scalability

to evolve schemas dynamically and grow to 10’s billions of

triples, incremental & parallel inference

Data Integration

to link structured & unstructured content, Loosely couple

business silos

Security

to protect data on a “need to know” basis

Integrated querying & manageability

SPARQL & SQL

for

RDF/OWL,

relational, XML, text, location, & multimedia data

Copyright 2010 Oracle Corporation

An Analytical Database Changes—

Everything!

Less

data movement =

faster

analytics, …and

faster analytics =

better

BI throughout enterprise

?x

Data Mining

Statistical Functions _{Text Mining} OLAP Predictive Analytics

For More Information

search.oracle.com

or

oracle.com

Copyright 2010 Oracle Corporation

Additional Information

• Preview of the new Oracle Data Miner 11g R2 “work flow” New GUI

• Oracle Data Mining 11gR2 presentation at Oracle Open World 2009

• Oracle Data Mining Blog

• Funny YouTube video that features Oracle Data Mining

• Oracle Data Mining on the Amazon Cloud

• Oracle Data Mining 11gR2 data sheet

• Oracle Data Mining 11gR2 white paper

• New TechCast (audio and video recording): ODM overview and several demos

• Fraud and Anomaly Detection using Oracle Data Mining 11g presentation

• Algorithm technical summary with links to Documentation

• Getting Started w/ ODM page w/ instructions to download

• Oracle Data Miner graphical user interface (GUI), • ODM Step-by-Step Tutorial

• Demo datasets

• ODM Discussion Forum on OTN (great for posting questions/answers)

• ODM 11g Sample Code (examples of ODM SQL and Java APIs applied in several use cases; great for

developers)

• Oracle’s 50+ SQL based statistical functions (t-test, ANOVA, Pearson’s, etc.)

Copyright 2010 Oracle Corporation