Observation that involves

Observation that involves

•  People

•  Data

•  Analytic tools

The Web Observatory aims to bring those

together into a global distributed resource

understanding  web  evolu1on:  

-­‐ 

observa1on  

The Web

Observatory

Web evolution

•  The Web of documents

•  The Web of people

•  The Web of data and online social networks

Hall, W., & Tiropanis, T. (2012). Web evolution and Web Science. Computer Networks, 56(18), 3859–3865. doi:10.1016/j.comnet. 2012.10.004

Digital Literacy Evolution

•  Engaging with the Web of Documents

•  E-Commerce

•  Web 2.0: Contributing content and data

•  Engaging with Data (Crowdsourcing, Mashups)

•  Contributing Applications

The Web as:

•  A mirror

•  A catalyst

•  A lens

The promise of Big Data

•  Insights – inform policy

•  Competitive advantage

•  Innovation

Issues:

Do we have enough analytic talent? Do we have enough datasets?

Big Data

The promise of the Web of Data

•  Publication of data

•  Combining data

•  Engagement with the crowd

Issues:

What about non-open data? Queries performance?

Levels of Sharing

Open Datasets Shareable Datasets Private & Organisational Datasets

The Web Observatory is about sharing

datasets and analytic applications on a

global, distributed scale

Sharing on the Web Observatory

•  Sharing Datasets

•  Sharing Ways of Interacting with Datasets

•  Sharing Analytics and Visualisations

EXPOSING BOTH DATASETS AND ANALYTICS LINKING EXPLICITLY ANALYTIC TOOLS TO

The case for analytics on distributed datasets

•  Up-to-date datasets

•  Reducing redundancy

•  Exposing part of a dataset when needed

–  Using Personal DataStores

The case for research on the WO - R dimensions

16

Web Observatory Value

•  Big Data analytics on the Web of Data

•  Sharing and discovering datasets

•  Discovery of analytic applications and visualisations

•  Distributed analytics on up-to-date datasets reducing

redundancy

•  Addressing the R-dimensions

•  Engaging communities with analytic resources

Open Data Cloud! e-Science Infrastructures! Internet Archive! Social Media! Open Web Analytics!

Semantic Catalogues! Harvesting and

Archival! Harmonised Access! Harmonisation with

Other Observatories!

Catalogues of Analytic

and Visualisation Tools! Open Standards!

Social Innovation! Interdisciplinary! e-Science!

Novel Analysis and Visualisation Methods!

Repositories! with Authorised

Access!

Web of Observatories

Tiropanis, T., Hall, W., Shadbolt, N., de Roure, D., Contractor, N., & Hendler, J. (2013). The Web Science Observatory. Intelligent Systems, IEEE, 28(2), 100–104. doi:10.1109/MIS.2013.50

Web of Observatories

WO Portal WO Tools WO Datastores WO Portal WO Tools WO Datastores WO Portal WO Tools WO Datastores WO Portal WO Tools WO Datastores WO Portal WO Tools WO Datastores

Web Observatory _{Web Observatory}

Web Observatory

Thanassis Tiropanis – University of Southampton

Web Observatory Web Observatory

Public Engagement

Resource Description

Resource Sharing

Distributed Queries

Performance

Searching

Web Observatory Actors

•  Web Science Trust

•  Web Science Trust network

•  Web Foundation

•  Open Data Institute

•  Web Science DTC

•  W3C Community Group

•  The Internet Archive

•  Industrial partners

•  Research projects

(SOCIAM, EINS) •  SoFWIReD

The Southampton

Web Observatory

Southampton Web Observatory (SUWO)

Datasets

Analytic &

Visualisation

Tools

Datasets

_Datasets

Analytic &

Visualisation

Tools

Analytic &

Visualisation

Tools

Sharing

Distribution

Optimisation

Interoperability

SUWO Infrastructure

 

Microblogging Datasource-centric

Harvesting Topic-centric Harvesting

SQL SUWO Portal

Analytic and Visualisation Apps

Clickstream data Web content Wikipedia access

HBase Hadoop

MongoDB 4Store

Dataset Description Schema & Catalogue Sharing and Querying across platforms

Web Observatory Concept

WO Portal

Engaging communities with analytics Publication of catalogues (schema.org)

Access with/without credentials Searching and Indexing

Distributed Queries

Plugged in Datastores and App Servers

Harvesting

Dataset enrichment/curation Dataset management Dataset profiling and preview

Provenance Optimisation

WO Datastores

Hosting of analytic apps Hosting of visualisation apps

Monitoring dependency on datasets

Monitoring dependency on tools

Explicit links between apps & datasets used

WO Apps WO Portal WO Apps WO Datastores WO Portal WO Apps WO Datastores

Links to resources in other Web Observatories

Thanassis Tiropanis – University of Southampton

People Data

Describing WO resources – Schema.org

http://tw.rpi.edu/web/WOschema 26 People Data Analytic tools

Harvesting Architecture

API access to shared datasets

28 Xin Wang – University of Southampton

SUWO server

OAuth server SUWO web interface

Authorise

Login to Authenticate

Access RESTful API User Client

SUWO Architecture & the Web of Observatories

WO Portal WO Tools WO Datastores WO Portal WO Tools WO Datastores WO Portal WO Tools WO Datastores WO Portal WO Tools WO Datastores WO Portal WO Tools WO Datastores

IIIT Bangalore Web Observatory

IIITB, INFOSYS, COGNIZANT

RPI Web Observatory

http://tw.rpi.edu/web/web_observatory

Discovering Web Observatories

32

EINS – Evidence base

http://dev-001.ecs.soton.ac.uk:8000

Web of Observatories

WO Portal WO Tools WO Datastores WO Portal WO Tools WO Datastores WO Portal WO Tools WO Datastores WO Portal WO Tools WO Datastores WO Portal WO Tools WO Datastores

Web Observatory _{Web Observatory}

Web Observatory

Web Observatory _{Web Observatory}

Public Engagement

Resource Description

Resource Sharing

Distributed Queries

Performance

Searching

Thanassis Tiropanis

tt2@ecs.soton.ac.uk