Service Oriented Collaboration and Community Grids

(1)

Service Oriented Collaboration and

Community Grids

CTS2006 May 17 2006

2006 International Symposium on Collaborative Technologies and Systems

http://www.engr.udayton.edu/faculty/wsmari/cts06/ Las Vegas NV

Geoffrey Fox Anabas Inc. and

Computer Science, Informatics, Physics Pervasive Technology Laboratories Indiana University Bloomington IN 47401

(2)

Database

S S

S

S SS SS SS SS SS SS SS SS

F S F S F S F S F S F S F S F

S SF

F S F S F S F S F S F S F S F S F S F S F

S

Por

tal

F S O S O S O S O S O S O S O S O S O S O S O S O S MD MD MD MD MD MD MD MD MD

MetaData Filter Service Sensor Service Other Service Anothe Grid

Raw Data  Data  Information  Knowledge  Wisdom

Decisions S S S S Anothe Service Anothe Service S S Anothe

Grid S S

(3)

Grid Service Philosophy I

n

Services

receive

data

in

SOAP messages, manipulate it

and produce

transformed data

as further messages

n

Knowledge is created

from information by services

•

Information

is created from data by services

n

Semantic Grid

comes from building

metadata rich

systems of services

n

Meta-data

is

carried

in

SOAP

messages

n

The Grid enhances

Web services with

semantically rich

system and application specific

management

n

One must exploit and work around the

different

approaches to

meta-data (state)

and their

manipulation

in Web Services

(4)

Grid Service Philosophy II

n

There are a horde of

support services

supplying security,

collaboration, database access, user interfaces

n

The support services are either associated with

system or

application where the former are WS-* and GS-*

which

implicitly or explicitly define many support services

n

There are generalized

filter services

which are applications that

accept messages and produce new messages with some data

derived from that in input

•

Simulations (including PDE’s and reactive systems)

•

Data-mining

•

Transformations

•

Agents

•

Reasoning

are all termed filters here

n

Agent Systems

are a

special case

of

Grids

n

Peer-to-peer systems

can be built as a

Grid

with particular

discovery and messaging strategies

(5)

Grid Service Philosophy III

n

Filters

can be a

workflow

which means they are “just

collections of other simpler services”

n

Grids

are

distributed systems

that accept

distributed

messages and produce distributed result messages

n

A

service

or a

workflow

is a

special case

of a

Grid

n

A collection of services

on a

multi-core chip

is a

Grid

n

Sensors

or

Instruments

are

“managed”

by

services;

they may

accept

non SOAP

control messages

and

produce data

as

messages

(that are not usually SOAP)

n

Collaborative services

share either

input

(replicated

model) or

output ports

n

Collaboration

involves a

sharing messaging system

(naturally

publish-subscribe) and a

control

formalism

(XGSP

is

SOAP

compatible

H323/SIP)

(6)

Database

S S

S

S SS SS SS SS SS SS SS

S SF

F S F S F S F S F S F S F S F S F S

Por

_tal

F S O S O S O S O S O S O S O S O S O S O S O S O S MD MD MD MD MD MD MD

Grid S S

(7)

W Displa y W Viewe r WS Displa y WS Viewe r Even (Message Service Master W Displa y WS Viewe r We Servic e F I U O F I S O Othe Participants We Servic e F I U O F I S O We Servic e F I U O F I S O

Shared Input Port (Replicated WS) Collaboration with UFI

(8)

W Displa

y W

Viewe r

WS Displa

y WS

Viewe r

Even (Message

Service

Master

W Displa

y WS

Viewe r

Application o

Content

source

WSD L

Web Service

F I U

O F

I S

O

Shared Output Port (Single WS) Collaboration that

Can be shared at any point on visualization pipeline

Othe

Participants

(9)

Database

S S

S

S SS SS SS SS SS SS SS SS

S SF

F S F S F S F S F S F S F S F S F S F

S

Por

tal

F S O S O S O S O S O S O S O S O S O S O S O S MD MD MD MD MD MD MD MD

Grid S S

Anothe Grid S S S S S S S S S S S S S S S S F S SO_AP Messag es

Por

_tal

Por

tal

Collaboration by Message Replication at any point i

filter chain MD

(10)

(11)

Latency ms

# Receivers One session Multipl

sessions

30 frames/sec

Video Performance

(12)

The Grid and Web Service Institutional Hierarchy

OGSA GS-*

and some WS-* GGF/W3C/…

XGSP (Collab)

WS-* fro OASIS/W3C Industry

Apache Axi .NET etc.

Must set standards to get interoperability

2: System Services and Features

**(WS-* from OASIS/W3C/Industry)**

Handlers like WS-RM, Security, UDDI Registry

3: Generally Useful Services and Features

(OGSA and other GGF, W3C)

Such as

“Collaborate”, “Access a Database” or “Submit a Job”

4: Application or Community of Interest (CoI

Specific Services

such as “Map Services”, “Run

BLAST” or “Simulate a Missile”

1: Container and Run Time (Hosting)

Environment (Apache Axis, .NET etc.)

XBM

XTCE VOTABLE CML

CellML

(13)

**The Ten areas covered by the 60 core WS-* Specifications**

WSRP (Remote Portlets)

10: Portals and User Interfaces

WS-Policy, WS-Agreement

9: Policy and Agreements

WSDM, WS-Management, WS-Transfer

8: Management

WSRF, WS-MetadataExchange, WS-Context

7: System Metadata and State

UDDI, WS-Discovery

6: Service Discovery

WS-Security, WS-Trust, WS-Federation, SAML, WS-SecureConversation

5: Security

BPEL, WS-Choreography, WS-Coordination

4: Workflow and Transactions

WS-Notification, WS-Eventing (Publish-Subscribe)

3: Notification

WS-Addressing, WS-MessageDelivery; Reliable Messaging WSRM; Efficient Messaging MOTM

2: Service Internet

XML, WSDL, SOAP

1: Core Service Model

Examples

**WS-* Specification Area**

RTI and NCOW needs all of

(14)

Activities in Global Grid Forum Working Groups

Authorization, P2P and Firewall Issues, Trusted Computing

7: Security

Resource/Service configuration, deployment and lifetime, Usage records and access, Grid economy model

6: Management

Network measurements, Role of IPv6 and high performance networking, Data transport

5: Infrastructure

Database and File Grid access, Grid FTP, Storage Management, Data replication, Binary data specification and interface, High-level publish/subscribe, Transaction management

4: Data

Job Submission, Basic Execution Services, Service Level Agreements for Resource use and reservation, Distributed Scheduling

3: Compute

Software Interfaces to Grid, Grid Remote Procedure Call, Checkpointing and Recovery, Interoperability to Job Submittal services, Information Retrieval,

2: Applications

High Level Resource/Service Naming (level 2 of slide 6), Integrated Grid Architecture

1: Architecture

**GS-* and OGSA Standards Activities**

GGF Area

(15)

The Global Information Grid Core Enterprise Services

Provisioning, operations and maintenance of applications.

CES9: Application

Retention, organization and disposition of all forms of data

CES8: Storage

Includes automated and manual methods of optimizing the user GiG experience (user agent)

CES7: User Assistance

Provision and control of sharing with emphasis on synchronous real-time services

CES6: Collaboration

Includes translation, aggregation, integration, correlation, fusion, brokering publication, and other transformations for services and data. Possibly agents

CES5: Mediation

Searching data and services

CES4: Discovery

Synchronous or asynchronous cases

CES3: Messaging

Supports confidentiality, integrity and availability. Implies reliability and autonomic features

CES2: Information

Assurance (IA)/Security

including life-cycle management

CES1: Enterprise Services Management (ESM)

Service Functionality

Core Enterprise Services

(16)

The Core Service Areas I

ECS WS9

FS10: Policy

CIM NCES1

GS6 WS8

FS9: Management

Globus MDS Semantic Grid WS7

FS8: System Metadata & State

NCES4 WS6

FS7: Discovery

Grid-Shib, Permis Liberty Alliance ... NCES2

GS7 WS5

FS6 : Security

Grid Programming NCES5

WS4

FS5 Workflow

JMS, MQSeries NCES3

WS3

FS4: Notification

Streams/Sensors NCES3

WS2

FS3: Service Internet, Messaging B: Core Services

Strategy for legacy subsystems and modular architecture

FS2: Grid of Grids

Core Service Model, Build Grids on Web Services. Industry best practice

WS1

FS1: Use SOA: Service Oriented Arch. A: Broad Principles

Comments NCES

(DoD) GS-*

WS-* Service or Feature

(17)

The Core Service Areas II

GS3

FS18: Scheduling and matching of Services and Resources

XGSP, Shared Web Service ports NCES6

GS7

FS17: Collaboration and Virtual Organizations

Ad-hoc networks GS5

FS16: Resources and Infrastructure

Standalone Services Proxies for jobs NCES9

GS2

FS15: Applications and User Services

JBI for DoD, WFS for OGC GS4

FS14: Information

NCOW Data Strategy NCES8

GS4

FS13: Data and Storage

GS3

FS12: Computing

Portlets JSR168, NCES Capability Interfaces NCES7

WS10

FS11: Portals and User assistance

B: Core Services (Continued)

Comments NCES

GS-* WS-*

Service or Feature

(18)

Physical Network (monitored by FS16)

7:

Discovery

8:Metadat

a

BioInformatics

Grid

Chemical Informatics

Grid

_…

_{Domain Specific}

Grids/Services

…

4:

Notification

6:

Security

Messaging

3:

_Workflow

5:

Management

9:

14: Informatio

Instrument/Senso

r

12: Computing

Core Low Level Grid Services

9: Management

18: Scheduling

10:

Policy

Using the Grid of Grids and Core Services of previous table to build

Multiple application grids re-using common components

15: Application Services

Screening Tools

Quantum

Calculations

15: Application Services

Sequencing Tools

Biocomplexity

Simulations

11:

Portals

17: Collaboration

Se

rvi

ce

s

13: Data

Access/Storage

(19)

Some Conclusions I

n

One can map

7.5 out of 9

NCOW/NCE and GiG core

**capabilities into Web Service (WS-) and Grid (GS-)**

architecture and core services

•

Analysis of Grids in NCOW/NCE document

inaccurate (confuse Grids and Globus and only

consider early activities)

n

Some “mismatches” on both NCOW and Grid sides

n

GS-/WS-

do

not

have

collaboration

and miss some

messaging

n

NCOW does not have at core level

system metadata

and

resource/service scheduling

and matching

n

Higher level services

of importance include

GIS

(Geographical Information Systems),

Sensors

and

data-mining

(20)

Some Conclusions II

n

Criticisms

of Web services in a paper by Birman seem to

be

addressed by Grids

or reflect immaturity of initial

technology implementations

n

NCOW/NCE does not seem to have any analysis of how

to build their systems on

WS-/GS-

technologies in a

layered fashion; they do have a layered service

architecture so this can be done

•

They agree with

service oriented architecture

•

They seem to have

no process

for agreeing to WS-*

**GS-* or setting other standards for CES**

n

Grid of Grids

allows modular architectures and natural

treatment of legacy systems

•

Note Grids, Services and Handlers are all “just”

entities with distributed

message-based input and

output

interfaces

(21)

Some Next Steps

n

Anabas Phase II SBIR:

n

Produce a Grid-based implementation for 9 CES for

NCOW adding ECS (Environmental Control Services) and

Metadata support (UDDI and WS-Context for C2IEDM

etc.)

n

Produce typical Collaboration, Sensor, Datamining and

GIS Grids

n

Produce a Tool to allow composition of services and grids

into (larger) Grids (Systems of Systems)

n

Community Grids Laboratory:

n

Continue Grids for Earth Science and Sensors with JPL

n

Build a “Complete Scientific Method” Grid covering data

through wisdom (instrument through publication) and

integration of “Web 2.0” tools like del.icio.us

(22)

Existing User Interface

Document-enhanced Research Grid

etc. Google Scholar

Manuscript Central Science.gov Windows Live Academic Search

Citeseer

CMT Conferenc Management

Existing Document-base Research Tools

Web servic Wrappers New Document-enhanced

Research Tools Integration

Enhancement User Interface Community Tools Generic Document Tools

MyResearc Database Bibliographic

Database Export RSS, Bibte

Endnote etc. CiteULike

Connotea Del.icio.us

Bibsonomy

Biolicious PubChem

PubMed Traditiona

Cyberinfrastructure

(23)

Location of software for Grid Projects in

Community Grids Laboratory

n

htpp://www.naradabrokering.org p

rovides Web service

(and JMS) compliant

distributed publish-subscribe

messaging

(software overlay network)

n

h

tpp://www.globlmmcs.org is

a

service oriented (Grid)

collaboration environment

(audio-video conferencing)

n

ht

tp://www.crisisgrid.org is

an OGC (open geospatial

consortium) Geographical Information System (GIS)

compliant

GIS and Sensor Grid

(with POLIS center)

n

htt

p://www.opengrids.org has

WS-Context, Extended

UDDI etc.

n

The work is still in progress but NaradaBrokering is

quite mature

n