Grids Web Services The Global Operating system of the World Its Killer Applications

Grids Web Service

The Global Operating system

of the Worl

Its Killer Applications

February 28 2005

Geoffrey Fox

Community Grids Lab

Indiana University

e-Infrastructure

 e-Infrastructure builds on the inevitable increasing performance

of networks and computers linking them together to support new flexible linkages between computers, data systems and people

• Grids and peer-to-peer networks are the technologies that build e-Infrastructure

• e-Infrastructure called CyberInfrastructure in USA

 We imagine a sea of conventional local or global connections

supported by the “ordinary Internet”

• Phones, web page accesses, plane trips, hallway conversations

• Conventional Internet technology manages billions of

broadcast or low (one client to Server) or broadcast links

 On this we superimpose high value multi-way organizations

(linkages) supported by Grids with optimized resources and system support and supporting virtual (electronic) enterprises

• Low multiplicity fully interactive real-time sessions

A typical Web Service

n In principle, services can be in any language (Fortran .. Java ..

Perl .. Python) and the interfaces can be method calls, Java RMI Messages, CGI Web invocations, totally compiled away (inlining)

n The simplest implementations involve XML messages (SOAP) and

programs written in net friendly languages like Java and Python Paymen

Credit Card

Warehous e

Shipping control

WSDL interfaces

WSDL interfaces

Securit

y Catalog

Porta Service

Services and Distributed Objects

or remote machine with a set of well defined interfaces (ports) specified in XML (WSDL)

n Web Services (WS) have many similarities with Distributed

Object (DO) technology but there are some (important) technical and religious points (not easy to distinguish)

• CORBA Java COM are typical DO technologies

• Agents are typically SOA (Service Oriented Architecture)

n Both involve distributed entities but Web Services are more

loosely coupled

• WS interact with messages; DO with RPC (Remote Procedure Call) • DO have “factories”; WS manage instances internally and

interaction-specific state not exposed and hence need not be managed

• DO have explicit state (statefull services); WS use context in the messages to

link interactions (statefull interactions)

n Claim: DO’s do NOT scale; WS build on experience (with

Philosophy of Web Service Grids

•

Much of Distributed Computing was built by natural

extensions of computing models developed for sequential

machines

•

This leads to the

distributed object

(DO) model represented

by Java and

CORBA

– RPC (Remote Procedure Call) or RMI (Remote Method Invocation) for Java

•

Key people think this is not a good idea as it scales badly

and ties distributed entities together too tightly

– Distributed Objects Replaced by Services

•

Note

CORBA

was considered too complicated in both

organization and proposed infrastructure

– and Java was considered as “tightly coupled to Sun” – So there were other reasons to discard

Web services

•

Web Services

build

loosely-coupled,

distributed

applications,

(wrapping existing codes

and databases) based on

the

SOA

(service

oriented architecture)

principles.

•

Web Services interact by

exchanging messages in

SOAP

format

•

The contracts for the

message exchanges that

implement those

What is a Grid?

•

You won’t find a clear description of what is Grid and how

does

differ

from a

collection of Web Services

– I see no essential reason that Grid Services have different requirements than Web Services

– Geoffrey Fox, David Walker, e-Science Gap Analysis, June 30 2003. Report UKeS-2003-01,

http://www.nesc.ac.uk/technical_papers/UKeS-2003-01/index.html.

– Notice “service-building model” is like programming language – very personal!

•

Grids were once defined as “

Internet Scale Distributed

Computing

” but this isn’t good as Grids depend as much if

not more on data as well as simulations

Community Resources

 Grid Community databases have analogy to Television and the

News Web that allow individuals to communicate instantly with each other via Web Pages and Headline News acting as proxies

 N resources deposit information and N can view – Complexity

Large and Small Grids



N

resources in a

community

(N is billions for the world

and

1000-10000

for many scientific fields)



Communities

are arranged hierarchically with real

work being done in

“groups” of M resources

– M could

be

10-100

in e-Science



Metcalfe’s law

: value of network grows like square of

number of nodes M – we call Grids where this true

Metcalfe or M

_Grids



Nature of Interaction depends on size of M or N

• Shared Information O(N) Complexity Grids for largish N

• Complexity M2 _{Metcalfe Grids for smaller M < N}



Grids must merge with peer-to-peer networks

to

M

2

_Interactions

•

Superimpose M

“Grids” on the sea

(heatbath) of O(N)

“ordinary”

Architecture of (Web Service) Grids



Grids built from

Web Services

communicating through

an overlay network built in SOFTWARE on the

“ordinary internet” at the application level



Grids provide the

special quality of service

(security,

performance, fault-tolerance) and customized services

needed for “distributed complex enterprises”



We need to work with Web Service community as they

debate the 60 or so proposed Web Service specifications

• Use Web Service Interoperability WS-I as “best practice”

• Must add further specifications to support high performance

• Database “Grid Services” for O(N) Community case

Web Services WS-*

• Java is very powerful partly due to its many “frameworks” that generalize libraries e.g.

– Java Media Framework

– Java Database Connectivity JDBC

• Web Services have a correspondingly collections of specifications that represent critical features of the distributed operating systems for “Grids of Simple Services”

– Some 60 active WS-* specifications for areas such as – a. Core Infrastructure Specifications

– b. Service Discovery

– c. Security

– d. Messaging

– e. Notification

– f. Workflow and Coordination

– g. Characteristics

– h. Metadata and State

A List of Web Services I

• a) Core Service Architecture

• XSD XML Schema (W3C Recommendation) V1.0 February 1998, V1.1 February 2004

• WSDL 1.1 Web Services Description Language Version 1.1, (W3C note) March 2001

• WSDL 2.0 Web Services Description Language Version 2.0, (W3C under development) March 2004

• SOAP 1.1 (W3C Note) V1.1 Note May 2000

• SOAP 1.2 (W3C Recommendation) June 24 2003

• b) Service Discovery

• UDDI (Broadly Supported OASIS Standard) V3 August 2003

• WS-Discovery Web services Dynamic Discovery (Microsoft, BEA, Intel …) February 2004

• WS-IL Web Services Inspection Language, (IBM, Microsoft)

A List of Web Services II

• c) Security

• SAML Security Assertion Markup Language (OASIS) V1.1 May 2004

• XACML eXtensible Access Control Markup Language (OASIS) V1.0 February 2003

• WS-Security 2004 Web Services Security: SOAP Message Security (OASIS) Standard March 2004

• WS-SecurityPolicy Web Services Security Policy (IBM, Microsoft, RSA, Verisign) Draft December 2002

• WS-Trust Web Services Trust Language (BEA, IBM, Microsoft, RSA, Verisign …) May 2004

• WS-SecureConversation Web Services Secure Conversation Language (BEA, IBM, Microsoft, RSA, Verisign …) May 2004

A List of Web Services III

• d) Messaging

• WS-Addressing Web Services Addressing (BEA, IBM, Microsoft)

March 2004

• WS-MessageDelivery Web Services Message Delivery (W3C Submission by Oracle, Sun ..) April 2004

• WS-Routing and Referral SOAP Routing Protocol (Microsoft) October 2001

• WS-RM Web Services Reliable Messaging (BEA, IBM, Microsoft, Tibco) v0.992 March 2004

• WS-Reliability Web Services Reliable Messaging (OASIS Web Services Reliable Messaging TC) March 2004

• SOAP MOTM SOAP Message Transmission Optimization Mechanism (W3C) June 2004

• e) Notification

• WS-Eventing Web Services Eventing (BEA, Microsoft, TIBCO)

January 2004

• WS-Notification Framework for Web Services Notification with WS-Topics, WS-BaseNotification, and WS-BrokeredNotification (OASIS) OASIS Web Services Notification TC Set up March 2004

A List of Web Services IV

• f) Coordination and Workflow, Transactions and Contextualization

• WS-CAF Web Services Composite Application Framework including WS-CTX,

WS-CF and WS-TXM below (OASIS Web Services Composite Application Framework TC) July 2003

• WS-CTX Web Services Context (OASIS Web Services Composite Application Framework TC) V1.0 July 2003

• WS-CF Web Services Coordination Framework (OASIS Web Services Composite Application Framework TC) V1.0 July 2003

• WS-TXM Web Services Transaction Management (OASIS Web Services Composite Application Framework TC) V1.0 July 2003

• WS-Coordination Web Services Coordination (BEA, IBM, Microsoft) September 2003

• WS-AtomicTransaction Web Services Atomic Transaction (BEA, IBM, Microsoft) September 2003

• WS-BusinessActivity Web Services Business Activity Framework (BEA, IBM, Microsoft) January 2004

• BTP Business Transaction Protocol (OASIS) May 2002 with V1.0.9.1 May 2004 • BPEL Business Process Execution Language for Web Services (OASIS) V1.1 May

2003

• WS-Choreography (W3C) V1.0 Working Draft April 2004

• WSCI (W3C) Web Service Choreography Interface V1.0 (W3C Note from BEA, Intalio, SAP, Sun, Yahoo)

A List of Web Services V

• h) Metadata and State

• RDF Resource Description Framework (W3C) Set of recommendations expanded from original February 1999 standard

• DAML+OIL combining DAML (Darpa Agent Markup Language) and OIL (Ontology Inference Layer) (W3C) Note December 2001

• OWL Web Ontology Language (W3C) Recommendation February 2004

• WS-DistributedManagement Web Services Distributed Management Framework with MUWS and MOWS below (OASIS)

• WSDM-MUWS Web Services Distributed Management: Management Using Web Services (OASIS) V0.5 Committee Draft April 2004

• WSDM-MOWS Web Services Distributed Management: Management of Web Services (OASIS) V0.5 Committee Draft April 2004

• WS-MetadataExchange Web Services Metadata Exchange (BEA,IBM, Microsoft, SAP) March 2004

• WS-RF Web Services Resource Framework including WS-ResourceProperties,

WS-ResourceLifetime, WS-RenewableReferences, WS-ServiceGroup, and

WS-BaseFaults (OASIS) Oasis TC set up April 2004 and V1.1 Framework March 2004

• ASAP Asynchronous Service Access Protocol (OASIS) with V1.0 working draft G June 2004

A List of Web Services VI

•

g) General Service Characteristics

•

WS-Policy

Web Services Policy Framework (BEA, IBM,

Microsoft, SAP)

May 2003

•

WS-PolicyAssertions

Web Services Policy Assertions

Language (BEA, IBM, Microsoft, SAP)

May 2003

•

WS-Agreement

Web Services Agreement Specification

(GGF under development)

May

2004

•

i) User Interfaces

•

WSRP

Web Services for Remote Portlets (OASIS)

OASIS Standard

August 2003

•

JSR168

:

JSR-000168 Portlet Specification for Java

A List of Web Services VII

• j) Recent Updates ………

• WS-Eventing important update of this notification specification with IBM, Sun and others joining Microsoft et al. as authors

• WS-Enumeration supporting the splitting of a single entity (file or stream) into multiple messages

• WS-Transfer supporting the creation, update (by get or put) or deletion of a resource

• WS-Management competes with WS-DM to provide a Web Service to manage resources

• WS-PolicyAttachment describes how to associate policies with UDDI and Endpoints and how to integrate with WSDL

• WS-DAI is a Web Service of the OGSA-DAI Grid linkage with databases

• WS-CIM is a Web Service rendering from DMTF (Distributed Management Task Force) of the industry standard CIM

(Common Information Model) of metadata for computer devices

Peer to Peer Grid

Database Database

Peers Peers

Peer to Peer Grid A democratic organization

User Facin

Web Service Interfaces

Service Facin

Web Service Interfaces

Event Messag Brokers

Event Messag Brokers

Importance of SOAP

•

SOAP defines a very obvious message structure with a

header

and a

body

•

The

header

contains information used by the “

Internet

operating system

”

–

Destination, Source, Routing, Context, Sequence

Number …

•

The

message body

is partly further information used by the

operating system and partly information for

application

when it is not looked at by “operating system” except to

encrypt, compress it etc.

–

Note WS-Security supports separate encryption for

different parts of a document

•

Much discussion in field revolves around

what is referenced

in header!

Deployment Issues for “System Services”

• “System Services” are ones that act before the real

application logic of a service

• They gobble up part of the SOAP header identified

by the namespace they care about and possibly part

or all of the SOAP body

–

e.g. the XML elements in header from the WS-RM

namespace

• They return a modified SOAP header and body to

next handler in chain

WS-R Handl

er

WS-……. Handler Header

Body

Messaging Structure

•

Communication Services are messaging

(transport protocol, routing) using SOAP

protocol

Invoke Other Services

from Header or Body

Messaging

Process SOA Header Body Process SOA

Body Header

Customizable Handle Chain processe

SOAP Header Servic

Bit

level

Internet

(OSI

Stack)

Layered Architecture for Web Services and Grids

Base Hosting Environment

Protocol HTTP FTP DNS …

Presentation XDR …

Session SSH …

Transport TCP UDP …

Network IP …

Data Link / Physical

Servic Internet

Application Specific Grids

Generally Useful Services and Grids

Workflow WSFL/BPEL

Service Management (“Context etc.”)

Service Discovery (UDDI) / Information

Service Internet Transport



Protocol

Service Interfaces WSDL

Servic Context

Higher Level

Working up from the Bottom

 We have the classic (CISCO, Juniper ….) Internet routing the

flood of ordinary packets in OSI stack architecture

 Web Services build the “Service Internet” or IOI (Internet on

Internet) with

• Routing via WS-Addressing not IP header

• Fault Tolerance (WS-RM not TCP)

• Security (WS-Security/SecureConversation not IPSec/SSL)

• Data Transmission by WS-Transfer not HTTP

• Information Services (UDDI/WS-Context not DNS/Configuration files)

• At message/web service level and not packet/IP address level

 Software-based Service Internet possible as computers “fast”  Familiar from Peer-to-peer networks and built as a software

overlay network defining Grid (analogy is VPN)

 SOAP Header contains all information needed for the “Service

Consequences of Rule of the Millisecond

•

Useful to remember

critical time scales

–

1)

0.000001 ms

– CPU does a calculation

–

2a)

0.001 to 0.01 ms

– Parallel Computing MPI latency

–

2b)

0.001 to 0.01 ms

– Overhead of a Method Call

–

3)

1 ms

– wake-up a thread or process

–

4)

10 to 1000 ms

– Internet delay

•

2a), 4) implies geographically distributed

metacomputing

can’t in general compete with parallel systems

•

3) << 4) implies a software overlay network is possible

without significant overhead

–

We need to explain why it adds value of course!

•

2b) versus 3) and 4) describes regions where

method

and

Linking Modules

n

From method based to RPC to message based to event-based

publish-subscribe Message Oriented Middleware

Module A Module

B

Method Call .001 to 1 millisecond

Service A Service

B Message_s

0.1 to 1000 millisecond latency

Coarse Grain Service Model

Closely coupled Java/Python …

Service B Service A

Publisher Post Events “Listener

Subscribe to Events

What is a Simple Service?

• Take any system – it has multiple functionalities

– We can implement each functionality as an independent distributed service

– Or we can bundle multiple functionalities in a single service • Whether functionality is an independent service or one of many

method calls into a “glob of software”, we can always make them as Web services by converting interface to WSDL

• Simple services are gotten by taking functionalities and making as small as possible subject to “rule of millisecond”

– Distributed services incur messaging overhead of one (local) to

100’s (far apart) of milliseconds to use message rather than method call

– Use scripting or compiled integration of functionalities ONLY when require <1 millisecond interaction latency

• Apache web site has many projects that are multiple functionalities presented as (Java) globs and NOT (Java) Simple Services

Grids of Grids of Simple Services

• Link via methods  messages  streams

• Services and Grids are linked by messages

• Internally to service, functionalities are linked by methods

• A simple service is the smallest Grid

• We are familiar with method-linked hierarch

Lines of Code  Methods  Objects  Programs  Packages

Overlay and Compose

Grids of Grids Methods Services Component Grids

CPUs Clusters Compute Resource Grids MPPs

Databases _DatabasesFederated

Sensor Sensor Nets

Data

Component Grids?

• So we build collections of Web Services which we

package as

component Grids

–

Visualization Grid

–

Sensor Grid

–

Utility Computing Grid

–

Person (Community) Grid

–

Earthquake Simulation Grid

–

Control Room Grid

–

Crisis Management Grid

Critical Infrastructure (CI) Grids built as Grids of Grids

Gas Service and Filters

Physical Network Registr

y Metadata

Flood Service and Filters

Flood CIGrid

_…

…

Data

Access/Storage Securit

y Notification Workflow Messaging Portal

s Visualization_Grid Collaboration

Grid

Sensor Grid Compute

Grid GIS Grid

a

Topography 1 km

Stress Change

Earthquakes

PBO

Site-specific Irregular

Scalar Measurements Constellations for Plate Boundary-Scale Vector Measurements

a

a

Ice Sheets Volcanoes

Long Valley, CA

Northridge, CA

Database Database Analysis and Visualizatio Portal Repositorie Federated Databases Data Filte Services

Field Trip Data

Streaming Data Sensor s

?

Geoscience Research and Education Grids

GI Grid

Sensor Grid Database Grid