Presentation

(1)

Web Services and Peer-to-Peer

Technologies for the Grid

PTLIU Laboratory for Community Grids

Geoffrey Fox

Computer Science, Informatics, Physics

Indiana University, Bloomington IN 4740

http://grids.ucs.indiana.edu/ptliupages/presentations/iccsapril02

[email protected]

(2)

The Application Service Model

n

There are generic

Grid system services

: security, collaboration,

persistent storage, universal access

• OGSA (Open Grid Service Architecture) is implementing these as

extended Web Services

n

An

Application Web Service

is a capability used either by another

service or by a user

• It has

input

and

output ports

– fed by devices or other services

n

A service “is a

component

” and is a replacement for a library in case

where performance allows

n

Services (components)

are a sustainable model of software development

– each service has documented capability with standards compliant

interfaces

• XML

defines interfaces at several levels

• WSDL

at Service interface level and

XSIL

or equivalent for

scientific data format

n

A service can be written as Perl, Python, Java Servlet, Enterprise

Javabean, CORBA (C++ or Fortran) Object …

n

Communication

protocol can be RMI (Java), IIOP (CORBA) or SOAP

(HTTP, XML) ……

(3)

What is a Web Service I

n

A

web service is a computer program

running on either the local

or remote machine with a set of well defined interfaces (ports)

specified in XML (WSDL)

n

In principle, computer program can be in any language

(Fortran .. Java .. Perl .. Python) and the interfaces can be

implemented in any way what so ever

• Interfaces can be method calls, Java RMI Messages, CGI Web

invocations, totally compiled away (inlining) but

n

The simplest implementations involve

XML messages (SOAP)

and programs written in net friendly languages like Java and

Python

n

Web Services separate the

meaning of a port (message) interface

from its implementation

n

Enhances/Enables Re-usable component model of ANY

(4)

etc.

XML WS to WS Interfaces

(Virtual) XML Knowledge (User)

Interface

Clients

(Virtual) XML Data

Interface

Raw

Data

Ra

Resource

s

Raw

Data

W

S

W

S

Web Service

(WS)

W

S

W

S

W

S

W

S

W

S

W

S

(5)

Classic Grid Architecture

Database

Netsolv

e

Computin

g

Securit

y

Collaboratio

n

Compositio

n

Content

Access

Resources

Client

Users and Devices

Middle Tie

Brokers

Service

Providers

Middle Tier becomes Web Services

(6)

What is a Web Service II

n

Web Services have important implication that

ALL

interfaces are XML messages based.

In contrast

n

Most Windows programs have interfaces defined as

interrupts due to user inputs (see WSIA and W3C

DOM)

n

Most software have interfaces defined as methods which

might be implemented as a message but this is often

NOT explicit

Securit

y

Catalo

g

Paymen

Credit

Card

Warehous

e

shipping

WSDL

interfaces

WSDL

interfaces

(7)

What is a Web Service III

n

This is a server NOT a client model

n

“Everything electronic” is a

resource (= distributed object or = service)

• Computers; Programs; People

• Data (from sensors to this presentation to email to

databases)

n

All

resources have interfaces

which are defined in

XML

for

both

properties

(data-structure) and

methods

(service,

function, subroutine)

• We can assume that a data-structure property has

getproperty()

and

setproperty(value)

methods to act as

interface

n

All resources are linked by

messages

with structure, which

must be specifiable in XML

(8)

WSDL Abstractions

n

WSDL

abstracts a program

as an entity that does

something given one or more inputs with its results

defined by streams on one or more outputs.

n

Functions are defined by method name and

parameter

methodname(parm1,parm2, … parmN)

• Where parameters are “Input” “Output” or both

n

In WSDL, we will have a

Web Service

which like a

(Java or CORBA Program) can be thought of as a

(distributed) object with many methods

• Instead of a function call, the “calling routine” sends an

XML message to the Web Service specifying

methodname

and values of the parameters

• Note name of function is just another parameter

(9)

Details of WSDL Protocol Stack

n

UDDI

finds where programs are

• remote( (distributed) programs

are just Web Services

n

WSFL

links programs togethe

(under revision?)

n

WSDL

defines interface (methods,

parameters, data formats)

n

SOAP

defines structure of message

including serialization of information

n

HTTP

is negotiation/transport

protocol

n

TCP/IP

is layers 3-4 of OSI

n

Physical Network

is layer 1 of OSI

UDDI or WSIL

WSFL

WSDL

SOAP or RMI

HTTP or SMTP

or IIOP or RMTP

TCP/IP

(10)

Examples of Web Services I

n

OGSA (Open Grid Service Architecture)

• Integrate Web Service and Grid Concepts and allows Globus

to be implemented as Web Services

n

Audio-Video Conferencing

as a Web Service

• Integrates H323, SIP, JXTA (etc.) protocols by mapping to

single XML Interface

• Provides VRVS reflector model from Messaging Web Service

n

Messaging or Event Web Service

provides intelligent routing

and buffering of messages

n

Computing

as a Web service

• Job submittal, status, composition, data services, visualization

• Performance WS

allows access to distributed monitoring

data, analysis, models, and final benchmarks with

interoperable XML interfaces

(11)

Examples of Web Services II

n

Education

as a Web Service

• One of easiest to do as object standards well defined (IMS)

and little performance issues

• Grading, Homework submission, registration, assessment etc.

n

Universal Access

and Web Services

• As Web Services allow multiple implementation of a

particular interface, one can adjust to needs of particular

clients (PDA v. versus, impaired sight etc.). See

WSRP

• Can build custom implementations of certain web services for

particular communities but re-use others

n

Collaborative Web Services

• As interfaces all message based,

much easier to share Web

(12)

Education as a Web Service

n

Can link to Science as a Web Service and substitute educational

modules

n

“

Learning Object

” XML standards already exist from IMS/ADL

h

ttp://www.adlnet.org –

need to update architecture

n

Web Services for

virtual university

include:

n

Registration

n

Performance

(grading)

n

Authoring

of Curriculum

n

Online laboratories

for real and virtual instruments

n

Homework submission

n

Quizzes

of various types (multiple choice, random parameters)

n

Assessment

data access and analysis

n

Synchronous Delivery

of Curricula

n

Scheduling

of courses and mentoring sessions

n

Asynchronous access, data-mining and

knowledge discovery

n

Learning Plan agents

to guide students and teachers

(13)

Message

Or Event

Based

Inte

Connection

Reso

urce

Data

base

Reso

urce

Sof

ware

Sof

ware

XM

Skin

e-Science is XML Specified Resource

connected by XML specified messages

XM

Skin

(14)

Biased History of Computing

n

In almost the beginning, there was

Fortran

and formats

(6I5, 5F10.4) for data

n

………..

n

1993-1997:

HTML

came along for Web Pages

n

1998-…:

XML

was developed to define information in

documents while HTML defining rendering

• But soon it became used for specifying

all data

and their

format

n

2001:

Web Services

allowed XML to specify

methods

(subroutines) as well as data

n

Java, C++, Python, Perl, .. Fortran

are now “just” the

insides of

XML

specified

programs

(15)

How do we organize all this

n

As everything is a

resource implemented as a Web

Service

, we need a common description framework for

• back end supercomputers and a petabyte data

• Microsoft PowerPoint and this file

n

Grids

tend to organize large back end resources but

peer to peer (P2P) technology

more natural for the

digital equivalent of the scraps of information in each

of the multiple sub-communities in a “virtual

organization”

n

Grids

and

P2P

approaches can be integrated by

building both in terms of

Web Services

with different

implementations of

core services

such as

discovery

, and

event or message transport/filter

…..

• Use JXTA as “typical” P2P technology as architecture clear

(16)

Peer to Peer Grid

Database

JXTA

Peer to Peer Grid

Web Service Interfaces

Event

Messag

Brokers

Integrate P2P

and Grid/WS

Event

Messag

Brokers

(17)

Broker Network

Data

base

Reso

urce

Broker

Software

multicast

(P2P) Community

For message/events service

(P2P) Community

(18)

Event Web Service

n

Filter

is mapping to PDA or slow communication channel

(universal access) – see our PDA adaptor

n

Workflow

natural as all messages “intercepted” by Event Web

Service

n

Routing

illustrated by JXTA

n

Destination-Source matching

illustrated by JMS

• Publish-subscribe

Web

Service 1

(Virtual

Queue

Service 2

Web

Destinatio

Source Matching

Filter

Routin

g

workflo

w

WSD

Ports

Broker

WSD

Ports

(19)

Messaging/Events as a Web Service

n

We can implement

messaging subsystem

(between

WSDL resources) with either direct messages or by a

queued system where you publish messages to queues

and subscribe as receiver to particular queues

• events are time stamped messages

n

There are many different publish/subscribe models

• JMS

is a

classic Grid-like cluster

of central servers

• JXTA

is a very

dynamic Peer to Peer

model where pipes are

queues and topics (metadata) are service advertisements

n

JXTA

naturally supports local dynamic structure but

n

JMS

is long distance “carrier” between JXTA peers and

(20)

Narada P2P Grid Event Service

n

http://grids.ucs.indiana.edu/ptliupages/projects/narada/

n

I

s a network of event brokers which can

reliably deliver XML

specified

events

with

UDP

or

TCP/IP

n

Using

openJMS

selection module, becomes a distributed or

conventional

Java Message Service

n

Linking selected JXT

rendezvous’s, it also

supports multiple

JXTA communities

n

Think of

JXTA JMS

and

Narada

as differen

bindings for

event/messagin

web service

(21)

e-Science is just a pile of XML

n

Each leaf is a piece of XML either defining a

nugget of

information or a Service

and/or containing links to other XML or

“raw resources”

(22)

XML (RSS) Specification of Information Nuggets

n

<item rdf:about

="http://xml.com/pub/2000/08/09/xslt/xslt.html">

n

<title>

Processing Inclusions with XSLT

</title>

n

<link>

ht

tp://xml.com/pub/2000/08/09/xslt/xslt.html </

link>

n

<description>

n

Processing document inclusions with general XML tools can be

problematic.

</description>

n

</item>

n

<item

rdf:about

="http://xml.com/pub/2000/08/09/rdfdb/index.html">

n

<title>

Putting RDF to Work

</title>

n

<link>

http://x

ml.com/pub/2000/08/09/rdfdb/index.html </link>

n

<description>

n

Tool and API support for the Resource Description

Framework is slowly coming of age.

</description>

n

</item>

n

</rdf:RDF>

Example of XML meta-data in the “pile

pointing to other (outside) resources

Links are essential as much meta-data wil

(23)

Distributed Information

Actually the XML is

distributed

(24)

Tree Structured Information

n

Roughly current organization of Web (Grid)

escience://root/one/two/botto

m

roo

t

one

two

bottom

Note XML specifie

both internal an

external nodes of tree

(25)

Unstructured and Structured XML

n

Peer to Peer natural for unstructured “mess” (local broadcast)

n

Get a Grid of P2P organized communities

escience://root/one/two/mes

s

roo

t

one

two

(26)

Database

Grid

Middleware

Grid

Middleware

Grid

Middlewar

e

Grid

Middlewar

e

MP Group

M

p

M

p

MP=Middleware Peer

(27)

Audio Video Conferencing as a Web Service

n

We can use

openh323

as core of a

web service

for audio

video conferencing

n

We have designed

XGSP

as a common XML

session protocol

including

SIP

and

H323

capabilities

n

We add additional commands to extend web service to

support heterogeneous clients and new fault tolerance and

audio mixing options

n

Will support traditional (H323, SIP) or new generatio

clients with native XML (JXTA)

n

Event Service

must support

UDP

and

Real-time

constraints

n

WSDL

must “

bind

” to

RTP transport

used in A/V field

n

Include

Internet Audio

(NetMeeting, HearMe SIP ….),

(28)

(29)

XML General Session Protocol I

<xs:complexType name="

SessionDes

">

<xs:sequence>

<xs:element name="

SessionName

" type="xs:string"/>

<xs:element name="

SessionID

" type="SessionID"/>

<xs:element name="

SessionCreator

" type="UserURL"/>

<xs:element name="

SessionInfo

" type="xs:string"/>

<xs:element name="

SessionTime

">

<xs:complexType>

<xs:sequence>

<xs:element name="

StartTime

" type="xs:dateTime"/>

<xs:element name="

EndTime

" type="xs:dateTime"/>

</xs:sequence>

</xs:complexType>

</xs:element>

<xs:element name="

SessionURI

" type="xs:anyURI"/>

<xs:element name="

SessionParticipants

" maxOccurs="unbounded">

<xs:complexType>

<xs:complexContent>

<xs:extension base="UserURL"/>

</xs:complexContent>

</xs:complexType>

</xs:element>

<xs:element name="

ContactInfo

" type="UserURL"/>

</xs:sequence>

(30)

XML General Session Protocol II

<xs:complexType>

<xs:sequence>

<xs:element name="MediaCodec" type="xs:string"/>

<xs:element name="CodecParam">

<xs:complexType>

<xs:choice>

<xs:element name="H.261" type="xs:H.261"/>

<xs:element name="H.263" type="xs:H.263"/>

<xs:element name="G.711" type="xs:G.711"/>

<xs:element name="G.722" type="xs:G.722"/>

<xs:element name="G.723" type="xs:G.723"/>

<xs:element name="G.728" type="xs:G.728 "/>

<xs:element name="G.729" type="xs:G.729"/>

<xs:element name="GSM" type="xs:GSM"/>

</xs:choice>

</xs:complexType>

</xs:element>

</xs:sequence>

</xs:complexType>

(31)

A Typical SIP Message

n

REGISTER

sip:registrar.biloxi.co

Via:

SIP/2.0/UDP 10.4.1.4:506

To:

Bob (sip:[email protected]

From:

Bob (sip:[email protected]);tag=45624

Call-ID:

84381763

[email protected]

CSeq: 1826

REGISTE

Contact:

(sip:[email protected]

Expires:

720 Contact-Length:

0 n

Initially build a wrapper that accepts such

(32)

Portals and Web Services

n

Web Services

allow us to build a

component model

(see

CCA) for resources.

n

Each resource

naturally has a

user interface

(which

might be customized for user) specified in

WSRP/WSIA from OASIS

n

Web Service <--> Portlet

n

Natural to use a component model for

portal

building

displayed web page from

collection of portlets

• So can customize each portlet and customize which portlets

you want

n

Apache Jetspeed

seems good open source technology

supporting this model

• JSP model

is better than say a client-side Java integration in

that also message based so this is “Portal as a Web Service”

(33)

WSRP Structure of a Portlet

n

When one has multiple components in the UI one must extend

Web Service picture to include Portal and Portlets

n

WSRP and indeed WSDL details unimportant. Need basic

architecture with message based middleware

Application o

Content

source

WSD

L

Web Service

S

R

W

P

Application as a WS

General Application Port

Interface with other We

Services

User Face o

Web Servic

WSRP Ports define

Portal

User Profil

Aggregat

UI

Fragments

Clien

t

Integrate Multiple Portlets

User Customizatio

at either Portal or if

WSRP i

(34)

Online Knowledge Center built from Portlets

n

Web Services

provide a

component model

for the

middleware (see large “

common component

architecture

” effort in Dept. of Energy)

n

Should match each WSDL component with a

corresponding user interface component

n

Thus one “must use” a

component model for the portal

with again an XML specification (

portalML

) of portal

component

A set of U

Components

(35)

Jetspeed Computing Portal: Choose Portlets

4 available portlet

(36)

Choose Portlet Layout

Choose 1-column Layout

Original 2-column Layout

(37)

(38)

Portal Shell

n

Our Gateway portal supports many methods or Sub

Web Services

• Over 80 is a lot to agree on – need to breal up into Services

with a few parts (methods)

n

Think of as a

Portal Shell

invoking many specific other

services such as

• run myws, ls, cd, cat mwsarchive, visualize ….

n

Both Shell and sub services could have separate UI

components

n

Services like

ls

could be instantaneous or updated as

needed

• If instantaneous, need to archive results as

portlet

lasts a

different length of time from

transient ls

(39)

Application

o

Portal WS

WSD

L

Web Service

S

R

W

P

Portal

Shell

WSD

L

Web Service

S

R

W

P

User Face o

Web Servic

WSRP Ports define

WS as a Portlet

Portal

User Profil

Aggregat

UI

Fragments

Integrate

Multiple Portlets

User Customizatio

at either Portal or if

complicated at WS

Persisten

Content

WS

WSD

L

WSRP

WSD

Archiv

e

or

Transient or “Permanent”

(40)

myWSarchive.wsdl I

n

Please imagine that Microsoft

Word becomes a Web Service

…..

n

We are familiar with a file like

fred.doc

on our PC where .doc indicates the

appropriate handler for file

• fred.doc will be mix of XML (meta)data and perhaps binary attachments

n

Several web services (Word, StarOffice ..) can produce fred.doc

• They all support same

Word.wsdl

interface

n

Even more web services can process (

handle

) fred.doc

• Word, StarOffice, Notepad (XML dump), Copy are

WordWS handlers

• These correspond to different ports specified directly (as XML) or

indirectly (as URI) in fred.doc

WordW

Factory

Fil

I/

WS

Wor

as

WS

Use

I/O

Porta

l

fred.doc

(41)

myWSarchive.wsdl II

n

We build our environments in terms of Web Services which can

be transient as well as persistent Web Service documents

analogous to fred.doc

myWS

Handle

r

Fil

I/

WS

Use

I/O

Porta

l

myWS.wsdl

myWS

(handler

Factory

Use

I/O

Porta

l

Job Porta

Shell

myW

S

Fil

I/

WS

Persisten

myWS.wsdl

Transient for length

Transient for

length of job

myWS.wsdl can b

multiple files with

XML wrappe

(42)

Collaborative Web Services

n

First note that there are two distinct concepts

n

Collaboration as a Web Service

• Such as “Audio-Video Conferencing” as a Web Service or

“Text Chat as a Web Service”

n

Collaborative Web Services

• Here we view a Web Service as specifying a (distributed)

object and wish to share an object

• Object could be a Web page, a Job status form, a scientific

visualization, a PowerPoint slide etc. (not all of these are Web

Services but all should be)

• There is an

overall framework

(part of collaboration as a web

service) specifying such items as members of collaborative

session and their preferences. There is a mechanism to make

Web services collaborative within this framework

(43)

Why Web Services for Collaboration

n

Well

everything is meant to be a Web Service

but also:

n

Web Services are MUCH EASIER to make

collaborative than other objects because

all input and

output is defined by uniform XML messages

• You need to teach your message service about collaboration!

• Note local applications are NOT Web Services – input is

things like “user mouse click” represented by “method

events” (UI program interrupts) not “XML message events”

n

The elegance of collaborative web services suggest that

it could be easiest to make object X (such as

PowerPoint or SVG) collaborative not by traditional

(44)

Key Types of Collaboration

n

Shared Display:

here one shares the rendering of a

Web Service. No modification is needed of the web

service. Rather this is handled by the portal controlling

the user interface

n

Collaborative Replicated Web Services:

here one

replicates several instances of a web service and the

task is to keep these copies consistent. This

synchronizes inputs to multiple Web Services

n

Collaborative Web Service Access:

here one has

multiple clients accessing a single instance of a web

service and obtaining consistent views. This

multicasts

output of a single Web service to multiple clients

(45)

Collaboration: Shared Display

n

Sharing can be done at any point on “object” pipeline

Objec

t

Object

’

Object’

’

Object

Displa

y

Object

Viewer

Object

Displa

y

Object

Displa

y

Even

(Message

Service

Shared Display shares

framebuffer with event

corresponding to change

pixels in master client.

Master

Share

Display

Share

Web Service

Share

Event

Shared

Export

(46)

Collaborative Web Service Access

n

Web Service either supports collaboration directly or uses event service

Objec

t

We

Servic

e

Object

Displa

y

Object

Viewer

Object

Displa

y

Object

Viewer

Even

(Message

Service

Web Service has a por

on which collaborativ

modes set

Web Service can b

“front-end” (in middl

tier) to complex

back-end object

Master

Object

Displa

y

Object

Viewer

Web Service Intercepto

Providing General Services

Set Collaboration Mode

(47)

Web Service Interceptor

n

Collaborative Web Services are implemented “just” by

replicating the messages that are output by the Web service

n

This replication is provided by the event service which needs

both client and service dependent information

• The service specific message function is provided by an interceptor or an

adaptor which takes care of issues like security, collaboration,

management, service information, which message service to use

• The client specific function specifies the client profile telling event service