Application Web Services and Event / Messaging Systems

(1)

Application Web Services and

Event / Messaging Systems

PTLIU Laboratory for Community Grids

Geoffrey Fox, Shrideep Pallickara, Marlon Pierce Computer Science, Informatics, Physics

Indiana University, Bloomington IN 4740

http:

//www.naradabrokering.org

http://grids.ucs.indiana.edu/ptliupages

gcf@indiana.edu

(2)

Application Portal in a Minute (box)

n

Systems like

Unicore

,

GPDK

,

Gridport

(

HotPage

),

Gateway, Legion

provide “

Grid or GCE Shell

”

interfaces to users (

user portals

)

•

Run a job; find its status; manipulate files

•

Basic UNIX Shell-like capabilities

n

Application Portals

(Problem Solving Environments)

are often built on top of “Shell Portals” but this can be

quite time confusing

•

Application Portal

=

Shell Portal Web Service

+

Application (factory) Web service

(3)

Application Web service

n Application Web Service is ONLY metadata

• Application is NOT touched

n Application Web service defined by two sets of schema:

• First set defines the abstract state of the application

n What are my options for invoking myapp? n Dub these to be “abstract descriptors”

• Second set defines a specific instance of the application

n I want to use myapp with input1.dat on

solar.uits.indiana.edu.

n Dub these to be “instance descriptors”.

n Each descriptor group consists of

• Application descriptor schema

• Host (resource) descriptor schema

(4)

(5)

Engineering Application WS

n

Schema wizard:

given a Schema, creates (JSP) web

page with form to specify XML Instances

• Use for application metadata

n

AntiSchema wizard:

given an HTML form, creates a

Schema

• Captures input parameters of application n

Castor

converts Schema to Java

• Use Python if you prefer!

n

Apache

converts Java into Web Services

n

Make this in a

portlet

for use in favorite portal

n

Being used today

in

DoD

…….. (with and without

(6)

(7)

(8)

(9)

Different Web Service Organizations

n

Everything is a

resource implemented as a Web

Service

, whether it be:

•

back end supercomputers and a petabyte data

•

Microsoft PowerPoint and this file

n

Web Services

communicate by

messages

…..

n

Grids

and

Peer to Peer (P2P) networks

can be

integrated by building both in terms of

Web Services

with different (or in fact sometimes the same)

implementations of

core services

such as

registration

,

discovery

,

life-cycle

,

collaboration

and

event or

message transport

…..

• Gives a Peer-to-Peer Grid

n

Here we discuss

Event

or

Message Service

linking web

(10)

Peer to Peer Grid

Database Database

Peers Peers

Peer to Peer Grid

Resource Facin

Web Service Interfaces

Event Messag Brokers

Integrate P2P and Grid/WS

Event Messag Brokers

A democratic organization

User Facin

Web Service Interfaces

(11)

Message Or Event Based Inte

Connection

Reso urce

Data base

Reso urce Sof

ware Sof ware

XM Skin

e-Science/Grid/P2P Networks are XML

Specified Resources connected by XML

specified message

Implementation of resource and connection may or may not be XML

(12)

Role of Event/Message Brokers

n We will use events and messages interchangeably

• An event is a time stamped message

n Our systems are built from clients, servers and “event brokers”

• These are logical functions – a given computer can have one

or more of these functions

• In P2P networks, computers typically multifunction; in Grids

one tends to have separate function computers

• Event Brokers “just” provide message/event services; servers

provide traditional distributed object services as Web services

n There are functionalities that only depend on event itself and

perhaps the data format; they do not depend on details of application and can be shared among several applications

• NaradaBrokering is designed to provide these functionalities • MPI provided such functionalities for all parallel computing

(13)

NaradaBrokering implements an

Event Web Service

n Filter is mapping to PDA or slow communication channel

(universal access) – see our PDA adaptor

n Workflow implements message process n Routing illustrated by JXTA

n Destination-Source matching illustrated by JMS using

Publish-Subscribe mechanism

Web

Service 1 (VirtualQueue Service 2Web Destinatio

Source Matching Filter Routin

g workflow

WSD

(14)

Engineering Issues Addresse

by Event / Messaging Service

n

Application level

Quality of Service

– give audio highest

priority

n

Tunnel through

firewalls

n

Filter messages

to slow (collaborative or real time)

clients

n

Hardware multicast

is erratically implemented (Event

service can dynamically use software multicast)

n

Scaling

of

software multicast

n

Elegant implementation of

Collaboration

in a Groove

Networks (done better) style

n

Integrate

synchronous

and

asynchronous

collaboration

(15)

Features of Event Service I

n MPI nowadays aims at a microsecond latency

n The Event Web Service aims at a millisecond latency

• Typical distributed system travel times are many milliseconds

(to seconds for Geosynchronous satellites)

• Different performance/functionality trade-off

n Messages are not sent directly from P to S but rather from P to

Broker B and from Broker B to subscriber S

• Actually a network of brokers

n Synchronous systems: B acts as a real-time router/filterer

• Messages can be archived and software multicast

n Asynchronous systems: B acts as an XML database and

workflow engine

n Subscription is in each case, roughly equivalent to a database

(16)

Features of Event Web Service II

n

In principle Message brokering can be virtual and

compiled away

in the same way that WSDL ports can

be bound in real time to optimal transport mechanism

• All Web Services are specified in XML but can be

implemented quite differently

• Audio Video Conferencing sessions could be negotiated using

SOAP (raw XML) messages and agree to use certain video codecs transmitted by UDP/RTP

n

There is a collection of XML Schema – call it GXOS –

specifying

event service

and requirements of message

streams and their endpoints

• One can sometimes compile message streams specified in

GXOS to MPI or to local method call

n

Event Service must support dynamic heterogeneous

protocols

(17)

Features of Event Web Service III

n

The event web service is naturally implemented as a

dynamic distributed network

• Required for fault tolerance and performance n

A new

classroom joins

my online lecture

• A broker is created to handle students – multicast locally my

messages to classroom; handle with high performance local messages between students

n

Company X sets up a

firewall

• The event service sets up brokers either side of firewall to

optimize transport through the firewall

n

Note

all message based applications

use

same message

service

• Web services imply ALL applications are (possibly virtual)

(18)

Single Server P2P Illusion

Collaboration Server Data

base

Traditional Collaboration Architectur e.g. commercial WebEx

(19)

Narada Broker Network

Data base Reso

urce

Broker

Broker Broker Broker

Broker

Software multicast (P2P) Community

(P2P) Community

For message/events service (P2P) Community

(20)

NaradaBrokering and JMS (Java Message Service)

Low Rate; Small

Messages (commercial_JMS)

(21)

JXTA just got slower

Pure Narada 2 hops

Client

Client Narada

Narada

Client  JXTA  JXTA  Client

Client  JXTA  Narada  JXTA  Client

(22)

W Displa y W Viewe r WS Displa y WS Viewe r Even (Message Service Master W Displa y WS Viewe r

Collaboration as a W Set up Session

We Servic e F I U O F I R O

Shared Input Port (Replicated WS) Collaboration

(23)

W Displa y W Viewe r WS Displa y WS Viewe r Even (Message Service Master W Displa y WS Viewe r Web Service Message