Presentation

Integration of Portals and

Web Services

PTLIU Laboratory for Community Grids

Geoffrey Fox, Marlon Pierce, Shrideep Pallickara, Choonhan Youn Computer Science, Informatics, Physics

Indiana University, Bloomington IN 4740

http://grids.ucs.indiana.edu/ptliupages/presentations/ncsaexpeditionmar02

[email protected]

What is a Web Service I

n A web service is a computer program running on either the local

or remote machine with a set of well defined interfaces (ports) specified in XML (WSDL)

n In principle, computer program can be in any language

(Fortran .. Java .. Perl .. Python) and the interfaces can be implemented in any way what so ever

• Interfaces can be method calls, Java RMI Messages, CGI Web

invocations, totally compiled away (inlining) but

n The simplest implementations involve XML messages (SOAP)

and programs written in net friendly languages like Java and Python

n Web Services separate the meaning of a port (message) interface

from its implementation

n Enhances/Enables Re-usable component model of ANY

What is a Web Service II

n

Web Services have important implication that

ALL

interfaces are XML messages based.

In contrast

n

Most Windows programs have interfaces defined as

interrupts due to user inputs

n

Most software have interfaces defined as methods which

might be implemented as a message but this is often

NOT explicit

Securit

y Catalog

Paymen Credit

Card

Warehous

What is a Web Service III

n

“Everything electronic” is a

resource

•

Computers; Programs; People

•

Data (from sensors to this presentation to email to

databases)

n

“Everything electronic” is a

distributed object

n

All

resources have interfaces

which are defined in

XML

for

both

properties

(data-structure) and

methods

(service,

function, subroutine) (

Resources

are

Services

)

•

We can assume that a data-structure property has

getproperty()

and

setproperty(value)

methods to act as

interface

n

All resources are linked by

messages

with structure, which

must be specifiable in XML

Classic Grid Architecture

Database Database

Netsolv e

Computin g

Securit y Collaboratio

n

Compositio n

Content Access

Resources

Middle Tie Brokers Service Providers

etc. XML WS to WS Interfaces

(Virtual) XML Knowledge (User) Interface

Clients

(Virtual) XML Data Interface Raw Data

Ra

Resource

s

S WS WS

W S

Render to XML Display Format

(Virtual) XML Rendering

WSDL Abstractions

n

WSDL

abstracts a program

as an entity that does

something given one or more inputs with its results

defined by streams on one or more outputs.

n

Functions are defined by method name and

parameter

methodname(parm1,parm2, … parmN)

•

Where parameters are “Input” “Output” or both

n

In WSDL, we will have a

Web Service

which like a

(Java or CORBA Program) can be thought of as a

(distributed) object with many methods

•

Instead of a function call, the “calling routine” sends an

XML message to the Web Service specifying

methodname

Details of WSDL Protocol Stack

n

UDDI

finds where programs are

•

remote( (distributed) programs

are just Web Services

n

WSFL

links programs togethe

(under revision?)

n

WSDL

defines interface (methods,

parameters, data formats)

n

SOAP

defines structure of message

including serialization of information

n

HTTP

is negotiation/transport

protocol

n

TCP/IP

is layers 3-4 of OSI

n

Physical Network

is layer 1 of OSI

UDDI or WSIL

WSFL

WSDL

SOAP or RMI

HTTP or SMTP or IIOP or RMTP

TCP/IP

Peer to Peer Grid

Database Database

JXTA

JXTA

Web Service Interfaces

Web Service Interfaces

Event Messag Brokers

Integrate P2P and Grid/WS

Examples of Web Services I

n

OGSA (Open Grid Service Architecture)

• Integrate Web Service and Grid Concepts and allows Globus

to be implemented as Web Services

n

Audio-Video Conferencing

as a Web Service

• Integrates H323, SIP, JXTA (etc.) protocols by mapping to

single XML Interface

• Provides VRVS reflector model from Messaging Web Service

n

Messaging or Event Web Service

provides intelligent

routing and buffering of messages

n

Performance

as Web Service (yesterday)

• Allow access to distributed monitoring data, analysis, models,

Examples of Web Services II

n Computing as a Web service

• More on this today – job submittal, status, composition, data services,

visualization etc.

n Education as a Web Service

• One of easiest to do as object standards well defined (IMS) and little

performance issues

• Grading, Homework submission, registration, assessment etc.

n Universal Access and Web Services

• As Web Services allow multiple implementation of a particular interface,

one can adjust to needs of particular clients (PDA v. versus, impaired sight etc.)

• Can build custom implementations of certain web services for particular

communities but re-use others

n Collaborative Web Services

Portals and Web Services

n

Web Services

allow us to build a

component model

(see

CCA) for resources.

n

Each resource

naturally has a

user interface

(which

might be customized for user)

n

Web Service <--> Portlet

n

Natural to use a component model for portal building

displayed web page from collection of portlets

• So can customize each portlet and customize which portlets

you want

n

Apache Jetspeed

seems good open source technology

supporting this model

• JSP model is better than say a client-side Java integration in

Component –based Portal

n

Web Services

provide a

component model

for the

middleware (see large “common component

architecture” effort in Dept. of Energy)

n

Should match each WSDL component with a

corresponding user interface component

n

Thus one “must use” a

component model for the portal

Meta-Data and Web Services

n

Enriching resources with meta-data is critical idea

• Enables one to identify and link resources around the globe • Allows one to find out “meaning” of a Web service not just

syntax of interface

n

Semantic Grid

implies linkage of Grid/Web services

enabled by meta-data leading to “digital brilliance”

phase transition

n

We can experiment with Semantic Web techniques for

specifying meta-data

RDF DAML OIL

n

These encompass both straightforward enriched data

RDF in a Nutshell

n The essence of RDF is a statement which has a subject and

predicate and an object

n The “subject” has the property “predicate” with value “object” n The Web Page has the title “RDF Model and Syntax”

n Mary had a husband called John

n Statements can specify metadata but more generally any facts are

expressible in this way

n One can associate multiple objects with a subject and so build up

a general node of a tree with parent as subject and objects as children.

n DAML and OIL extend RDF with more powerful relations and

ontologies

An Example of RDF and Dublin Core

n <rdf:RDF xmlns:rdf=“http://www.w3.org/1999/02/22-rdf-syntax-ns#”

xmlns:dc=“http://purl.org/metadata/dublin_core#”>

n <rdf:Description about=“http://www.dlib.org”>

n <dc:Title>D-Lib Program - Research in Digital Libraries</dc:Title>

<dc:Description>The D-Lib program supports the community of people with research interests in digital libraries and electronic publishing.

</dc:Description

<dc:Publisher>Corporation For National Research Initiatives</dc:Publisher>

<dc:Date>1995-01-07</dc:Date>

n <dc:Subject>

• <rdf:Bag> <rdf:li> Research; statistical methods</rdf:li>

<rdf:li> Education, research, related topics</rdf:li <rdf:li> Library use Studies </rdf:li> </rdf:Bag>

n </dc:Subject> <dc:Type>World Wide Web Home Page</dc:Type> n <dc:Format>text/html</dc:Format>

What could one do?

n

Get applications together and decide on set of

common

web services

• Service and methods within services

• At “Grid Computing” and “Application” level

n

Implement generalized

meta-data service

at

application level – generate, access

• Role of RDF DAML OIL IMS etc.

n

Define both middle tier (WS) and user interface

(portal) components

n

Agree on

pilot projects

demonstrating both

interoperable

WS/Portal components and

complete

applications

built from WS/Portal components

Collaborative Web Services

n

First note that there are two distinct concepts

Collaboration as a Web Service

•

Such as “Audio-Video Conferencing” as a Web

Service or “Text Chat as a Web Service”

n

Collaborative Web Services

•

Here we view a Web Service as specifying a

(distributed) object and wish to share an object

•

Object could be a Web page, a Job status form, a

Why Web Services for Collaboration

n

Well

everything is meant to be a Web Service

but also:

Web Services are MUCH EASIER to make

collaborative than other objects because

all input and

output is defined by uniform XML messages

• You need to teach your message service about collaboration! • Note local applications are NOT Web Services – input is

things like “user mouse click” represented by “method

events” (UI program interrupts) not “XML message events”

n

The elegance of collaborative web services suggest that

it could be easiest to make object X (such as

CollabWS: Collaboration as a Web Service

n

This includes Audio-Video conferencing, Instant

Messengers, shared applications and tools

n

There are basic standards

H323

and

SIP

n

There are peer to peer collaboration approaches like

JXTA

from Sun

n

There are various commercial products –

Groove

Networks

effectively defines some shared application

interfaces

n

CollabWS

defines a superset of collaboration

properties and subservices in XML/WSDL

n

Then you provide several bindings of

collabws.wsdl

SI

H323

JXTA

A Typical SIP Message

n

REGISTER

sip:registrar.biloxi.co

Via:

SIP/2.0/UDP 10.4.1.4:506

To:

Bob (sip:[email protected]

From:

Bob (sip:[email protected]);tag=45624

Call-ID:

[email protected]

CSeq: 1826

REGISTE

Contact:

(sip:[email protected]

Expires:

720

Contact-Length:

0

n

Initially build a wrapper that accepts such

SIP as XML

n

<sipasxml>

<register>

sip:registrar.biloxi.com

</register>

<via>

SIP/2.0/UDP 10.4.1.4:5060

</via>

<to>

Bob (sip:[email protected])

</to

<from tag="

456248

" >

Bob (sip:[email protected])

</from

<call-id>

[email protected]

</call-id

<cseq>

1826 REGISTER

</cseq

<contact>

(sip:[email protected])

</contact

<expires>

7200

</expires

<contact-length>

0

</contact-length

</sipasxml>

n

So this is either an internal format to which we map SIP

Messaging/Events as a Web Service

n

We can implement

messaging subsystem

(between

WSDL resources) with either direct messages or by a

queued system where you publish messages to queues

and subscribe as receiver to particular queues

n

There are many different publish/subscribe models

• JMS is a cluster of central servers

• JXTA is a very dynamic Peer to Peer model where pipes are

queues and topics (metadata) are service advertisements

n

Implement

JMS API

with

JXTA

protocol – different

WSDL bindings here have different fault

tolerance/reliability semantics

• Could use JMS as long distance “carrier” between JXTA peers • JXTA provides higher performance than JMS for nearby

Event Web Service

n

Filter is mapping to PDA or slow communication

channel (universal access) – see our PDA adaptor

n

Workflow natural as all messages “intercepted” by

Event Web Service

n

Routing illustrated by JXTA

n

Destination-Source matching illustrated by JMS

Web

Service 1 (VirtualQueue Service 2Web

Destinatio

Source Matching Filter

Routin

g workflow

WSD

Broker Network

Data base

Reso urce

Broker

Broker Broker

Broker

Broker

Software multicast

Narada JXTA/JMS Integration

n Is a network of event brokers which can reliably deliver XML

specified events

n Using openJMS selection module, becomes a distributed or

conventional Java Message Service

n Linking special JXT

proxies, it can link

JXTA communities

n Think of JXTA JMS an

Narada as differen

bindings t

Education as a Web Service

n Can link to Science as a Web Service and substitute educational

modules

n “Learning Object” XML standards already exist from IMS/ADL

http://www.adlnet.org – need to update architecture

n Web Services for virtual university include:

n Registration

n Performance (grading)

n Authoring of Curriculum

n Online laboratories for real and virtual instruments

n Homework submission

n Quizzes of various types (multiple choice, random parameters)

n Assessment data access and analysis

n Synchronous Delivery of Curricula

n Scheduling of courses and mentoring sessions

n Asynchronous access, data-mining and knowledge discovery

CMS (Content Management)

as a Web Service

n

For

meta-data, build

federated database as a Web

Service

• Storage, update, lookup of objects

• Registration of different control engines

n

For

content, if W3C DOM and Java API as for SVG

and “decent” XHTML browsers, then can

wrap DOM

as a web service

for all interface functions

• Can of course do this for IE5 but document object model

unreliable

n

Build collaboration systems as

sharing Web Service

Integration of XML/Web Services

into an Enterprise I

n You are the CIO of a little known state’s (call it Xanadu)

education system and been tasked to position it as a 2005 leader in technology enhanced education

n Your initial review reveals a multitude of incompatible existing

information systems distributed over Xanadu’s 50 campuses

n Your cradle to grave community education mandate (which got

the Governor elected) implies integration of all aspects of education in your information system

n You note that performance issues are much less important in

Enterprise Web Services II

n We start a requirements analysis and a catalog of existing “raw

resources”. This will lead to the clean internal XML(WSDL) interfaces XIF and to the external XML interfaces (XEF) to the raw resources

n Xanadu Internal Framework (XIF) built on top of XML

Messaging, Security etc. Services

n We find that The Grid Forum community has defined computing

interfaces which we will adopt to support infrastructure needed

for use of simulations and data repositories in curriculum

n We find that Microsoft, the Liberty Alliance and the Grid have

Security schemes but are relieved that with a little bit of XSLT (transformations) we can map between them and can define

authentication & access control as a Web Service

n We join the IMS consortium and adopt their standards for

learning meta-data and user related information

n We adopt the AICC and ADL SCORM standards for “learning

Enterprise Web Services III

RDF/DAML-OIL to be able to capture educational goals

n We expect commercial standards compliant solutions will be

available for most other needed Educational web services

n We find that we must interface with a dreadful collection of

external raw resources coming from legacy PeopleSoft SAP Oracle and Microsoft products

• As our new model is object (XML) and open standards based we build a

separate XII (Xanadu Information Infrastructure) supporting XIF objects

n We analyze structure of legacy systems and their relation to XIF

and define the external framework XEF which captures much of the raw resources

• This XEF Schema is extensible to capture quirks of each legacy system • We build a set of Web services that convert XEF into XIF

n We inform vendors that we will only install XIF compliant

7 Primitives in WSDL

n types: which provides data type definitions used to describe the

messages exchanged.

n message: which represents an abstract definition of the data being

transmitted. A message consists of logical parts, each of which is associated with a definition within some type system.

n operation– an abstract description of an action supported by the

service.

n portType: which is a set of abstract operations. Each operation

refers to an input message and output messages.

n binding: which specifies concrete protocol and data format

specifications for the operations and messages defined by a particular portType.

n port: which specifies an address for a binding, thus defining a

single communication endpoint.

Discussion of 7 WSDL Primitives

n message specifies collections of types and is equivalent to set of

arguments in a method call. Note that it is an “abstract method” in Java terminology

n operation is a a collection of input output and fault messages; there are

4 types of operation one-way(service just receives a message), request-response(RPC), solicit-response, notification (services pushes out a message)

n portType represents a single channel that can support multiple

operations. It is “abstract” as specified as a set of operations. It is equivalent to a “interface or abstract class” in Java

n binding tells you transport and message format for a porttype (which

can have multiple bindings to reflect say performance-portability trades)

n port combines a binding and an endpoint network address (URL) and is

like a “class instance”

n service consists of multiple ports and is equivalent to a “program” in

Gateway and Web Services

n

We will use the Gateway Computing Portal as an

example (http://www.gatewayportal.org)

• It is largely built using CORBA with a Java Server Pages

front end

• http://community.ucs.indiana.edu:8004/GCWS/BatchScriptGen/Main.jsp

n

Several capabilities have been interfaced using WSDL

• Job Submission (11 Methods including execute local and

remote command, copy files etc. as well as Submit Job)

• Manage WebFlow Session (67 Methods)

• Generate Batch Script (just 1 method but two

implementations developed – one at SDSC and one at Indiana – with UDDI to manage)

Gateway Architecture

n As needed DoD approved Kerberos/SecureID security,

does not usually link to standard Grid technology (Globus)

backen d

Globu

or submit to loca queue system

Corba middle tie SECIOP wit Java modules JSP SSL backen d SOAP wrappe JSP Servlet CORBA WSDL Will replac historical CORBA By EJB Add Casto Java to XML

SOAP and Gateway Portal I

n

Having specified service in WSDL, the run-time is

implemented in SOAP which is “just” an XML header

(info needed by transport – empty here) and body

n

Here is SOAP transported by HTTP message

n

This is

execLocalCommand

WSDL operation to run

one particular command (ls) on current WebFlow

directory

Specify ls as

Argument of operation

HTTP Header

n

And this is the result of

ls

sent back to client in

SOAP

over

HTTP

SOAP and Gateway

Portal II

HTTP Header

SOA

WSDL Message Example

<message name="

submitRequest

">

<part name="

xmljob

" type="

xsd:string

"/>

</message>

<message name="

submitResponse

">

<part name="

response

" type="

xsd:string

"/>

</message>

For the batch script service, we pass the XML description of the job as a string and get back the script as a string. In general, any XML primitive or complex types can be used in messages.

WSDL portTypes Example

<portType name="

BatchScriptServicePortType

">

<operation name="batchGen">

<output message="

tns:submitResponse

"

name="

submitResponse

"/>

<input message="

tns:submitRequest

"

name="

submitRequest

"/>

</operation>

</portType>

WSDL SOAP Binding Example

<binding name="BatchBinding" type="tns:BatchScriptServicePortType">

<soap:binding style="rpc"

transport="http://schemas.xmlsoap.org/soap/http"/> <operation name="batchGen">

<soap:operation soapAction=""/> <input>

<soap:body use="encoded“ namespace="urn:BatchScriptService" encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"/>

</input> <output>

<soap:body use="encoded" namespace="urn:BatchScriptService“ encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"/>

</output>

</operation>

WSDL Ports and Services

<service name="BatchScriptService">

<documentation>

BS stands for Batch Script

</documentation>

<port binding="

BatchBinding

”

name="

BatchPort

">

<soap:address location=

"

http://yourserver/soap/servlet/rpcrouter/

"/>

</port>

</service>

</definitions>

What is a Virtual XML Interface

implement in XML.

n Example 1:We aren’t likely to change syntax of mai

Reply-to:Geoffrey Fox <[email protected]

To: Geoffrey Fox <[email protected] Subject: A Test for Tutoria

A simple mail messag

Geoffrey Fox [email protected] FAX 812856797 Phones Cell 315-254-6387 Home 812323919

n But we could specify and indeed store in XML with transport

done using conventional SMTP.

n So conventional mail is easy to give a virtual XML specification

for wit

name:value becomin

Mail in XML

n

<mailasxml

<smtpheaders

<reply-to email=“[email protected]” >Geoffrey Fox</reply-to>

<to email=“[email protected]” >Geoffrey Fox</to>

<subject>A Test for Tutorial</subject

</smtpheaders <smtpbody>

<message whitespace="collapse"> A simple mail

message</message>

<signature personuri=“ssn://123/45/6789” whitespace=“preserve” >

Geoffrey Fox [email protected] FAX 812856797

Phones Cell 315-254-6387 Home 812323919 </signature>

</smtpbody

</mailasxml>

Virtual XML for MPI I

MPI_SEND(buf, count, datatype, dest, tag, comm)

[ IN buf] initial address of send buffer (choice)

[ IN count] number of elements in send buffer (nonnegative integer)

[ IN datatype] datatype of each send buffer element (handle)

[ IN dest] rank of destination (integer) [ IN tag] message tag (integer)

[ IN comm] communicator (handle)

n

Now our SPMD code is a Web service running in some

large scale parallel machine – perhaps integrated with

an Apache server running on node

• Need to support distributed and shared memory

n

We want to define all interfaces in WSDL – at least all

Virtual XML for MPI II

n datatype becomes an XML Schema specifying structure of

transmitted data – could either be a URI for Schema or its realization

• There would be a built-in schema suitable for handling standard

Fortran/C datatypes

n buf is a handle to an XML instance of Schema datatype

n count is still an integer (xs:integer where xs is Schema

namespace xmlns:xs=http://www.w3.org/2001/XMLSchema)

• For some bindings count is implied by buf

n dest becomes a URI with special notation such as

legacympi://procnum/local/nnn for processor numbers

n tag becomes an instance of (new) mpitag Schema

Virtual XML for MPI III

n

So one defines

mpi.wsdl

with all 100 or so methods

defined as (abstract) operations

• We will define several bindings including the classic HPCC

method calls for Fortran C C++ and Java with either distributed memory or shared memory mode

• For C++ and Java we would implements bindings for either

“object” or “packed consecutive byte” interpretation of buf

n Note one can use both static and run-time compilation to map

XML specified services into particular implementations

n MPI_INIT() might be implemented with JXTA (peer-to-peer

advertisments) to find nodes; the communicator might be transmitted by SOAP while buf transmission uses MPICH

n

This brings Web Services to MPI but also MPI to web