Architecture o
Web Service Grids
Beijing
Chinese Academy of Sciences
August 28 2004
Geoffrey Fox
Community Grids Lab
Indiana University
Philosophy of Web Service Grids
•
Much of Distributed Computing was built by natural
extensions of computing models developed for sequential
machines
•
This leads to the
distributed object
(DO) model represented
by Java and
CORBA
– RPC (Remote Procedure Call) or RMI (Remote Method Invocation) for Java
•
Key people think this is not a good idea as it scales badly
and ties distributed entities together too tightly
– Distributed Objects Replaced by Services
•
Note
CORBA
was considered too complicated in both
organization and proposed infrastructure
– and Java was considered as “tightly coupled to Sun” – So there were other reasons to discard
Web services
•
Web Services
build
loosely-coupled,
distributed applications,
based on the
SOA
principles.
•
Web Services interact
by exchanging
messages in
SOAP
format
•
The contracts for the
message exchanges that
implement those
interactions are
described via
WSDL
What is a Grid?
•
You won’t find a clear description of what is Grid and how
does
differ
from a
collection of Web Services
– I see no essential reason that Grid Services have different requirements than Web Services
– Geoffrey Fox, David Walker, e-Science Gap Analysis, June 30 2003. Report UKeS-2003-01,
http://www.nesc.ac.uk/technical_papers/UKeS-2003-01/index.html.
– Notice “service-building model” is like programming language – very personal!
•
Grids were once defined as “
Internet Scale Distributed
Computing
” but this isn’t good as Grids depend as much if
not more on data as well as simulations
e-Infrastructure
e-Infrastructure builds on the inevitable increasing performance
of networks and computers linking them together to support new flexible linkages between computers, data systems and people
• Grids and peer-to-peer networks are the technologies that build e-Infrastructure
• e-Infrastructure called CyberInfrastructure in USA
We imagine a sea of conventional local or global connections
supported by the “ordinary Internet”
• Phones, web page accesses, plane trips, hallway conversations
• Conventional Internet technology manages billions of
broadcast or low (one client to Server) or broadcast links
On this we superimpose high value multi-way organizations
(linkages) supported by Grids with optimized resources and system support
• Low multiplicity fully interactive real-time sessions
N plus N Community Resources
Grid Community databases have analogy to Television and the
News Web that allow individuals to communicate instantly with each other via Web Pages and Headline News acting as proxies
Large and Small Grids
N resources in a community (N is billions for the world and
1000-10000 for many scientific fields)
Communities are arranged hierarchically with real work being
done in “groups” of M resources – M could be 10-100 in e-Science
Metcalfe’s law: value of network grows like square of number of
nodes M – we call Grids where this true Metcalfe or M2 Grids
Nature of Interaction depends on size of M or N
• N plus N Shared Information Grids for largish N
• M2 Metcalfe Grids for smaller M < N
Technology support depends on M/N – might use a relatively
static DHT (Distributed Hash Table) for large N and a distributed shared memory for small M
Grids must merge with peer-to-peer networks to support both N
M
2
Interactions
• Superimpose M way “Grids” on the sea
(heatbath) of “2 by N” or N plus N
“ordinary” interactions
Implement Grid as a softwar
Information Gri
Enterprise Gri
Compute Grid
Campus Grid R2 R1
Teacher
Students
Dynamic light-weight Peer-to-peer
Collaboration Training Grid
Architecture of (Web Service) Grids
Grids built from
Web Services
communicating through
an overlay network built in SOFTWARE on the
“ordinary internet” at the application level
Grids provide the
special quality of service
(security,
performance, fault-tolerance) and customized services
needed for “distributed complex enterprises”
We need to work with Web Service community as they
debate the 60 or so proposed Web Service specifications
• Use Web Service Interoperability WS-I as “best practice”
• Must add further specifications to support high performance
• Database “Grid Services” for N plus N case
Web Services
• Java is very powerful partly due to its many “frameworks” that generalize libraries e.g.
– Java Media Framework
– Java Database Connectivity JDBC
• Web Services have a correspondingly collections of specifications that represent critical features of the distributed operating systems for “Grids of Simple Services”
– Some 60 active WS-* specifications for areas such as – a. Core Infrastructure Specifications
– b. Service Discovery
– c. Security
– d. Messaging
– e. Notification
– f. Workflow and Coordination
– g. Characteristics
– h. Metadata and State
A List of Web Services I
• a) Core Service Architecture
• XSD XML Schema (W3C Recommendation) V1.0 February 1998, V1.1 February 2004
• WSDL 1.1 Web Services Description Language Version 1.1, (W3C note) March 2001
• WSDL 2.0 Web Services Description Language Version 2.0, (W3C under development) March 2004
• SOAP 1.1 (W3C Note) V1.1 Note May 2000
• SOAP 1.2 (W3C Recommendation) June 24 2003
• b) Service Discovery
• UDDI (Broadly Supported OASIS Standard) V3 August 2003
• WS-Discovery Web services Dynamic Discovery (Microsoft, BEA, Intel …) February 2004
• WS-IL Web Services Inspection Language, (IBM, Microsoft)
A List of Web Services II
• c) Security
• SAML Security Assertion Markup Language (OASIS) V1.1 May 2004
• XACML eXtensible Access Control Markup Language (OASIS) V1.0 February 2003
• WS-Security 2004 Web Services Security: SOAP Message Security (OASIS) Standard March 2004
• WS-SecurityPolicy Web Services Security Policy (IBM, Microsoft, RSA, Verisign) Draft December 2002
• WS-Trust Web Services Trust Language (BEA, IBM, Microsoft, RSA, Verisign …) May 2004
• WS-SecureConversation Web Services Secure Conversation Language (BEA, IBM, Microsoft, RSA, Verisign …) May 2004
A List of Web Services III
• d) Messaging
• WS-Addressing Web Services Addressing (BEA, IBM, Microsoft)
March 2004
• WS-MessageDelivery Web Services Message Delivery (W3C Submission by Oracle, Sun ..) April 2004
• WS-Routing Web Services Routing Protocol (Microsoft) October 2001
• WS-RM Web Services Reliable Messaging (BEA, IBM, Microsoft, Tibco) v0.992 March 2004
• WS-Reliability Web Services Reliable Messaging (OASIS Web Services Reliable Messaging TC) March 2004
• SOAP MOTM SOAP Message Transmission Optimization Mechanism (W3C) June 2004
• e) Notification
• WS-Eventing Web Services Eventing (BEA, Microsoft, TIBCO)
January 2004
• WS-Notification Framework for Web Services Notification with WS-Topics, WS-BaseNotification, and WS-BrokeredNotification (OASIS) OASIS Web Services Notification TC Set up March 2004
A List of Web Services IV
• f) Coordination and Workflow, Transactions and Contextualization
• WS-CAF Web Services Composite Application Framework including WS-CTX,
WS-CF and WS-TXM below (OASIS Web Services Composite Application Framework TC) July 2003
• WS-CTX Web Services Context (OASIS Web Services Composite Application Framework TC) V1.0 July 2003
• WS-CF Web Services Coordination Framework (OASIS Web Services Composite Application Framework TC) V1.0 July 2003
• WS-TXM Web Services Transaction Management (OASIS Web Services Composite Application Framework TC) V1.0 July 2003
• WS-Coordination Web Services Coordination (BEA, IBM, Microsoft) September 2003
• WS-AtomicTransaction Web Services Atomic Transaction (BEA, IBM, Microsoft) September 2003
• WS-BusinessActivity Web Services Business Activity Framework (BEA, IBM, Microsoft) January 2004
• BTP Business Transaction Protocol (OASIS) May 2002 with V1.0.9.1 May 2004
• BPEL Business Process Execution Language for Web Services (OASIS) V1.1 May 2003
• WS-Choreography (W3C) V1.0 Working Draft April 2004
• WSCI (W3C) Web Service Choreography Interface V1.0 (W3C Note from BEA, Intalio, SAP, Sun, Yahoo)
A List of Web Services V
• h) Metadata and State
• RDF Resource Description Framework (W3C) Set of recommendations expanded from original February 1999 standard
• DAML+OIL combining DAML (Darpa Agent Markup Language) and OIL (Ontology Inference Layer) (W3C) Note December 2001
• OWL Web Ontology Language (W3C) Recommendation February 2004
• WS-DistributedManagement Web Services Distributed Management Framework with MUWS and MOWS below (OASIS)
• WSDM-MUWS Web Services Distributed Management: Management Using Web Services (OASIS) V0.5 Committee Draft April 2004
• WSDM-MOWS Web Services Distributed Management: Management of Web Services (OASIS) V0.5 Committee Draft April 2004
• WS-MetadataExchange Web Services Metadata Exchange (BEA,IBM, Microsoft, SAP) March 2004
• WS-RF Web Services Resource Framework including WS-ResourceProperties,
WS-ResourceLifetime, WS-RenewableReferences, WS-ServiceGroup, and
WS-BaseFaults (OASIS) Oasis TC set up April 2004 and V1.1 Framework March 2004
• ASAP Asynchronous Service Access Protocol (OASIS) with V1.0 working draft G June 2004
A List of Web Services VI
•
g) General Service Characteristics
•
WS-Policy
Web Services Policy Framework (BEA, IBM,
Microsoft, SAP)
May 2003
•
WS-PolicyAssertions
Web Services Policy Assertions
Language (BEA, IBM, Microsoft, SAP)
May 2003
•
WS-Agreement
Web Services Agreement Specification
(GGF under development)
May
2004
•
i) User Interfaces
•
WSRP
Web Services for Remote Portlets (OASIS)
OASIS Standard
August 2003
Importance of SOAP
• SOAP defines a very obvious message structure
with a
header
and a
body
• The
header
contains information used by the
“
Internet operating system
”
–
Destination, Source, Routing, Context, Sequence
Number …
• The
message body
is only used by the
application
and will never be looked at by “operating system”
except to encrypt, compress it etc.
• Much discussion in field revolves around
what is in
header!
WS-I Interoperability
•
Critical underpinning of Grids and Web Services is the
gradually growing set of specifications in the Web Service
Interoperability Profiles
•
Web Services Interoperability
(WS-I) Interoperability
Profile 1.0a." h
ttp://www.ws-i.org.
gives us
XSD,
WSDL1.1, SOAP1.1, UDDI
in basic profile and parts of
WS-Security
in their first security profile.
•
We imagine the “60 Specifications” being checked out
and evolved in the
cauldron of the real world
and
occasionally best practice identifies a new specification to
be added to
WS-I
which
gradually increases in scope
Web Services Grids and WS-I+
• WS-I Interoperability doesn’t cover all the capabilities need to support Grids
• WS-I+ is designed to minimal extension of WS-I to support “most current” Grids: it adds support for
– Enhanced SOAP Addressing (WS-Addressing) – Fault tolerant (reliable) messaging
– Workflow as in IBM-Microsoft standard BPEL
• Security and Notification best practice and support will probably get added soon
– There are Web Service frameworks here but various IBM v Microsoft v Globus differences to be resolved
• Portlet-based User Interfaces could be added
• UK OMII Open Middleware Infrastructure Institute is adopting this approach to support UK e-Science program
– Currently UK e-Science largely either uses GT2 (as in EDG) or Simple Web Services for “database Grids”
Bit
level
Internet
(OSI
Stack)
Layered Architecture for Web Services and Grids
Base Hosting Environment
Protocol HTTP FTP DNS …
Presentation XDR …
Session SSH …
Transport TCP UDP …
Network IP …
Data Link / Physical
Servic Internet
Application Specific Grids
Generally Useful Services and Grids
Workflow WSFL/BPEL
Service Management (“Context etc.”)
Service Discovery (UDDI) / Information
Service Internet Transport
Protocol
Service Interfaces WSDL
Servic Context
Higher Level
Working up from the Bottom
We have the classic (CISCO, Juniper ….) Internet routing the
flood of ordinary packets in OSI stack architecture
Web Services build the “Service Internet” or IOI (Internet on
Internet) with
• Routing via WS-Addressing not IP header
• Fault Tolerance (WS-RM not TCP)
• Security (WS-Security/SecureConversation not IPSec/SSL)
• Information Services (UDDI/WS-Context not DNS/Configuration files)
• At message/web service level and not packet/IP address level
Software-based Service Internet possible as computers “fast” Familiar from Peer-to-peer networks and built as a software
overlay network defining Grid (analogy is VPN)
SOAP Header contains all information needed for the “Service
Consequences of Rule of the Millisecond
•
Useful to remember
critical time scales
–
1)
0.000001 ms
– CPU does a calculation
–
2a)
0.001 to 0.01 ms
– Parallel Computing MPI latency
–
2b)
0.001 to 0.01 ms
– Overhead of a Method Call
–
3)
1 ms
– wake-up a thread or process
–
4)
10 to 1000 ms
– Internet delay
•
2a), 4) implies geographically distributed
metacomputing
can’t in general compete with parallel systems
•
3) << 4) implies a software overlay network is possible
without significant overhead
–
We need to explain why it adds value of course!
•
2b) versus 3) and 4) describes regions where
method
and
Linking Modules
n
From method based to RPC to message based to event-based
publish-subscribe Message Oriented Middleware
Module A Module
B
Method Call .001 to 1 millisecond
Service A Service
B Messages
0.1 to 1000 millisecond latency
Coarse Grain Service Model
Closely coupled Java/Python …
Service B Service A
Publisher Post Events “Listener
Subscribe to Events
What is a Simple Service?
• Take any system – it has multiple functionalities– We can implement each functionality as an independent distributed service
– Or we can bundle multiple functionalities in a single service • Whether functionality is an independent service or one of many
method calls into a “glob of software”, we can always make them as Web services by converting interface to WSDL
• Simple services are gotten by taking functionalities and making as small as possible subject to “rule of millisecond”
– Distributed services incur messaging overhead of one (local) to
100’s (far apart) of milliseconds to use message rather than method call
– Use scripting or compiled integration of functionalities ONLY when require <1 millisecond interaction latency
• Apache web site has many projects that are multiple functionalities presented as (Java) globs and NOT (Java) Simple Services
Grids of Grids of Simple Services
• Link via methods messages streams • Services and Grids are linked by messages
• Internally to service, functionalities are linked by methods • A simple service is the smallest Grid
• We are familiar with method-linked hierarch
Lines of Code Methods Objects Programs Packages
Overlay and Compose
Grids of Grids Methods Services Component Grids
CPUs Clusters Compute
Resource Grids MPPs
Databases DatabasesFederated
Sensor Sensor Nets
Data
Component Grids?
• So we build collections of Web Services which we
package as
component Grids
–
Visualization Grid
–
Sensor Grid
–
Utility Computing Grid
–
Person (Community) Grid
–
Earthquake Simulation Grid
–
Control Room Grid
–
Crisis Management Grid
• We build bigger Grids by
composing component
Critical Infrastructure (CI) Grids built as Grids of Grids
Gas Service and Filters
Physical Network Registr
y Metadata
Flood Service and Filters
Flood CIGrid
…
Electricity Gas CIGridCIGrid
…
Data
Access/Storage Securit
y Notification Workflow Messaging Portal
s VisualizationGrid Collaboration
Grid
Sensor Grid Compute
Grid GIS Grid
Database Database Analysis and Visualizatio Portal Repositorie Federated Databases Data Filte Services
Field Trip Data
Streaming Data Sensor s
?
Discovery Services SERVOGrid Researc Simulation s Research Education Customization Services From Researc to Education Educatio Grid Computer FarmGeoscience Research and Education Grids
GI Grid
Sensor Grid Database Grid
IOI and CIE
• Let us study the two layers IOI (Service Internet On the Bit Internet) and CIE (Context and Information Environment)
• IOI is most “straightforward” as it is providing reasonably well understood capabilities at a new “level”
• CIE is roughly the inter-service “shared memory” used to manage and control them at “distributed operating system level
– Critical is “shared” (a database service) versus message based CIE
IOI
Application Specific Grids
Generally Useful Services and Grids
Workflow WSFL/BPEL
Service Management (“Context etc.”)
Service Discovery (UDDI) / Information
Service Internet Transport
Protocol
Service Interfaces WSDL
CIE Higher Level
Minicompute r Firewall Compute r Serve r PDA Mode m Laptop computer Workstatio n Peer s Peer s Audio/Video Conferencing Client Audio/Video Conferencing Client NaradaBrokering Broker Network
NaradaBrokering
Web Service B
Stream
Server-enhance Messaging
NB supports messages and streams
Prototype implementation of WS-ReliableMessaging Web Service reliability
NaradaBrokering enhanced Grid-FTP. Bridge to the Globus TK3. Grid Application Support
Java Message Service (JMS) 1.0.2b compliant Support for routing P2P JXTA interactions. Messaging Related
Compliance
Compression and Decompression of payloads Fragmentation a nd Coalescing of payloads Message Payload options
Message-level WS-Security compatible security Security
Recovery from failures and disconnects. Replay of events/messages at any time. Recovery and Replay
Producer Order and Total Order over a message type
Time Ordered delivery using Grid-wide NTP based absolute time Ordered delivery
Robust and exactly-once delivery of messages in presence of failures
Reliable delivery
Subscription can be Strings, Integers, XPath queries, Regular Expressions, SQL and tag=value pairs.
Subscription Formats
Transport protocols supported include TCP, Parallel TCP streams, UDP, Multicast, SSL, HTTP and HTTPS.
Communications through authenticating proxies/firewalls & NATs. Network QoS based Routing
Multiple transport support In publish-subscribe
Paradigm with different Protocols on each link
NaradaBrokering and IOI
• “Software Overlay Network” features• Support for Multiple Transport protocols • Support for multiple delivery mechanisms
– Reliable Delivery
– Exactly-once Delivery – Ordered Delivery
– Optional Delivery optimization modules for different modes • Compression/Decompression of payloads with optional module • Coalescing/Fragmentation of payloads with optional module • NTP Time Service
• Security Service
• Performance Monitoring
• Performance optimized routing with optional module
Virtualizing Communication
n Communication specified in terms of user goal and Quality of
Service – not in choice of port number and protocol
n Bit Internet Protocols have become overloaded e.g. MUST use
UDP for A/V latency requirements but CAN’t use UDP as firewall will not support ………
n A given “Service Internet” communication can involve multiple
transport protocols and multiple destinations – the latter possibly determined dynamically
Dial-u Filter Satellit
UDP
Firewal HTTP
A
B1
Hand-Hel Protocol Fas
Link
Software Multicast B2
B3
Performance Monitoring
n
Every broker
incorporates a
Monitoring service
that
monitors links
originating from the node.
n
Every link measures and exposes a set of
metrics
• Average delays, jitters, loss rates, throughput.
n
Individual links can
disable
measurements for
individual or the entire set of metrics.
n
Measurement
intervals can
also be varied
n
Monitoring Service,
returns measured
metrics to
Pentium-3, 1GHz, 256 MB RAM
100 Mbps LAN
JRE 1.3 Linux
hop-3 0 1 2 3 4 5 6 7 8 9 100 1000 Transit Delay (Milliseconds)
Message Payload Size (Bytes)
Mean transit delay for message samples in NaradaBrokering: Different communication hops
hop-2
NaradaBrokering Service Integration
S1 P1 P2 S2
S1 S2
Proxy Messaging
Handler Messaging
S? Service P? Proxy
Transport Controlled by Overlay Network
Internal to Service: SOAP Handlers/Extensions/Plug-ins Java (JAX-RPC) .NET Indigo and special cases: PDA's gSOAP, Axis C++
Mechanisms for Reliable Messaging I
•
There are essentially sequence numbers on each message
•
Unreliable transmission detected by non-arrival of a
message with a particular sequence number
•
Remember this is “just some TCP reliability” built at
application level
•
One can either use ACK’s – Receiver (service B) positively
acknowledges messages when received
– Service A fully responsible for reliability
•
Or NAK’s – Service B is partially responsible and tracks
message numbers – sends a NAK if sequence number
missing
Service B Service A
Mechanisms for Reliable Messaging II
•
Each message has a retransmission time; messages are
retransmitted if ACK’s not received in time
– Uses some increasing time delay if retransmit fails
•
Note need to be informed (eventually) that OK to throw
away messages at sender; pure NAK insufficient
•
Note this is reliability from final end-point to beginning
end-point: TCP reliability is for each link and has different
grain size and less flexible reliability mechanisms
•
There are several efficiency issues
– Divide messages into groups and sequence within groups
– Do not ACK each message but rather sequences of messages
Custom Message Reliability
Narada Broker
Filter 1 Filter
2
WS-RM Wireless
Optimized WS-RM
WS-Reliability
2 second PDA reply latency!
Different endpoints may well need different
reliability schemes. Another reason to use application layer.
NaradaBrokering and Fault Tolerance
•
As well as reliable messaging, NaradaBrokering supports
performance based dynamic routing
– Choose both route and protocol (UDP, Parallel TCP ..)
•
It will also support automatic fail-over among replicated
services subscribing to same message stream
•
Provides scriptable control of streams for custom
management schemes
•
Saves ALL messages in faul
tolerant storage for eithe
session replay or recovery
•
Will support reliable
BitTorrent
P2P file swapping model
(better than GridFTP?)
Fast Web Service Communication I
• IOI Application level Internet allows one to optimize
message streams at the cost of “startup time”,
Web Services
can deliver the fastest possible interconnections
with or
without reliable messaging
• Typical results from Grossman (UIC) comparing Slow
SOAP over TCP with binary and UDP transport (latter
gains a factor of
1000
)
Pure SOAP SOAP over UDP Binary over UDP
Fast Web Service Communication II
• Mechanism only works for
streams
– sets of related
messages
• SOAP header in streams is constant
except for
sequence number (Message ID), time-stamp ..
• One needs two types of new Web Service
Specification
• “
WS-StreamNegotiation
” to define how one can use
WS-Policy to send messages at start of a stream to
define the methodology for treating remaining
messages in stream
Fast Web Service Communication III
•
Then use “WS-StreamNegotiation” to
negotiate stream
in
Tortoise SOAP – ASCII XML over HTTP and TCP –
–
Deposit basic SOAP header through connection – it is part of
context for stream (linking of 2 services)
–
Agree on firewall penetration, reliability mechanism, binary
representation and fast transport protocol
–
Naturally transport UDP plus WS-RM
•
Use “WS-FlexibleRepresentation” to define encoding of a
Fast
transport
(On a different port) with messages just having
“FlexibleRepresentationContextToken”, Sequence Number,
Time stamp if needed
–
RTP packets have essentially this structure
–
Could add stream termination status
•
Can monitor and control with original negotiation stream
CIE: Common Service Information and Metadata
• Consider a collection of services working together– Workflow tells you how to specify service interaction but more basically there is shared information or context
specifying/controlling collection
• WS-RF and WS-GAF have different approaches to contextualization – supplying a common “context” which at its simplest is a token to represent state
• More generally core shared information includes dynamic service metadata and the equivalent of configuration information.
• One can supports such a common context either as pool of messages or as message-based access to a “database” (Context Service)
• Two services linked by a stream are perhaps simplest example of a collection of services needing context
• Note that there is a tension between storing metadata in messages and services.
Web Service Metadata and State I
•
The Semantic Grid and Semantic Web are important
frameworks for metadata but handicapped by lack of
“compelling” tools
•
RDF
Resource Description Framework (W3C) Set of
recommendations expanded from original February 1999
standard h
ttp://www.w3.org/RDF/
and the heart of the
Semantic Web and Grid h
ttp://www.semanticgrid.org
•
DAML+OIL
combining DAML (Darpa Agent Markup
Language) and OIL (Ontology Inference Layer) (W3C)
Note December 2001 ht
tp://www.w3.org/TR/daml+oil-reference
Web Service Metadata and State II
• WS-DistributedManagement Web Services DistributedManagement Framework with MUWS and MOWS below (OASIS) http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wsdm
• Management includes issues like monitoring quality of service, enforcing service level agreements, controlling tasks and managing life-cycles.
• WSDM-MUWS Web Services Distributed Management:
Management Using Web Services (OASIS) V0.5 Committee Draft April 2004 ht
tp://www.oasis-open.org/committees/download.php/6234/cd-wsdm-muws-0.5.pdf • WSDM-MOWS Web Services Distributed Management:
Management of Web Services (OASIS) V0.5 Committee Draft April 2004 htt p://www.oasis-open.org/committees/download.php/6255/cd-wsdm-mows-0.5-20040402.pdf
• WS-MetadataExchange Web Services Metadata Exchange (BEA,IBM, Microsoft, SAP) March 2004 http
://www-106.ibm.com/developerworks/library/specification/ws-mex/
– Describes how metadata can be exchanged between services rather than by looking it up in registries like UDDI or higher level metadata catalogs; the old OGSI standard used such service-resident metadata extensively
Web Service Metadata and State III
• WS-RF Web Services Resource Framework includingResourceProperties, ResourceLifetime,
WS-RenewableReferences, WS-ServiceGroup, and WS-BaseFaults
(OASIS)
http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wsrf with Oasis TC set up April 2004 and V1.1 Framework March 2004 http://www-
106.ibm.com/developerworks/library/ws-resource/ws-modelingresources.pdf
– Uses rich metadata to define stateful interactions – its use of SOAP header creates interoperability problems
• ASAP Asynchronous Service Access Protocol (OASIS)
• http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=asap with V1.0 working draft G June 2004 htt
p://www.oasis-open.org/committees/download.php/7151/wd-asap-spec-01g.pdf – Another approach to issues discussed in WSDM and WSRF
• WS-GAF Web Service Grid Application Framework (Arjuna, Newcastle University) http://www.neresc.ac.uk/ws-gaf/
Database SDE SDE Servic e SDE SDE Servic e SDE SDE Servic e SDE SDE Servic e SDE SDE Servic e SDE SDE Servic e SDE SDE Servic e Individual Services Web Service Ports
Grid or Domain Specific Metadata Catalogs System or Federated Registry or Metadata
Catalog
Database1 Database2 Database3
Four Metadata Architectures
Messages
Stateful Interactions
•
There are (at least) four approaches to specifying
state
– OGSI use factories to generate separate services for each session in standard distributed object fashion
– Globus GT-4 and WSRF use metadata of a resource to identify state associated with particular session
– WS-GAF uses WS-Context to provide abstract context
defining state. Has strength and weakness that reveals less about nature of session
– WS-I+ “Pure Web Service” leaves state specification the application – e.g. put a context in the SOAP body
Explicit and Implicit Factories
• Stateful interactions are typified by amazon.com where messages carry correlation information allowing multiple messages to be linked together
– Amazon preserves state in this fashion which is in fact preserved in its database permanently
• Stateful services have state that can be queried outside a particular interaction
• Also note difference between implicit and explicit factories
– Some claim that implicit factories scale as each service manages its own instances and so do not need to worry about registering instances and lifetime management
Notification Architecture
• Point-to-Poin
• Or Brokere
• NaradaBrokering will support both WS-Eventing
and WS-Notification as well as Java Message
Service JMS that is Java Notification standard
Service B Service A
Bro
ker Publish
Subscribe
Queues Messages Supports creation and subscription of topics
Service B Service A
NaradaBrokering and NTP
• NaradaBrokering includes an implementation of the Network Time Protocol (NTP)
• All entities within the system use NTP to communicate with atomic time servers maintained by organizations like NIST and USNO to compute offsets
– Offset is the computed difference between global time and the local time.
– The offset is computed based on the time returned from multiple atomic time servers.
• The NTP algorithms weighs results from individual time clocks based on the distance of the atomic server from the entity.
• This ensures that all entities are within 1 millisecond of each other.
• The timestamps account for clock drifts that take place on machines
Collaboration and Web Services
n
Collaboration
has
• Mechanism to set up members (people, devices) of a
“collaborative sessions”
• Shared generic tools such as text chat, white boards,
audio-video conferencing
• Shared applications such as Web Pages, PowerPoint,
Visualization, maps, (medical) instruments ….
n
b)
and
c)
are “just shared objects” where objects
could be Web Services but rarely are at moment
• We can port objects to Web Services and build a generalapproach for making Web services collaborative
n
a)
is a “Service” which is set up in many different
Shared Event Collaboration
n All collaboration is about sharing events defining state changes
• Audio/Video conferencing shares events specifying in
compressed form audio or video
• Shared display shares events corresponding to change in
pixels of a frame buffer
• Instant Messengers share updates to text message streams • Microsoft events for shared PowerPoint (file replicated
between clients) as in Access Grid
n Finite State Change NOT Finite State Machine architecture n Using Web services allows one to expose update events of all
kinds as message streams
n Need publish/subscribe approach to share messages (NB) plus n System to control “session” – who is collaborating and rules
• XGSP is XML protocol for controlling collaboration building
XGSP Web Service MCU Architecture
SIP H323 AccessGrid NativeXGSP Admire
Gateways convert to uniform XGSP Messaging
High Performance (RTP and XML/SOAP and ..
Media Servers
Filters Session Server
XGSP-based Control
NaradaBrokerin g
All Messaging
Use Multiple Media servers to scale to many codecs and many versions of audio/video mixing
NB Scales a distributed
We Services
Global-MMCS Community Grid
n We are building an open source protocol independent WebService “MCU” which will scale to an arbitrary number of users and provide integrated thousands of simultaneous users
collaboration services.
n The function of A/V media server is distributed using
NaradaBrokering architecture.
• Media Servers mix and convert A/V streams
n Open XGSP MCU based on the following open source projects
• openh323 is basis of H323 Gateway
• NIST SIP stack is basis of SIP Gateway
• NaradaBrokering is open source messaging
• Java Media Framework basis of Media Servers
• Helix Community http://www.helixcommunity.org for Real
Media
n http://www.globalmmcs.org open source “non advertised”
Break up into Web Services
n Monolithic MCU becomes many different “Simple Services”• Session Control
• Thumbnail “image” grabber
• Audio Mixer
• Video Mixer
• Codec Conversion • Helix Real Streaming • PDA Conversion
• H323/SIP Gateways
n As independent can replicate particular services as needed
• Codec conversion might require 20 services for 20 streams
spread over 5 machines
n 1000 simultaneous users could require:
• 1 session controller, 1 audio mixer, 10 video mixers, 20 codec
converters, 2 PDA converters and 20 NaradaBrokers
n Support with a stream optimized Grid Farm in the sky
• Future billion way “Video over IP” serving 3G Phones and home media
GlobalMMCS and NaradaBrokering
n All communication – both control and “binary” codecs are
handled by NaradaBrokering
n Control uses SOAP and codecs use RTP transport n Each stream is regarded as a “topic” for NB
n Each RTP packet from this stream is regarded as an “event” for
this topic
n Can use replay and persistency support in NB to support
archiving and late clients
n Can build customized stream management to administer replay,
and who gets what stream in what codec
n NaradaBrokering supports unicast and multicast
n Use firewall penetration and network monitoring services in NB
0 1 0 2 0 3 0 4 0 5 0 6 0
0 20
0 400 600 800 1000 1200 1400 1600 1800 2000
Delay (Milliseconds)
Packet Number
Average delays per packet for 50
video-clientsNaradaBrokering Avg=2.23 ms, JMF Avg=3.08
ms
NaradaBrokering-RTP
0 1 2 3 4 5 6 7 8
0 20
0 400 600 800 1000 1200 1400 1600 1800 2000
Jit
ter
(Milliseconds)
Packet Number
Average jitter (std. dev) for 50 video clients.
NaradaBrokering Avg=0.95 ms, JMF Avg=1.10
ms
NaradaBrokering-RTP
Polycom, Access Grid
and RealVideo views of
Integration of PDA, Cell phone and Desktop Grid Access
Grids and e-globalcommunity
nPeer-to-peer networks
already are a good example of
value of Information Technology supporting broad
global communities
• File sharing, text chats, bulletin boards
n
Grids must include these capabilities and extend in
terms of increased
functionality
and
quality of service
n
This will support business and cultural interactions
between nations
n
Several interesting applications can be supported by
• Replacing files by multi-media streams so can collaborate in
real-time
• Adding traditional tools like audio-video conferencing and shared applications to P2P set
n
This integration of P2P and Grid to give M
2Grids
Streaming M
2
Grids
e-Textilemanufacturing involves Clothes designers in USA and
manufacturers in Hong Kong exchanging designs which are
streams of images
e-Sports is a possible collaboration between Indiana University and
Beijing Sport University
• Basket ball coaches (teacher) interact with aspiring NBA players in China
• Martial Arts masters in China train neophytes in Indiana
• Faculty recreational sports adviser works from university with faculty exercising at home
• Hope to have working incredibly well by the 2008 Olympics
Interactive TV Grid: allows anybody to discuss professional or
home video (of sports or other events) within a custom Grid
Multi-player distributed games which should be supported with
exactly the same overlay Grid
Video Game Production Grid links artistic direction (design) in one
country with digital animation (manufacturing) in another
e-Science: Physics and Environmental Science Sensors
Surveillance Grid enables security personnel to annotate and
Some Technology for Streaming M
2Grids
Basic capability is
collaborative annotatable multimedia
tool
for images, sensors and real-time video streams
•
Allow Grid participants to view real-time streams,
rewind on the fly and add text and graphical
comments
•
Similar to instant replay on TV but far more flexible
Need rich
metadata system
to label and correlate
streams, images and annotations
Extend
Grid and P2P file access
paradigms to
stream
storage, browsing and access
Core Technologies shared with
distance education
Using http:/
/www.globalmmcs.org for mu
ltimedia
GlobalMMCS Futures I
• 1. New Collaboration tools– SVG game ( stable ) – Whiteboard ( stable ) – e-Sport ( prototype)
– Jabber IM client ( prototype)
• 2. JMF Audio/Video client ( stable)
– performance enhancement finished
– new codec ( MPEG4 finished; try H.264) – support different platform ( Linux, Mac ) – support NAT/firewall
– screen codec for shared display ( prototype )
• 3. Replay & Archive (prototype)
– Replay Engine based on NaradaBroker Storage Service – XGSP-RTSP gateway
– Extend RTSP and NaradaBrokering for Instant Replay
• 4. Web Server ( stable)
GlobalMMCS Futures II
•
5. Conferencing Media Processing Service ( Stable)
– Fix the bugs and add scheduling – Support new codec ( MPEG4 )
•
6. H.323 Gateway ( Stable)
– Import it to Linux platform
•
7. RealStreaming Gateway ( Stable )
– Import it to Linux
– Support Mobile clients
•
8. Global-MMCS deployment & test
– Test under the setting of multiple NaradaBroker and NAT/Firewall
– support deployment for AFRL, NASA, DOE portals – test with remote sites
P2P and Server based solutions
n Peer-to-peer architectures have advantage that they can be deployed
just using client resources and no system commitment is needed
n Typically clients do not have good network QoS and it is hard for
example to support rich multi-point audio video conferencing in this way
n M2 Grids typically require multicast so average load in P2P case on
client legs goes like O(M)
• Server-side multicast puts O(M) load on backbone and O(1) load
on clients and can lead to much better scaling and performance
• N plus N Grids may not see such large improvements with server
side support
n So Grids should support initial P2P deployment with a seamless
upgrade to add better QoS using Servers.
n Extend familiar P2P paradigms like BitTorrent to Grids and
Streaming
n Grid and peer-to-peer linkage combines scalable performance with
ease of deployment P2P
Grid Farm in the Sky (clouds)