Web Service Information
Systems and Applications
GGF16 Semantic Grid Workshop Athens Greece
February 15 2006 Geoffrey Fox
Computer Science, Informatics, Physics Pervasive Technology Laboratories Indiana University Bloomington IN 47401
http://grids.ucs.indiana.edu/ptliupages/presentations/
Different Metadata Systems
n There are many WS-* specifications addressing meta-data
defined broadly
• WS-MetadataExchange
• WS-RF
• UDDI
• WS-ManagementCatalog
n And many different implementations from (extended) UDDI
through MCAT of the Storage Research Broker
n And of course representations including RDF and OWL
n Further there is system metadata (such as UDDI for core
services) and metadata catalogs for each application domain such as WFS (Web Feature Service) for GIS (Geographical Information Systems)
n They have different scope and different QoS trade-offs
• e.g. Distributed Hash Tables (Chord) to achieve scalability in large scale networks
• WS-Context
• ASAP
• WBEM
• WS-GAF
Different Trade-offs
n It has never been clear to me how a poor lonely service is meant
to know where to look up meta-data and if it is meant to be
thought up as a database (UDDI, Ws-Context) or as the contents of a message (WS-RF, WS-MetadataExchange)
n We identified two very distinct QoS tradeoffs
n 1) Large scale relatively static metadata as in UDDI catalog of all
the world’s services
n 2) Small scale highly dynamic metadata as in dynamic workflows
for sensor integration and collaboration
• Fault-tolerance and ability to support dynamic changes with
few millisecond delay
• But only a modest number of involved services (up to 1000’s
in a session)
• Need Session NOT Service/Resource meta-data so don’t use
XML Databases of Importance
n We choose a message based interface to a backend database
n We have two pieces of technology with different trade-offs but
each could store any meta-data but with different QoS
• WS-Context designed for controlling a workflow
• (Extended) UDDI exemplified by semantic service discovery
n WFS or ASFS (see later) provides general application specific
XML data/meta-data repository built on top of a hybrid system supported by UDDI and WS-Context
n These have different performance, scalability and data unit size
requirement
n In our implementation, each is currently “just an
Oracle/MySQL” database front ended by filters that convert
between XML (GML for WFS) and object-relational Schema
• Example of Semantics (XML) versus representation (SQL)
difference
n OGSA-DAI offers Grid interface to databases – we could use but
don’t as we only need to expose WFS and not MySQL to Grid
n
We also designed and implemented an
extended
UDDI XML Metadata Service
(alternative to OGC
Web Registry Services).
This service
,
n
supports WFS GIS Metadata Catalog (
functional
metadata),
user-defined
metadata ((name, value)
pairs),
up-to-date
service information (leasing),
dynamically updated
registry entries.
n
Our approach enables advanced query capabilities
•
geo-spatial
and
temporal
queries ,
•
metadata oriented queries,
•
domain independent queries
such as XPATH,
XQuery on metadata catalog.
n
http://www.opengrids.org/extendeduddi/index.html
WMS uses WFS that uses data sources
<gml:featureMember>
<fault>
<name> Northridge2 </name> <segment> Northridge2
</segment>
<author> Wald D. J.</author>
<gml:lineStringProperty>
<gml:LineString
srsName="null">
<gml:coordinates>
118.72,34.243 -118.591,34.176
</gml:coordinates>
</gml:LineString>
</gml:lineStringProperty>
</fault>
</gml:featureMember>
Electric Power and Natural Gas data from LANL
Interdependent Critical Infrastructure Simulations
Zoom-in Zoom-out
FeatureInfo mode Measure distance mode Clear Distance
Drag and Drop mode Refresh to initial map
Google maps can be
integrated with Web Feature Service
Archives to filter and
browse seismic records.
Context as Service Metadata
n
We define all metadata (static, semi-static, dynamic)
relevant to a service as “
Context
”.
n
Context
can be associated to a single service, a
session (service activity) or both.
n
Context can be independent of any interaction
n
slowly varying, quasi-static
context
n
Ex: type or endpoint of a service, less likely to
change
n
Context
can be generated as result of service
interactions
n
dynamic, highly updated
context
n
information associated to an activity or session
n
Ex: session-id, URI of the coordinator of a
workflow session
Hybrid XML Metadata Services –>
WS-Context + extendedUDDI
n
We
combine
functionalities of these two services:
WS-Context AND extendedUDDI in one hybrid service to
manage
Context
(service metadata).
• WS-Context controlling a workflow
• (Extended) UDDI supporting semantic service discovery
n
This approach enables
uniform query capabilities
on
service metadata catalog.
n
This service supports collection of services that
• are set of actively collaborating managed services for a
particular common goal
• generate events as result of interactions
• are very small part of the whole GRID
Information Service WSDL IS Client WSDL WSDL HTTP(S) WSD L IS Client DB JDBC Extended WS-Context Service dynamic metadata IS Client WSDL DB JDBC Extended UDDI Registry Service UDDI Version 3.0 WSDL
Service Interface Descriptions
uddi_api_v3_portType.wsdl
WSDL WSDL WSDL
n
CGL GIS Research focuses on open source software for
Grids to support scientific/visualization applications.
• some focus areas are earthquake prediction and simulating
energy interdependencies with GIS Grids.
• http://www.crisisgrid.org
n
We
use WS-Context complaint Information Services
for storing transitory metadata for distributed state
information.
n
We use extended UDDI Information Services for
providing search/store/access to geo-spatial
domain-specific metadata catalogs
n
All data filters output data as though they were a WFS
so data filters and sensors interchangeable
Database
S S
S
S SS SS SS SS SS SS SS SS
F S F S F S F S F S F S F S F
S SF
F S F S F S F S F S F S F S F S F S F S F
S Portal
F S O S O S O S O S O S O S O S O S O S O S O S O S MD MD MD MD MD MD MD MD MD
MetaData Filter Service Sensor Service Other Service Anothe Grid
Raw Data Data Information Knowledge Wisdom Decisions S S S S Anothe Service Anothe Service S S Anothe
Grid S S
Typical use of Grid Messaging in NASA
Datamining Grid
Sensor Grid
Grid Eventing GIS Grid
Real Time GPS
and Google Maps
Subscribe to live GPS station. Position data from SOPAC is
combined with Google map clients.
Select and zoom to GPS station location, click icons for more information.
Applications II – Managing and Scripting Web
Services: HPSearch
n
HPSearch is simply a scripting environment designed
for managing distributed workflows.
• HPSearch runs scripts which in turn invokes various services
to start the flow.
• The HPSearch works in tandem with WS-Context Service for
communicating with services.
• More info available at http://www.hpsearch.org
n
The
WS-Context is used by HPSearch as transitory
metadata store for storing shared, session related data.
n
HPSearch updates the session state as the services
completes their execution-cycle.
<?xml version="1.0" encoding="UTF-8"?>
<soap:Envelope xmlns:soap="http://www.w3...">
<soap:Header encodingStyle=“URL" mustUnderstand="true"> <context xmlns=“ctxt schema“ timeout="100">
<context-id>http..</context-id>
<context-service> http.. </context-service> <context-manager> http.. </context-service> <activity-list mustUnderstand="true" mustPropagate="true"> <p-service>http://../WMS</p-service> <p-service>http://../HPSearch</p-service> </activity-list> </context> </soap:Header> SOAP header for Context
The Pattern Informatics GIS-SOA based workflow application
5,6: WMS starts a session, invokes HPSearch to
run workflow script for PI Code with a session id
7,8,9:HPSearch runs the workflow script and generates output file in GML format (& PDF Format) as result
10: HPSearch writes the URI of the of the output
file into Context
11: WMS polls the information from Context
Service
12: WMS retrieves the generated output file by
workflow script and generates a map
<context xsd:type="ContextType"timeout=“100">
<context-service>http://.../HPSearch</ context-service> <content> HPSearch associated additional data generated
during execution of workflow. </content> </context>
service associated
<context xsd:type="ContextType"timeout=“100">
<context-service>http://.../WMS</ context-service>
<activity-list mustUnderstand="true" mustPropagate="true"> <service>http://.../WMS</service> <service>http://.../HPSearch</service> </activity-list> </context> session <context xsd:type="ContextType"timeout=“100"> <context-service>http://.../HPSearch</ context-service> <parent-context>http://../abcdef:012345<parent-context/> <content> profile information related WMS </content>
</context> user profile <context xsd:type="ContextType"timeout=“100"> <context-id>http://../abcdef:012345<context-id/> <context-service>http://.../HPSearch</ context-service> <content>http://danube.ucs.indiana.edu:8080\x.xml</content> </context> shared state <context xsd:type="ContextType"timeout=“100">
<context-service>http://.../HPSearch</ context-service> <parent-context>http://../abcdef:012345<parent-context/> <content> shared data for HPSearch activity </content>
<activity-list mustUnderstand="true" mustPropagate="true"> <service>http://.../DataFilter1</service> <service>http://.../PICode</service> <service>http://.../DataFilter2</service> </activity-list> </context> activity 3 WMS WFS http://..../..../..txt HP Search Data Filter PI Code Data Filter http://..../..../x.gml Context Information Service 4 7,8,9 10 6 5,11 WMS Client Extended UDDI 0 1 2
n
Handheld Flexible Representation
(HHFR) is an open
source software for fast communication in mobile Web
Services. HHFR supports:
• streaming messages, separation of message contents and
usage of context store.
• http://www.opengrids.org/hhfr/index.html
n
We use WS-Context service as
context-store for
redundant message parts
of the SOAP messages.
• redundant data is static XML fragments encoded in every
SOAP message
• Redundant metadata is stored as context associated to service
conversion in place
n
The empirical results show that we gain
83%
in
message size
and on avg.
41%
on
transit time
by using
WS-Context service.
Applications III – Context Store usage in
communication of mobile Web Services
Optimizing Grid/Web Service Messaging
Performance
n GlobalMMCS is audio/video conferencing system.
http://www.globalmmcs.org
n WS-Context service is used as session metadata catalog for
describing audio/video streams in a session.
• stream metadata might have various descriptive information such as
starting and ending RTP packet numbers, what topic is the stream is published, etc..
• metadata is stored associated with sessions
n WS-Context service provides search/store/access interface for
stream metadata as archival store.
• Queries support archived (find all relevant data in past) metadata
• Queries support session – metadata relations (find all metadata for a given
session)
• Session discovery (find a session with given properties)
n Clients query context store for information to be able to
replay/playback an audio/video stream during or after
conference session.
Applications IV – Context Store usage in multimedia
collaboration environment such as GlobalMMCS
GlobalMMCS SWT Client
Chat TV
Webcam Video
Mixer GIS
e - Annotation Playe
r
Archived stream playe
r
Annotatio
nplaye / WB r
Archieved
stream Real timestream
e -Annotation Whiteboar
d
Real time stream playe
Archived Real Time Real Tim
e-Annotation Archived Stream Annotated e-Annotation
Information Architecture and Semantic Grid
n
WS-*
provides key low level capability but deliberately
does not define an information (data) architecture
and
leaves this to domain specific specification activities such
as
CellML/SBML for biology
,
WFS/GML
for
GIS
and
XGSP for Collaboration
n
WS-*
does define a primitive
service discovery
(UDDI)
and
meta-data
capabilities including
Context,
WS-RF, RDF and WS-MetadataExchange
already discussed.
n
GGF
defines Grid data capabilities including
info-D
(publish/subscribe) and
OGSA-DAI
for data repositories
n
Semantic Grid
uses
WS-* and GS-*
extending meta-data
and service discovery with data-mining and reasoning
Information Management/Processing
n SOAP messages transport information expressed in a
semantically rich fashion between sources and services that enhance and transform information so that complete system provides
• Semantic Web technologies like RDF and OWL help us have
rich expressivity
n Data Information Knowledge transformation
n We build application specific information
management/transformation systems ASIS for each application
domain
n One special domain is the system itself where the metadata
Generalizing a GIS
n
Geographical Information Systems
GIS have been
hugely successful in all fields that study the earth and
related worlds
• They define Geography Syntax (GML) and ways to store,
access, query, manipulate and display geographical features
• In SOA, GIS corresponds to a domain specific XML language
and a suite of services for different functions above
n
However such a universal information model has
not
been developed in other areas
even though there are
many fields in which it appears possible
• BIS Biological Information System
• MIS Military Information System
• IRIS Information Retrieval Information System
• PAIS Physics Analysis Information System
• SIIS Service Infrastructure Information System
ASIS Application Specific Information System I
n a) Discovery capabilities that are best done using WS-*
standards
n b) Domain specific metadata and data including
search/store/access interface. (cf WFS). Lets call generalization
ASFS (Application Specific Feature Service)
• Language to express domain specific features (cf GML). Lets call
this ASL (Application Specific language)
• Tools to manipulate information expressed in language and key data of application (cf coordinate transformations). Lets call this
ASTT (Application specific Tools and Transformations)
• ASL must support Data sources such as sensors (cf OGC metadata
and data sensor standards) and repositories. Sensors need
(common across applications) support of streams of data
• Queries need to support archived (find all relevant data in past) and streaming (find all data in future with given properties)
• Note all AS Services behave like Sensors and all sensors are
wrapped as services
• Any domain will have “raw data” (binary) and that which has been
ASIS Application Specific Information System II
n Lets call this ASVS (Application Specific Visualization Services)
generalizing WMS for GIS
n The ASVS should both visualize information and provide a way of
navigating (cf GetFeatureInfo) database (the ASFS)
n The ASVS can itself be federated and presents an ASFS output
interface
n d) There should be application service interface for ASIS from which all
ASIS service inherit
n e) There will be other user services interfacing to ASIS
n All user and system services will input and output data in ASL using
filters to cope with ASBD
AS Tool (generic
) A
“Sensor ” A Repository
AS Service (user defined)
ASVS Displa
y AS Tool
(generic )
Messages using ASL
Filter, Transformation, Reasoning, Data-mining, Analysis
