The Problem: Integrating Data,
Applications, and Client Devices
n The key issue we try to solve is building the distributed computing infrastructure
that can connect
• Legacy data archives • Executable codes
• Real time data sources
• Client tools for collaboration
n Audio/Video systems, whiteboard annotators, etc
n Various application-specific grids can be built out of the common infrastructure • Science Grids
• Emergency planning, crisis response
n We choose certain fixed points for our foundations • Other standards where available: GIS standards • Web Service standards: SOAP and WSDL
• Universal messaging substrate for SOAP and other messages:
OGC Standards for GIS
n
OpenGIS Consortium is an international group that defines
common formats for geospatial data and service standards.
n
Main data format language is the XML-based
Geography
Markup Language (GML).
• GML is widely accepted as the universal encoding for geo-referenced
data.
n
OGC defines services.
• Some example OGC services include
n Web Feature Service (WFS): for retrieving GML encoded features, like
faults, roads, county boundaries, GPS station locations,….
n Web Map Service (WMS): for creating maps out of Web Features
What Are Web Services?
n Web Services framework is a way
for doing distributed computing with XML.
• WSDL:Defines interfaces to functions of remote components. This is the programming interface. • SOAP: Defines the message format
that you exchange between components. This carries the messages over the network.
• Web Services are notweb pages, CGI, servlets, applets.
n XML provides cross-language
support
n Suitable for both human and
application clients.
n We are building data grid
components in the same fashion.
• Allows us to build general purpose client environments like Web
portals and command line tools.
Service Oriented Architecture
n
Key SOA Features
(Following W3C’s WS Architecture working group.)
•
SOAs are composed of services that present
programmatic access to resources to remote client
applications.
•
Services communicate using messages. Messages are
usually encoded using SOAP.
•
SOAs are metadata rich. We must describe service
Publish/Find/Bind paradigm in a GIS Grid Service providers
Service requestors Service registries
NaradaBrokering (NB)
n
NaradaBrokering is a distributed messaging infrastructure.
n
It may be thought of a as topic-based publish/subscribe
messaging system:
• interested entities can register to a NaradaBrokering node to send and
receive messages on particular topics.
n
It supports multiple underlying transport protocols, intelligent
message routing, asynchronous messaging, reliable delivery,
replay of events, compression/fragmentation, discovery etc.
Pattern Informatics in a Grid
Environment
n
PI in a Grid environment:
• Hotspot forecasts are made using publicly available seismic records.
n Southern California Earthquake Data Center
n Advanced National Seismic System (ANSS) catalogs
• Code location is unimportant, can be a service through remote execution • Results need to be stored, shared, modified
• Grid/Web Services can provide these capabilities
n
Problems:
• How do we provide programming interfaces (not just user interfaces) to
the above catalogs?
• How do we connect remote data sources directly to the PI code. • How do we automate this for the entire planet?
n
Solutions:
• Use GIS services to provide the input data, plot the output data
• Use HPSearch tool to tie together and manage the distributed data sources
<?xml version="1.0" encoding="UTF-8"?>
<soap:Envelope xmlns:soap="http://www.w3..."> <soap:Header encodingStyle=“WSCTX URL"
mustUnderstand="true">
<context xmlns=“ctxt schema“ timeout="100"> <context-id>http..</context-id>
<context-service> http.. </context-service> <context-manager> http.. </context-service> <activity-list
mustUnderstand="true" mustPropagate="true"> <p-service>http://../WMS</p-service>
<p-service>http://../HPSearch</p-service> </activity-list> </context> </soap:Header> ... SOAP header for Context
The Pattern Informatics GIS-SOA based workflow application
5,6: WMS starts a session, invokes HPSearch to run workflow script for PI Code with a session id 7,8,9:HPSearch runs the workflow script and generates output file in GML format (& PDF Format) as result
10: HPSearch writes the URI of the of the output file into Context
11: WMS polls the information from Context Service
12: WMS retrieves the generated output file by workflow script and generates a map
<context xsd:type="ContextType"timeout=“100">
<context-service>http://.../HPSearch</ context-service> <content> HPSearch associated additional data
generated during execution of workflow. </content> </context>
service associated
<context xsd:type="ContextType"timeout=“100">
<context-service>http://.../WMS</ context-service> <activity-list mustUnderstand="true" mustPropagate="true"> <service>http://.../WMS</service> <service>http://.../HPSearch</service> </activity-list> </context> session <context xsd:type="ContextType"timeout=“100"> <context-service>http://.../HPSearch</
context-service>
<parent-context>http://../abcdef:012345<pa rent-context/>
<content> profile information related WMS </content> </context>
user profile
<context xsd:type="ContextType"timeout=“100">
<context-id>http://../abcdef:012345<context-id/>
<context-service>http://.../HPSearch</ context-service> <content>http://danube.ucs.indiana.edu:8080\x.xml</c ontent> </context> shared state <context xsd:type="ContextType"timeout=“100"> <context-service>http://.../HPSearch</ context-service> <parent-context>http://../abcdef:012345<parent-context/> <content> shared data for HPSearch activity </content>
The Interdependent Energy Infrastructure Simulatio
System (IEISS) in a Grid Environment
n
IEISS in a Grid environment:
• IEISS provides modeling, simulation and analysis of interdependent
energy infrastructures. Input is data models in XML format.
n Los Alamos National Laboratory n Argon National Laboratory
• Code location is unimportant, can be a service through remote execution • Results need to be stored, shared, annotated
• Grid/Web Services can provide these capabilities
n
Problems:
• How do we provide programming interfaces (not just user interfaces) to
the input data models?
• How do we connect remote data sources directly to the IEISS code?
n
Solutions:
• Use a SOA with GIS services to provide the input data, plot the output
data
• Use WS-I compatible XML metadata services to support GIS workflow
WFS and WMS publish their WSDL URL to the UDDI Registry 1-2-3 - WMS Client -> WMS Server -> UDDI -> WFS
4-5 - WFS publishes the results as GML FeatureCollection document into a topic (“/NISAC/WFS”) in a pub/sub based messaging system. WFS -> WMS Server (creates a map overlay) and IEISS receive this GML document. WMS Server -> WMS Client (displays it)
7 - On receiving invocation message, IEISS updates the shared state
8 – On receiving the notification, WMS Client makes a request to the WFS-L for the IEISS output
Some Challenges
n Security: These applications have interesting security requirements
• Authentication, authorization and priority for different users • Time/event dependent security for crisis response
n Performance: Are GIS services suitable for non-trivial data transfers?
• We are using streaming data transfers for higher performance
• We research fast data transfer methodologies for higher performance
n Fault tolerance and scalability: We must support SOA based distributed application with reliable information sources.
• We design and build fault tolerant and high performance decentralized Information
Services to support metadata requirements of SOA.
n Adoption: We must get the tools and services to the point where science application developers want to use them early in the development process rather than later. • Web Service client libraries to remote GIS data
• Develop codes to work with data streams rather than files.
Location of information and software for
dependent Grid Projects
n
htpp://www.naradabrokering.org p
rovides Web service
(and JMS) compliant
distributed publish-subscribe
messaging
(software overlay network)
n
ht
tp://www.crisisgrid.org is
an OGC (open geospatial
consortium) Geographical Information System (GIS)
compliant
GIS and Sensor Grid.
n
