Information Federation in Grid
Information Services
Mehmet S. Aktas
Talk Outline
§
Use Cases and Challenges
§
Research Issues
§
Architecture
Ø
Hybrid Grid Information Service
§
Performance Evaluation
§
Conclusions
Introduction
§
Grid Information Services in Service Oriented Architectures
Ø
1) Large scale relatively static metadata as in catalog of all the
world’s services
• Interaction-independent, slowly-varying metadata
Ø
2) Small scale highly dynamic metadata as in dynamic workflows
for sensor integration and collaboration
• Interaction-dependent, dynamic metadata • Dynamic Grid/Web Service Collections*
– Dynamically assembled relatively small number of services (sub-grid) – Gathered at any one time to support a specific task
Motivating Use Cases
§
Geophysical Data Grids - CGL
Ø Service Oriented Architecture for Geographical Information Systems
Supporting Real Time Data Grids
§
Pattern Informatics (PI) - UC Davis
Ø Earthquake forecasting code developed by Prof. John Rundle (UC
Davis) and collaborators, uses seismic archives.
§
Interdependent Energy Infrastructure Simulation System
(IEISS) - LANL
Ø Models infrastructure networks (e.g. electric power systems and
natural gas pipelines) and simulates their physical behavior, interdependencies between systems.
§
eSports System - CGL
Ø Annotative collaboration application. Supports archive, replay,
Background
§
Specifications for interaction-independent metadata
Ø UDDI Specification
Ø Glue Specification
Ø EbXML Specification
Ø Web Registry Service Specification
§
Specifications for interaction-dependent metadata
Ø Point-to-point approach
• Web Service Resource Framework (WSRF) Specification
Ø Third-party approach
Challenges
§
Standardization and Unification Issues
Ø Customized Grid Information Services
Ø Fat clients
§
Performance and Centralization Issues
Ø Low performance
Ø Low fault tolerance
§
UDDI Specification Issues
Ø Lack of up-to-date, metadata-oriented registry Ø Lack of domain-specific metadata management
§
WS-Context Specification Issues
Research Issues I
§
Unification
Ø How to combine different information services?
§
Federation
Ø How to federate different information services?
§
Flexibility
Ø How to accommodate broad range of specific application domains?
§
Interoperability
Ø How to facilitate connection with wide range of information service
Research Issues II
§
Performance
Ø
How to provide efficient information management strategies?
• high-performance, scalable in-memory storage • efficient request distribution
• adaptation to instantaneous client-demand changes
§
Fault-tolerance
Ø
How to provide efficient replica-content placement strategies?
§
Consistency
Hybrid Grid Information Service
§
Unification
§
Federation
ØUnified Schema ØQuery/Publish API
§
Flexibility
§
Interoperability
ØExtended UDDI ØWS-Context ØGlue
Ø…
Hybrid Grid Information Service
§
Unification
§
Federation
ØUnified Schema ØQuery/Publish API
§
Flexibility
§
Interoperability
ØExtended UDDI ØWS-Context ØGlue
UDDI instance
WS-Context instance
Support for interaction-independent
metadata:
Extended UDDI Service
§
There are other extensions of UDDI
§
Supports different types of metadata
ØUser-defined metadata ØFunctional metadata
§
Enables advanced query capabilities
ØGeo-spatial, metadata-oriented, domain-independent queries
§
Provides additional capabilities
ØUp-to-date service registry information (leasing) ØDynamic aggregation of capabilities of services
e.g. geospatial capabilities
[GGF16-Semantic Grid Workshop] Web Service Information Systems and Applications http://www.semanticgrid.org/OGF/ggf16/papers/GGF16SemGrid-CGL.pdf
[SKG06 – IEEE Proceedings] XML Metadata Services
Support for interaction-dependent
metadata:
WS-Context Service
§
OASIS Standard
§
Context Manager Service
Ø Data model and communication protocol
§
Supports Dynamic Web Service Collections
Ø Distributed state based systems e.g. workflow-style grids Ø Session metadata management
e.g. real-time replay and session-failure recovery capabilities
§
Provides various capabilities
Ø Notification capability
Ø Up-to-date metadata registry (leasing)
[SKG05 – IEEE Proceedings] Information Services for Dynamically Assembled Semantic Grids http://grids.ucs.indiana.edu/ptliupages/publications/skg05-56-maktas-ieee-version.pdf
Support for federated service
metadata:
Information Federation
Support for federated service
metadata:
Information Federation
§
Federating Grid Information Services
ØUnified Schema and communication protocol ØExtended UDDI, WS-Context and Glue Schemas
§
Approach taken for Unified Schema [Schema Integration]
ØSchema Matching
Identify overlapping information in given two Schemas: S1 and S2 ØSchema Merging
Use the identified overlapping information to guide merge of S1 and S2
§
Communication protocol
ØPublish: save_ (create, update), delete_ e.g. save_service, delete_service
ØInquiry: find_ , get_
Schema Matching: Identifying Matching Concepts
serviceAttributeEntity: Information about metadata associated to services
Site
Service ComputingElement StorageElement
site:information about a site where services, computing elements and storage elements are aggregated
ServiceData
service:all information about a Service
ExtUDDI.businessEntity 1:N GLUE.site
Extended UDDI
GLUE
metadata:information about metadata associated to service
bindingTemplate:Technical information about a service point
tModel:Description of Specifications for services or taxonomies
publisherAssertions:
Defines relationships between two business entities
computingElement:all info. required to manage
computing resources
storageElement:all information required to manage storage resources
businessEntity:information about the party who publishes information about entities
service:all information about a service
site:all information about a concept to aggregate services and resources
site contains one to n computing element
has references to
site contains one to n services
site contains one to n storage element business contains
one to n services has references to
service contains one to n metadata service contains one to n
technical information
business contains one to n site
Schema Merging: Unifying Schemas
Unified Schema GLUE
Key Design Features
§
In-Memory storage
Ø
High performance metadata access/storage
§
Access distribution
Ø
Redirecting client request to an appropriate replica server
§
Replica content placement for performance
Ø
Dynamic replication
Moving/replicating metadata to where they are demanded.
§
Replica content placement for fault-tolerance
Ø
Permanent replication
Replicating data on an appropriate replica server
§
Consistency enforcement
In-Memory Storage
§
Light-weight implementation of JavaSpaces
Ø
Data sharing, associative lookup
§
Integrated in-memory storage capability
Ø
Ex: UDDI-type, WS-Context-type
Ø
Today’s servers are capable of holding such small size metadata
in memory.
§
Persistency
Ø Newly-inserted/updated metadata is backed-up into appropriate
information service back-end.
Ø If the physical memory wiped out, at the bootstrap,
Access Distribution and Dynamic Replication
§
Broadcast-based request dissemination
Ø
Pub-sub system for message broadcast
Ø
Requests are broadcast only to those servers that can answer
ØNo need to keep track of metadata locations
§
Replica-content placement for performance
Ø
Popular copies are moved/replicated where they are demanded
ØDynamic migration/replication algorithm
*
Access Distribution Experimen
Benchmark Methodology
T
1 T2 T3
Time = T1 + T2 + T3
One-broker case
Experiment Results
Experiment Results
T
1 T2 T3
Time = T1 + T2 + T3
Dynamic Replication Performance Experimen
Benchmark Methodology
Simulation parameters
message size / message rate 2.7 Kbytes / 10 msg/sec replication decision frequency every 100 seconds
§
The decrease in average latency shows that the algorithm
manages to move replica copies to where they are demanded.
Replication and Consistency
§
Permanent replication for fault tolerance
Ø Each node keeps information about other servers Ø Replica Server(s) Selection
• Load and proximity metrics
• Selection algorithm by Rabinovich et al
Ø Unicast-based replica-content placement
§
Primary-copy approach
Ø Updates are unicast to primary-copy
Ø Updates are broadcast by the primary-copy holder to
• a) permanent-copy holding servers
Fault-tolerance Experiment
Benchmark Methodology
T
1 T2 T3
Time = T1 + T2 + T3
Experiment Results
Consistency Enforcement Experimen
Benchmark Methodology
T
1 T2 T3
Time = T1 + T2 + T3
One-broker case
Experiment Results
§
Overhead of consistency enforcement is few milliseconds.
Contributions
§
Systems Research
Ø
Hybrid Grid Information Service Architecture
vUnification, Federation and Interoperability of grid information services
vStrategies for high-performance, scalable in-memory storage vStrategies for efficient distribution, replica-content placement, consistency enforcement by utilizing pub-sub based messaging schemes
vSelf-adaptation to changing-client demands
vExtensions to semantics of UDDI and WS-Context Web Service Specifications
Ø
Detailed evaluation of the system components and algorithms
§
Systems Software
Ø
An implementation of Extended UDDI Specification
vGeographical Information Systems-specific, metadata-oriented
Ø
An implementation of WS-Context Specification
Future Research Directions
§
Use the proposed approach to solve OGF Grid
Interoperation Now (GIN) problem for information
services
§
Investigate an information security mechanism for
the decentralized Hybrid Service
Ø
