•
Three main issues in Grid environment:
–
Heterogeneity – multiplicity of resources
–
Scalability
–
Adaptability - Resource managers or applications
dynamic to extract the maximum performance
from the available resources and services.
Grid Middleware
•
System software between applications and
operating system
–
Provide services to application
• Discovery, storage, execution, information, service
integration, resource monitoring, failure detection and recovery,…
–
Hide heterogeneous of the Grid environment
–
Provide standardised interfaces to services.
Purposes of Middleware
•
Open, general-purpose and standard
•
Standard protocols
– Defines the contents and sequence of message exchanges used to request remote operation
– Important and essential to achieve the interoperability
that Grid depends on
•
Standard APIs
– Interfaces to code libraries
– Facilitate construction of Grid components by allowing code components to be reused
Component of Middleware
•
Connectivity layer
•
Resource layer
Connectivity layer
•
Communicating easily and securely
•
Communication protocol: exchange data
•
Authentication protocol: cryptographically secure
mechanism for verifying the identity of users and
resources.
– Single sign-on – Delegation
– Integration with various local security solutions – User-based trust relationships
Resource layer
•
Sharing single resources
•
Define protocols (and APIs and SDKs) for the secure
negotiation, initiation, monitoring, control,
accounting, and payment of sharing operations on
individual resources
•
Information protocols - obtain information about the
structure and state of a resource
•
Management protocols - negotiate access to a
shared resource
Collective layer
•
Coordinating multiple resources
•
Implement a wide variety of sharing behaviors
without placing new requirements on the resources
being shared
•
Example –
Directory services, co-allocation, scheduling, brokering services, Monitoring and diagnostics services, Data replication services, Grid-enabled programming systems,…
gLite Toolkit
•
The Enabling Grids for E-sciencE project (EGEE)
•
Middleware stack that combines components
gLite middleware
•
A layer between services
and resources
•
gLite follows a Service
Oriented Architecture
gLite -principles
•
Service oriented approach
– Allow for multiple interoperable implementations
•
Lightweight (existing) services
– Easily and quickly deployable
– Use existing services where possible
• Condor, EDG, Globus, LCG, …
•
Portable
– Being built on Scientific Linux and Windows
•
Security
gLite - principles
•
Performance/Scalability & Resilience/Fault Tolerance
– Comparable to deployed infrastructure
•
Co-existence with deployed infrastructure
– Co-existence with LCG-2 and OSG (US) are essential for the EGEE Grid services
•
Site autonomy
– Reduce dependence on ‘global, central’ services
gLite - Access
•
All of the gLite services are accessible via
APIs and CLIs.
–
API - Application Programming Interface
–
CLI - Command Line Interface
gLite - Security Services
• Authentication
– identify entities
– PKI (X.509) infrastructure with CA third party for signature
• Authorization
– allows or denies access to specific services
– three basic authorization models, classified as agent, push and
pull.
• Dynamic Connectivity Service
– tackle the problems that arise when connectivity is restricted by the resource owner.
• Auditing
– monitoring and providing for post-mortem analysis of security related events.
gLite - Infomation and Monitoring Services
•
Infomation and Monitoring Services a vital low-level
component of any grid
gLite - Infomation and Monitoring Services
•
Job Monitoring
– use the APIs offered by the apache logging services (log4j)
•
Service Discovery
– a facility offered to both end users and to other services to locate suitable services.
•
Network Performance Monitoring
– define a standard interface to monitoring frameworks. – NPM Publisher for use by gLite components.
gLite - Helper Services
•
Configuration and Instrumentation Service
– common, standard-based configuration and
instrumentation functionality to the gLite services.
•
Bandwidth Allocation and Reservation Service
– control and balance the usage of the network and categorise and prioritise traffic flows.
•
Agreement Service
– implements the communication protocol used to
exchange information about Service Level Agreements (SLAs) and defines the SLA structure.
gLite - Data Services
•
Catalog
– store information about the data and metadata
•
Storage Element
– is the Grid service responsible for saving/retrieving files to/from some data store which can provide a wide range of quality of service to the user.
•
Data movement
– provide scalable and robust managed data transfer between Grid sites
gLite - Job Management Services
•
Accounting
– accumulates information about the usage of Grid resources
•
Computing element
– representing a computing resource
– job management and other capabilities
•
Workload Management
– comprises a set of Grid middleware components
responsible for the distribution and management of tasks across Grid resources
gLite - Job Management Services
•
Job service
– keep track of the definition of submitted jobs, execution conditions and environment.
•
Package Manager
– is a helper service that automates the process of installing, upgrading configuring, and removing software packages from a shared area
Globus Toolkit
•
Evolution
•
Globus Toolkit version 2
•
Globus Toolkit version 3
•
Globus Toolkit version 4
Globus evolution
1997 GT2.0 2003 GT3.0 OGSA 2005 GT4.x WSRFGlobus Toolkit version 2
•
Community-based, open-architecture, open source
set of services and software libraries that support
Grids and Grid applications.
•
De facto standard for Grid computing
•
Focus on usability and interoperability
•
Middleware
– GT2 – Connectivity layer – GT2 - Resource layer
GT2 – Connectivity
Authentication protocol
•
Public-key based GSI protocols
– GSI - Grid Security Infrastructure
– authentication, communication protection, authorization
•
GSI builds on and extends the TLS protocols
– TLS - Transport Layer Security
– TLS - defines a protocol to provide privacy and data integrity between two communicating applications.
GT2 – Connectivity layer
Communication protocol
•
Internet layered protocol architecture
–
Internet layer - IP and ICMP
–
Transport - TCP, UDP
GT2 - Resource layer
•
A Grid Resource Information Protocol – base on
Lightweight Directory Access Protocol (LDAP)
•
HTTP-based GRAM protocol
– GRAM - Grid Resource Access and Management – allocation of computational resources
– monitoring and control of computation on resources
•
GridFTP - a management protocol for data access
GT2 - Collective layer
•
build on Globus Connectivity and Resource protocols
•
Meta Directory Service
– introduces Grid Information Index Servers (GIISs) – LDAP information protocol
• Access resource-specific GRISs to obtain resource state • GRRP used for resource registration.
•
DUROC resource co-allocation library provides an
SDK and API for resource co-allocation
Globus Toolkit version 3
•
Build on and significantly extend GT2 concepts and
technologies
•
Base on Open Grid Services Architecture
– Factoring of component behaviors – identify essential Grid functions and express
– Service orientation – enable a uniform treatment of all network entities
– Align with Web services – WSDL (Web Services Definition Language)
Globus Toolkit version 4
•
developed by The Globus Alliance
•
OGSA requires “stateful services”
•
WSRF – Web Service Resources Framework
–
developed by OASIS
–
WSRF provides the stateful services that OGSA
needs
Comparison
• Globus Toolkit - faster and more efficient
• gLite - sophisticated and developer friendly
• Document
– Globus documents - sketchy and missing
– gLite documents - sufficient, available
