The Grid

(1)

GT3

• Grid background

• Open Grid Services Architecture

• Open Grid Services Infrastructure

• Beyond OGSI: other OGSA services

• Globus Toolkit v3 implementation

• Early GT3 performance measurements

• Scientific and commercial perspectives

• Summary

(2)

The Grid

• Infrastructure (“middleware” & “services”) for establishing, managing, and evolving multi-

organizational federations

– Dynamic, autonomous, domain independent

– On-demand, ubiquitous access to computing, data, and services

• Mechanisms for creating and managing workflow within such federations

– New capabilities constructed dynamically and transparently from distributed services

– Service-oriented, virtualization

(3)

What is a Grid?

• Three key criteria

– Coordinates distributed resources …

– using standard, open, general-purpose protocols and interfaces …

– to deliver non-trivial qualities of service.

• What is not a Grid?

– A cluster, a network attached storage device, a scientific instrument, a network, etc.

– Each may be an important component of a Grid, but by itself does not constitute a Grid

(4)

eScience is the Initial Motivator …

• New approaches to enquiry based on

– Deep analysis of huge quantities of data – Interdisciplinary collaboration

– Large-scale simulation – Smart instrumentation

• Enabled by an infrastructure that enables access to, and integration of, resources &

services without regard for location

(5)

Resource Integration

as a Fundamental Challenge

R Discovery

Many sources of data, services, computation

R Registries organize

services of interest to a community

Access

Data integration activities

may require access to, & Exploration & analysis may involve complex, RM

RM

RM RM RM

Resource management is needed to ensure progress & arbitrate competing demands Security

service Security

service Policy

service Policy service

Security & policy

must underlie access

& management decisions

(6)

Distributed Computing

• The concept of sharing distributed resources is not new

– In 1965, MIT's Fernando Corbató and the other Multics OS designers envisioned a computer facility operating

“like a power company or water company”

– In their 1968 article “the computer as a communications device,” J. C. R. Licklider and Robert W. Taylor anticipated Grid-like scenarios

– Since the late 1960s, much work has been devoted to developing distributed systems, with mixed success

(7)

Overview

• Grid background

• Open Grid Services Architecture

• Early GT3 performance results

• Summary

(8)

Why Open Standards Matter

• Ubiquitous adoption demands open, standard protocols

– Standard protocols enable interoperability – Avoid product/vendor lock-in

– Enables innovation/competition on end points

• Further aided by open, standard APIs

– Standard APIs enable portability

– Allow implementations to port to different vendor platforms

• Internet and Web as exemplars

(9)

Increased functionality, standardization

Custom

Open Grid Services Arch

Real standards

Multiple implementations

Web services, etc.

Managed shared virtual systems

Computer science research

Globus Toolkit

Defacto standard Single implementation

Internet standards

The Emergence of

Open Grid Standards

(10)

Open Grid Services Architecture

• Service orientation to virtualize resources

– Everything is a service

• From Web services

– Standard interface definition mechanisms – Evolving set of other standards: security, etc.

• From Grids (Globus Toolkit)

– Service semantics, reliability & security models – Lifecycle management, discovery, other services

A framework for the definition & management of composable, interoperable services

(11)

Web Services

• XML-based distributed computing technology

• Web service = a server process that exposes typed ports to the network

• Described by the Web Services Description Language, an XML document that contains

– Type of message(s) the service understands & types of responses &

exceptions it returns

– “Methods” bound together as “port types”

– Port types bound to protocols as “ports”

• A WSDL document completely defines a service and how to access it

(12)

WSDL Example

<wsdl:definitions targetNamespace=“…”>

<wsdl:types>

<xsd:element name=“fooInput” …/>

<xsd:element name=“fooOutput” …/>

</schema>

</wsdl:types>

<wsdl:message name=“fooInputMessage”>

</wsdl:message>

<wsdl:message name=“fooOutputMessage”>

</wsdl:message>

<wsdl:portType name=“fooInterface”>

<wsdl:operation name=“foo”>

(13)

Transient Service Instances

• “Web services” address discovery & invocation of persistent services

– Interface to persistent state of entire enterprise

• In Grids, must also support transient services, created/destroyed dynamically

– Interfaces to the states of distributed activities – E.g. workflow, video conf., dist. data analysis

• Significant implications for how services are managed, named, discovered, and used

– In fact, much of our work is concerned with the management of services

(14)

OGSA Structure

• A standard substrate: the Grid service

– Standard interfaces and behaviors that address key distributed system issues: naming, service state, lifetime, notification

– A Grid service is a Web service

• … supports standard service specifications

– Agreement, data access & integration, workflow, security, policy, diagnostics, etc.

– Target of current & planned GGF efforts

• … and arbitrary application-specific services based on these & other definitions

(15)

Overview

• Grid background

• Open Grid Services Infrastructure

• Summary

(16)

OGSI Specification

• Defines WSDL conventions and extensions

– For describing and naming services

– Working with W3C WSDL working group to drive OGSI extensions into WSDL 1.2

• Defines fundamental interfaces (using extended WSDL) and behaviors that define a Grid Service

– A unifying framework for interoperability &

establishment of total system properties

• http://www.ggf.org/ogsi-wg

(17)

Fundamental

Interfaces & Behaviors

• OGSI defines basic patterns of interaction, which can be combined with each other and with custom patterns in a myriad of ways

• OGSI Specification focuses on:

– Atomic, composable patterns in the form of portTypes/interfaces

• Define operations & associated service data elements

– A model for how these are composed

• Compatible with WSDL 1.2

• Complete service descriptions are left to other groups that are defining real services

(18)

OGSI: Standard Web Services Interfaces & Behaviors

• Naming and bindings (basis for virtualization)

– Every service instance has a unique name, from which can discover supported bindings

• Lifecycle (basis for fault resilient state management) – Service instances created by factories

– Destroyed explicitly or via soft state

• Information model (basis for monitoring & discovery) – Service data (attributes) associated with GS instances – Operations for querying and setting this info

– Asynchronous notification of changes to service date

• Service Groups (basis for registries & collective svcs) – Group membership rules & membership management

• Base Fault type

(19)

OGSI Service Data

• Attributes: Publicly visible state of the service

• Want to bring full power of XML to attributes

– getXXX/setXXX is too limiting

• How to get/set multiple?

• Want richer queries across attributes (e.g., join)

– Use XML Schema, XPath, XQuery, XSLT, etc.

– OGSI service data:

• Attributes defined using XML Schema

• Attributes combined into a single (logical) document within the service

• Rich pull/push/set operations against service data document

• Should declare attributes in WSDL interface

(20)

Open Grid Services Infrastructure

Implementation

Service data element

Other standard interfaces:

factory, notification, collections

Service data element

GridService (required)

Data access

Lifetime management

• Explicit destruction

• Soft-state lifetime Introspection:

• What port types?

• What policy?

• What state?

Client

Grid Service Handle

Grid Service Reference

handle resolution

(21)

Service registry

Service requestor (e.g. user

application) Service

factory

Create Service

Grid Service Handle Resource

allocation

Service instances

Register Service

Service discovery Service data Keep-

alives Notifications Service invocation Authentication &

authorization are applied to all requests

Open Grid Services Infrastructure (OGSI)

(22)

GWSDL

• OGSI requires interface extension/composition

• We worked within W3C WSDL working group to define standard interface extension in WSDL 1.2 that meets OGSI requirements

• But could not wait for WSDL 1.2

• So defined gwsdl:portType that extends WSDL 1.1 portType with:

– WSDL 1.2 portType extension – WSDL 1.2 open content model

• Define GWSDL  WSDL 1.1 & 1.2 mappings

(23)

GWSDL Example

<wsdl:definitions>

<wsdl:types>…</wsdl:types>

<wsdl:message>…</wsdl:message>

…

<gwsdl:portType name=“foo”

extends=“ns:bar ogsi:GridService”>

<wsdl:operation name=“op1”>…</wsdl:operation>

<wsdl:operation name=“op2”>…</wsdl:operation>

<ogsi:serviceData … />

</gwsdl:portType>

…

</wsdl:definitions>

(24)

Example:

Reliable File Transfer Service

Performance Policy Faults

service data elements

Pending

File Transfer

Internal State Grid

Service

Notf’n Source

Policy

interfaces

Query &/or subscribe to service data Fault

Monitor

Perf.

Monitor

Client Client Client

Request and manage file transfer operations

(25)

OGSI Implementations

• Globus Toolkit version 3.0 (Java, C client)

• U Virginia OGSI.NET (.NET)

• LBNL pyGlobus (Python)

• U Edinburgh (.NET)

• U Manchester (PERL)

• Fujitsu Unicore (Java)

(26)

Overview

• Grid background

• Beyond OGSI: other OGSA services

(27)

Web Services: Basic Functionality

OGSA

Open Grid Services Architecture

OGSI: Interface to Grid Infrastructure Applications in Problem Domain X

Compute, Data & Storage Resources

Application & Integration Technology for Problem Domain X Users in Problem Domain X

Generic Virtual Service Access and Integration Layer

Structured Data Integration

Structured Data Access

Structured Data

Transformation Registry

Job Submission

Data Transport Resource Usage Banking

Brokering Workflow

Authorisation

(28)

OGSA Standardization

& Implementation

• OGSI defines core interfaces and behaviors for manageable services

– Supported by strong open source technology & major commercial vendors

• Efforts are underway within GGF, OASIS, and other bodies to define standards for

– Agreement negotiation

– Common management model – Data access and integration – Security and policy

– Etc., etc., etc.

(29)

Transactions & Contexts

• WS-Coordination & WS-Transaction

– IBM/MS (not in standards org)

• WS-CAF (Coordinated Application Framework)

– Sun/Oracle/Arjuna/Fujitsu (not in standards org) – WS-CTX (Context)

– WS-CF (Coordination Framework) – WS-TXM (Transaction Management)

• Both take a “contextualization” approach

– Context (id) threaded through SOAP header

– OGSI for context creation, naming & lifecycle???

(30)

Security Standards

• Many core security standards are from IETF

– X.509, Kerberos, etc.

– X.509 Proxy Certificates (RFC soon hopefully)

• Used by Globus Toolkit GSI

• OASIS appears to be leader in Web services security standards

– WS-Security: SOAP message security – SAML: signed assertions using XML – XACML: access control lists using XML

• GGF OGSA Security WG evaluating security specifications for applicability to OGSA

(31)

IBM/Microsoft

WS Security Architecture

• Large set of specifications for doing Web services security, most of which should be appropriate for OGSA

• Announced April 2002

• Initial spec in July 2002 (WS-Security)

– Submitted to OASIS

• New crops of specs arrive periodically

– WS-Policy*, WS-Trust, WS-Federation, etc.

– But… Not yet in any standards organization

(32)

WS Security

Current/Proposed WSS-specs

SOAP Foundation WS-Security

WS-Policy WS-Trust WS-Privacy

WS-Secure

Conversation WS-Federation WS-Authorization

In progress

(33)

OASIS SAML & XACML

• SAML: Security Assertion Markup Language

– Good for asserting properties such as group membership, etc

• XACML: eXtensible Access Control Markup Language

– For defining access control policies

• These are gaining considerable momentum, but WS-Policy* leaves them in question

(34)

Project Liberty Alliance

• V1.x specifications for identity federation

– Allows cross-organization identification – Privacy preserving model

• Based on SAML

(35)

WS-Agreement

• Recall key criteria of a Grid:

– Coordinates resources that are not subject to centralized control …

– using standard, open, general-purpose protocols and interfaces …

– to deliver non-trivial qualities of service.

• Implies need to express and negotiate agreements that govern the delivery of services to clients

– Agreement = what will be done, QoS, billing, compliance monitoring

(36)

WS-Agreement Contents

• Standard agreement language

– A composition of a set of terms that govern a service’s behavior with respect to clients

– Agreement language uses WS-Policy (currently)

– Standard attributes for terms that express current state of negotiation

– Other groups define specific terms

• Standard agreement negotiation protocol

– Establish, monitor, re-negotiate agreement – Expressed using OGSI GWSDL interfaces – Each agreement represented by a service

(37)

WS-Agreement Applicability

• All interesting Web/Grid services interactions will be governed by agreements!

• WS-Agreement (language and interfaces) should be used by specifications that define domain-

specific services

– Data services – Job submission

– Specialized services – Etc.

(38)

Platform’s Community Scheduling Framework & WS-Agreement

Internet

Queuing Service

GT3.0

Job Service Reservation

Service

RM Adapter

GT3.0

LSF

Site A

Site B

RM Adapter

GT3.0

PBS

Site C

(39)

WSDM / WSMF / CMM

• OASIS Web Services Distributed Management (WSDM) technical committee

– Management using/of Web Services

– HP submitted its Web Services Management Framework (WSMF) to WSDM in July 2003

• WS-Events: event schema, subscription, message queues

• WSMF-Foundation: management using Web services

• WSM: management of Web services

• GGF Common Management Model (CMM) WG

– IBM submission overlaps WSMF-Foundation

• Working to bring WSDM & CMM together, based on OGSI foundation

(40)

Data as Service:

OGSA Data Access & Integration

• Service-oriented treatment of data appears to have significant advantages

– Leverage OGSI introspection, lifetime, etc.

– Compatibility with Web services

• Standard service interfaces being defined

– Service data: e.g., schema

– Derive new data services from old (views) – Externalize to e.g. file/database format

(41)

Data Services

• GGF Data Access and Integration Svcs (DAIS)

– OGSI-compliant interfaces to access relational and XML databases

– Needs to be generalized to encompass other data sources (see next slide…)

• Generalized DAIS becomes the foundation for:

– Replication: Data located in multiple locations – Federation: Composition of multiple sources – Provenance: How was data generated?

(42)

“OGSA Data Services”

(Foster, Tuecke, Unger, eds.)

• Describes conceptual model for representing all manner of data sources as Web services

– Database, filesystems, devices, programs, … – Integrates WS-Agreement

• Data service is an OGSI-compliant Web service that implements one or more of base data interfaces:

– DataDescription, DataAccess, DataFactory, DataManagement

– These would be extended and combined for specific

(43)

1a. Request to

Registry for sources of data about “x”

1b. Registry responds with Factory handle

2a. Request to Factory for access to database

2c. Factory

returns handle of GDS to client

3a. Client queries GDS with XPath, SQL, etc

3c. Results of query returned

SOAP/HTTP service creation

API interactions

Registry

Factory

2b. Factory creates

GridDataService to manage access

Grid Data Service Client

XML /

Relational database

Data Access & Integration Services

(44)

Overview

• Grid background

• Globus Toolkit v3 implementation

(45)

The Globus Alliance

• A group of people with a common mission:

– “Make Grid computing an everyday reality.”

• Argonne/U.Chicago, USC-ISI, EPCC, & KTH-PDC

– Led by Ian Foster (Argonne), Carl Kesselman (ISI), Malcolm Atkinson (EPCC), Lennart Johnsson (PDC)

– Includes researchers, software developers, software architects & designers, systems engineers, & others

– Collaborations (or at least acquaintances) with most Grid activities in the world

• All activities contribute to our common mission

– Research

– Software development (prototypes, reference implns) – Application consulting

– Infrastructure consulting

(46)

Globus Toolkit:

A Story of Evolution

• Definition of Grid problem has been stable since original Globus Project proposal in 1995

– Though we’ve gotten better at articulating it

• But our approach to its solution has evolved:

– From APIs and custom protocols…

– to standard protocols…

– to Grid services (OGSA)

• Driven by experience implementing and deploying the Globus Toolkit, and building real applications with it

(47)

Globus Toolkit

^®

v3.0

• All of the GT v2.4 services and clients

• Complete Java implementation of OGSI v1.0

– Rich, container-based implementation – Built on Apache Axis

• Globus “proprietary” services built on OGSI

– Managed Jobs (akin to GT2 GRAM) – Reliable File Transfer (RFT)

– Index Services (akin to GT2 GIIS)

• Some services not yet OGSI-fied:

– GridFTP, Replica Location Services (RLS)

(48)

GT2

Components RLS

GT-OGSA Grid Service Infrastructure

GT3 Distribution

(49)

Security Infrastructure System-Level Services

Base Services

User-Defined Services Grid Service Container

Web Service Engine OGSI Spec Implementation

(50)

Base Services

(51)

OGSI Implementation

• GT3 includes a set of primitives that fully implement the interfaces and behaviors defined in the OGSI Specification

– Defines how entities can create, discover and interact with a Grid service

• The OGSI Specification defines a protocol:

GT3 provides a programming model for that protocol

(52)

Implementation of the GridService

portType: GridServiceImpl and PersistentGridServiceImpl

• destroy()

• setServiceData()

• findServiceData()

• requestTerminationAfter()

• requestTerminationBefore()

• addOperationProvider()

(See docs for complete set of methods)

Implementation of the OGSI Spec

Additional

functionality can be added to a

Grid Service using OperationProviders

Deployment descriptor

(53)

Building an OGSI-Compliant Grid Service using GT3

Write service-specific logic that also

implements the GT3 OperationProvider

(54)

Combine with one of the two GT3 implementations of base GridService

functionality:

(55)

functionality:

An

OGSI-Compliant grid service

(56)

An

OGSI-Compliant grid service OperationProviders are

configured at deployment time or added at runtime

functionality:

(57)

A Grid Service Can be Composed of Multiple OperationProviders

• OPs can be designed as atomic bits of functionality to facilitate reuse

• OP approach eases the task of bringing legacy code into OGSI- compliance

• OPs allow Grid Services to be formed dynamically (in contrast to the inheritance approach)

(58)

Several OperationProviders

are Included in the GT3 Distribution

NotificationSourceProvider

HandleResolverProvider

ServiceGroupRegistrationProvider ServiceGroupProvider

FactoryProvider

(59)

OGSI Specification: Notifications

• Our NotificationSourceProvider implementation allows any Grid Service to become a sender of notification messages

• A subscribe request on a NotificationSource triggers the creation of a NotificationSubscription service

• A NotificationSink can receive notification msgs from NotificationSources

– Sinks are not required to implement the GridService portType

• Notifications can be set on SDEs

(60)

OGSI Specification: Factory

Factory portType

• Factories create services

• Factories are typically persistent services

• Factory is an optional OGSI interface

(Grid Services can also be instantiated by other mechanisms)

(61)

OGSI Specification: Service Groups

Service group portTypes

• A ServiceGroup is a grid service that maintains information about a group of other grid services

• The classic registry model can be implemented with the ServiceGroup portTypes

• A grid service can belong to more than one ServiceGroup

• Members of a ServiceGroup can be heterogeneous or homogenous

• Each entry in a service group can be represented as its own service

• Service group portTypes are optional OGSI interfaces

(62)

OGSI Specification: Handle Resolver

HandleResolver portType

• Defines a means for resolving a GSH (Grid Service Handle) to a GSR (Grid Service Reference)

– A GSH points to a Grid Service

(GT3 uses a hostname-based GSH scheme)

– A GSR specifies how to communicate with the Grid Service

(GT3 currently supports SOAP over HTTP, so GSRs are in WSDL format)

Includes 0 or more Grid Service Handles (GSHs) Service locator

(63)

A Service Creation Scenario*

Registry

1. From a

known registry, the client

discovers a factory by querying the service data of the

registry

Client

(64)

Client Registry

2. The client calls the createService operation on the factory

Factory

1. From a

registry

A Service Creation Scenario

(65)

Client Registry

Factory

Service

3. The factory creates a service 1. From a

registry

2. The client calls the createService operation on the factory

A Service Creation Scenario

(66)

Client Registry

Factory

Service

4. The factory

returns a locator to the newly created service

1. From a

registry

2. The client calls the createService operation on the

factory 3. The

factory creates a service

A Service Creation Scenario

(67)

Client Registry

Factory

Service

5. The client and 1. From a

registry

2. The client calls the createService operation on the

factory 3. The

factory creates a service

A Service Creation Scenario

4. The factory

returns a locator to the newly created service

(68)

NotificationSink

A Notification Scenario

1. NotificationSink calls the subscribe operation on NotificationSource

NotificationSou rce

(69)

2.NotificationSourc e creates a

subscription service 1. NotificationSink calls

the subscribe operation on NotificationSource

A Notification Scenario

NotificationSink

NotificationSou rce

Notification Subscription

(70)

3. Notification Source returns a

locator to the subscription service 1. NotificationSink calls the subscribe operation on NotificationSource

subscription service

A Notification Scenario

NotificationSink

NotificationSou rce

(71)

4.b The NotificationSink and Subscription service interact 4.a deliverNotification

stream continues for the lifetime of

NotificationSubscription

locator to the subscription service

A Notification Scenario

NotificationSink

NotificationSou rce

(72)

NotificationS ink

NotificationS ource

NotificationS ubscription

subscribe

locator to the subscription service

4.b The NotificationSink and 4.a deliverNotification

stream continues for the lifetime of

NotificationSubscription

The sole mandated cardinality: 1 to 1

A Notification Scenario

(73)

Base Services

(74)

• Transport Layer Security/Secure Socket Layer (TLS/SSL)

– To be deprecated

• SOAP Layer Security

– Based on WS-Security, XML Encryption, XML Signature

• GT3 uses X.509 identity certificates for authentication

• It also uses X.509 Proxy certificates to support

Security Infrastructure

(75)

Base Services

(76)

System-Level Services

• General-purpose services that facilitate the use of Grid services in production environments

• The 3.0 distribution includes three system-level services

– An administration service – A logging service

– A management service

(77)

Base Services

(78)

Grid Service Container

Includes the OGSI Implementation, security infrastructure and system-level services, plus:

• Service activation, deactivation, construction, destruction, etc.

• Service data element placeholders that allow you to fetch service data values dynamically, at query time

• Evaluator framework (supporting ByXPath and ByName notifications and queries)

• Interceptor/callback framework (allows one to

(79)

Grid Service Container (cont.)

Interface Layer

Transport Layer

Implementation Layer

Layers in the Web Services Model

OGSI Spec is here

Transport/binding layer (e.g., GT3:

SOAP over HTTP) Container is here

(80)

Base Services

(81)

Hosting Environment

GT3 currently offers support for four Java hosting environments:

• Embedded

• Standalone

• Servlet

• EJB

(82)

Virtual Hosting Environment Framework

• Virtual hosting allows Grid services to be

distributed across several remote containers

• Useful in implementing solutions for

problems common to distributed computing

– Load balancing

– User account sandboxing

(83)

Base Services User-Defined Services Grid Service Container

(84)

MMJFS

Resource Management

• GRAM Architecture rendered in OGSA

• The MMJFS runs as an unprivileged user, with a small highly-constrained setuid

executable behind it

• Individual user environments are created using virtual hosting

MJS MJS MJS

User 1

User 2 Master User

MJS

MMJFS: Master

Managed Job Factory Service

MJS: Managed Job Service

(85)

Client

GRAM Job Submission Scenario

Index Service

1. From an index service, the client chooses an MMJFS

(86)

Client

GRAM Job Submission Scenario

Index

Service 2. The client calls the createService

operation on the factory and supplies RSL

MMJFS

(87)

Client

GRAM Job Submission Scenario

Index

Service MMJFS

MJS

3. The factory creates a Managed Job Service

2. The client calls the createService

operation on the factory and supplies RSL

(88)

Client

GRAM Job Submission Scenario

Index

Service MMJFS

MJS

4. The factory returns a locator 1. From an

index service, the client chooses an MMJFS

operation on the factory

and supplies RSL 3. The factory

creates a Managed Job Service

(89)

Client

GRAM Job Submission Scenario

Index

Service MMJFS

MJS

5. The client subscribes to 1. From an

index service, the client chooses an MMJFS

operation on the factory

and supplies RSL 3. The factory

creates a Managed Job Service

4. The factory returns a locator

(90)

Information Services

• Index service as caching aggregator

– Caches service data from other Grid services

• Index service as provider framework

– Serves as a host for service data providers that live outside of a Grid service to publish data

(91)

Reliable File Transfer

• OGSI-compliant service exposing GridFTP control channel functionality

– 3rd-party transfer between GridFTP servers

• Recoverable Grid service

– Automatically restarts interrupted transfers from the last checkpoint

• Progress and restart monitoring ^GridFTP

Server 1 RFT

(92)

Reliable File Transfer Service

Performance Policy Faults

service data elements

Pending

File Transfer

Internal State Grid

Service

Notf’n Source

Policy

interfaces

Query &/or subscribe to service data Fault

Monitor

Perf.

Monitor

Client Client Client

Request and manage file transfer operations

(93)

Base Services

(94)

GT3 User-Defined Services

• GT3 can be viewed as a Grid service development kit that includes:

– Primitives, APIs, tools and runtime services designed to ease the task of building OGSI- compliant services

– Primitives for provisioning security

– Base services that provide an infrastructure with which to build higher-level services

(95)

GT3 User-Defined Services (cont.)

ANT User source files

GT3 build files

User build file

Grid service executable files

(Diagram inspired by Borja Sotomayor’s excellent GT3 tutorial)

(96)

Overview

• Grid background

• Early GT3 performance results

(97)

Early GT3 Performance Analysis

• Goal: explore GT3 under heavy load

– Maximal throughput/rate of GT3 services – Identify limiting factors

• Three GT3 Grid services measured

– GRAM

– DummyService – IndexService

• Note: Preliminary results, performance continues to improve rapidly

• Work performed by Massimo Lamanna, David Foster, et al., at CERN

(98)

GT3 GRAM Performance: Setup

• GRAM in GT3 standalone container

• Managed-job-globusrun clients started

simultaneously on up to 32 client nodes (lxplus) in non-batch mode used to submit jobs to GT3 GRAM

• GRAM hardware: 2 * Intel Pentium III 600MHz processors, 256MB RAM

• Note: 1 managed-job-globusrun client is able to

(99)

GT3 GRAM performance

• Results: service node

– Saturation throughput for job submission on the service node:

3.8 jobs/minute with an average CPU user+system usage of 62%

– –

• Comments:

– Scalability issue for heavily used servers

(100)

GT3 GRAM performance

• Results: client node

– Using a 2 * Intel Pentium III 600MHz processors, 256MB RAM client node, a managed-job-

globusrun client consumes at average 16 seconds CPU user+system time (on both CPU’s) for 1 job

• Comment:

– Lightweight clients (e.g., written in "C") needed

(101)

DummyService performance

• Setup (1)

– Each DummyService client executes these steps:

1. Calls DummyServiceFactory to create a DummyService instance

2. Executes 2 simple methods (echo and getTime) on the DummyService instance 3. Calls DummyService instance to destroy itself

– Up to 1000 clients talking to the DummyService from up to 45 client nodes (lxplus)

– With & without GSI message level authentication

– Grid service node hardware: 2 * Intel Pentium III 600MHz processors, 256MB RAM

• Setup (2)

– Same, but repeats step #2 one hundred times

(102)

DummyService performance

• Preliminary Results

• Security overhead needs further investigation

• Cross check our implementation/setup with Globus team foreseen

setup authentication

service container

saturation throughput

average CPU u+s usage, %

1 - GT3 standalone 41 services/s 89

1 yes GT3 standalone 1.3 services/s 88

1 - Tomcat 60 services/s 89

1 yes Tomcat 1.2 services/s 88

2 - GT3 standalone 300 method calls/s 96

2 yes GT3 standalone 10 method calls/s 72

2 - Tomcat 290 method calls/s 96

2 Yes Tomcat 13 method calls/s 79

(103)

DummyService Performance

• Conclusions

– Security overhead needs further investigation – More tests on more powerful machines

– Container: depending on the setup the Tomcat container may be a bit slower or up to 50% faster, compared to the

standalone container

• Notes:

– Results table shows top saturation rates

– With varying number of clients throughput goes down by up to 30% and the average CPU u+s usage varies accordingly

(104)

(105)

• The first time the client contacts the DummyServiceFactory, creates a

DummyService instance, and calls the first method, it takes about 10s to accomplish it

• These actions take about 1s on subsequent

(106)

IndexService performance

• Setup (1): IndexService acting as a notification source (pushing data)

– Multiple notification sinks subscribe to the IndexService "Host" Service Data Element (SDE), and are notified about each update of "Host" SDE,

happening at a fixed rate – No security

• Setup (2): IndexService responding to findServiceData requests (pulling data)

– Multiple ogsi-find-service-data clients are run sequentially and in parallel asking for IndexService "Host" Service Data Element

– No security

(107)

IndexService Performance

setup service container

saturation throughput

average CPU u+s

usage, % 1 GT3 standalone 10-15 notifications/s 81 – 87

1 Tomcat -

2 GT3 standalone 200 requests/s 88

2 Tomcat 200 requests/s 90

(108)

IndexService Performance

• Saturation throughput with findServiceData is ~13-20 times higher than with notifications

• Setup (1) measurement using Tomcat failed due to a thread bug, is fixed, fix announced to appear in next (4.1.28) Tomcat version

• Preliminary measurement with a faster service node (2 * Intel(R) Xeon(TM) 2.40GHz processors, 1GB RAM):

– Saturation throughput for setup (1) was about 32 notifications/s for 800 listeners compared to 10 notifications/s with 400

listeners on the 2 * 600MHz machines – not quite 4 times faster

(109)

(110)

(111)

Overview

• Grid background

• Scientific and commercial perspectives

• Summary

(112)

Scientific Perspectives

• Importance of Grid technologies growing with

“eScience” & “cyberinfrastructure”

• OGSA offers big opportunities

– Leverage commercial technologies

– Achieve collaborative software ecosystem

• But also concerns

– Potentially competing demands of industry – Sometimes slow pace of standards definition – Critical mass of R, D, and support

(113)

NEESgrid Earthquake Engineering Collaboratory

Network for Earthquake Engineering Simulation

Field Equipment Laboratory

Equipment Remote Users

High- Performance

Network(s) Instrumented

Structures and Sites

Leading Edge Computation Curated Data

Repository (Faculty, Students, Practitioners)

U.Nevada Reno

www.neesgrid.org

(114)

NEESgrid Core Capabilities

• Tele-control and tele-observation of experiments

• Data cataloging and sharing

• Remote collaboration and visualization tools and services

• Simulation execution and integration

(115)

NEESgrid High-level Structure

S h a k e ta b le w ith in s tru m e n ta tio n

DataGeneral

P O P

L o c a l c o m p u te rs

& s to ra g e

N E E S E q u ip m e n t S ite

E d g e R o u te r

W id e A re a N e tw o rk

E q u ip m e n t S ite

U s e r S ite

N E E S g rid O p e ra tio n s

R e s o u rc e S ite G ig a b it E th e rn e t > G b /s W A N

(116)

Architecture of NEESgrid

Equipment Site

(117)

Tele-Control Services

• Transaction-based protocol and service (NTCP) to control physical experiments and computational simulations.

• OGAI based implementation (GT3.0)

• Plug-ins to interface the NTCP service

– A computational simulation written in Matlab – Shore Western control hardware

– MTS control hardware (via Matlab and xPC)

• Security architecture, including GSI authentication and a flexible, plug-in-based authorization model.

(118)

Multi-site Online Simulation Test (MOST)

• A large-scale experiment conducted in multiple geographical locations.

– Combine physical experiments with numerical simulation

• Psuedo-dynamic testing

– Ideal for network latency reasons

(119)

Substructure Pseudo-dynamic Testing

Structural Test

Structural FE Simulation 1

Physical Tests

Physical Tests Numerical SimulationNumerical Simulation