Grid of Grids Information Management Kick off Meeting

(1)

Grid of Grids Information

Managemen

Kick off Meeting

Anaba

(2)

ACKNOWLEDGMENTS

We are grateful to:

• Bill McQuay,

• Raymund Garcia

(3)

General Goal

• Build Net Centric Core Enterprise Services

in fashion compatible with GGF/OGF and

Industry

• Add key additional services including those

for sensors and GIS

• Support Systems of Systems by federating

Grids of Grids supporting a heterogeneous

software production model allowing DoD

greater sustainability and choice of vendors

• Build tool to allow easy construction of Grids

(4)

Gerstner and Commercialization of GoG Architecture

• Reference [1]: Who Says Elephants Can't Dance, Louis Gerstner, Jr., 2002 is practically

a primer on IBM's vision and strategy to re-gain its influence to setting IT industry

agenda.

• "The industry's disaggregation into thousands of niche players would make IT integration

services a huge growth segment of the industry." [1]

–

Grid of Grids builds the information management infrastructure for efficient

integration of components

.

• "Over the next decade, customers would increasingly value companies that could

provide solutions - solutions that integrated technology from various suppliers and more

importantly, integrated technology into the processes of an enterprise." and "The two

companies that have enjoyed the highest market valuation in the IT industry in over

much of the last decade have been component manufacturers -- Intel and Microsoft.

There is no doubt that a strategy built around providing fundamental building blocks of

the computing infrastructure has proven to be extremely successful in this industry." [1]

–

Grid of Grids has huge commercial potential by offering both the integration

infrastructure for component manufacturers and providing Network-Centric

Enterprise Services as essential building blocks .

• "The dazzlingly wired-world would also change the course of computing in profound

ways. For one thing, it was virtually certain that world would be built on open industry

standards. There would be no other way to fulfill the promise of ubiquitous connections

among all the businesses users, devices, and systems that would participate in a truly

networked world. If that standards-based world came to pass, it would represent a major

shift in the prevailing competitive landscape." [1]

(5)

Technical Advisory Board

• Build a group of some 6-10 knowledgeable people

to guide project

• Mainly from Industry including Boeing, Lockheed

Martin, Ball Aerospace, Aerospace Corporation,

Mitre, DISA, and the core technology companies

(IBM, Microsoft)

• Use to guide both technology and

commercialization strategy

• Meet F2F, email and telecons.

• Brings significant external expertise to Grid of

(6)

Database

S S

S

S SS SS SS SS SS SS SS SS

F

S

F

S

F

S

F

S

F

S

F

S

F

S

F

S

F

S

F

S

F

S

F

S

F

S

F

S

F

S

F

S

F

S

F

S

F

S

Por

tal

F

S

O

S

O

S

O

S

O

S

O

S

O

S

O

S

O

S

O

S

O

S

O

S

O

S

MD

MetaData

Filter Service

Sensor Service

Other

Service

Anothe

Grid

Raw Data



Data



Information



Knowledge



Wisdom

Decisions

S S S S

Anothe

Service

Anothe

Service

S S

Anothe

Grid

S S

(7)

The Grid and Web Service Institutional Hierarchy

OGSA

GS-*

and some WS-*

GGF/W3C/…

XGSP (Collab)

WS-*

fro

OASIS/W3C

Industry

Apache Axi

.NET etc.

Must set standards to get interoperability

2: System Services and Features

**(WS-* from OASIS/W3C/Industry)**

Handlers like WS-RM, Security, UDDI Registry

3: Generally Useful Services and Features

(OGSA and other GGF, W3C)

Such as

“Collaborate”, “Access a Database” or “Submit a Job”

4: Application or Community of Interest (CoI

Specific Services

such as “Map Services”, “Run

BLAST” or “Simulate a Missile”

1: Container and Run Time (Hosting)

Environment (Apache Axis, .NET etc.)

XBM

XTCE VOTABLE

CML

(8)

**The Ten areas covered by the 60 core WS-* Specifications**

WSRP (Remote Portlets)

10: Portals and User Interfaces

WS-Policy, WS-Agreement

9: Policy and Agreements

WSDM, WS-Management, WS-Transfer

8: Management

WSRF, WS-MetadataExchange, WS-Context

7: System Metadata and State

UDDI, WS-Discovery

6: Service Discovery

WS-Security, WS-Trust, WS-Federation, SAML,

WS-SecureConversation

5: Security

BPEL, WS-Choreography, WS-Coordination

4: Workflow and Transactions

WS-Notification, WS-Eventing (Publish-Subscribe)

3: Notification

WS-Addressing, WS-MessageDelivery; Reliable

Messaging WSRM; Efficient Messaging MOTM

2: Service Internet

XML, WSDL, SOAP

1: Core Service Model

(9)

Activities in Global Grid Forum Working Groups

Authorization, P2P and Firewall Issues, Trusted Computing

7: Security

Resource/Service configuration, deployment and lifetime, Usage

records and access, Grid economy model

6: Management

Network measurements, Role of IPv6 and high performance

networking, Data transport

5: Infrastructure

Database and File Grid access, Grid FTP, Storage Management, Data

replication, Binary data specification

and interface, High-level

publish/subscribe, Transaction management

4: Data

Job Submission, Basic Execution Services, Service Level Agreements

for Resource use and reservation, Distributed Scheduling

3: Compute

Software Interfaces to Grid, Grid Remote Procedure Call,

Checkpointing and Recovery, Interoperability to Job Submittal services,

Information Retrieval,

2: Applications

High Level Resource/Service Naming (level 2 of slide 6),

Integrated Grid Architecture

1: Architecture

(10)

Net-Centric Core Enterprise Services

Provisioning,

operations

and

maintenance

of

applications.

NCES9: Application

Retention, organization and disposition of all forms of

data

NCES8: Storage

Includes automated and manual methods of optimizing

the user GiG experience (user agent)

NCES7: User Assistance

Provision and control of sharing with emphasis on

synchronous real-time services

NCES6: Collaboration

Includes

translation,

aggregation,

integration,

correlation, fusion, brokering publication, and other

transformations for services and data. Possibly agents

NCES5: Mediation

Searching data and services

NCES4: Discovery

Synchronous or asynchronous cases

NCES3: Messaging

Supports confidentiality, integrity and availability.

Implies reliability and autonomic features

NCES2: Information

Assurance (IA)/Security

including life-cycle management

NCES1: Enterprise Services

Management (ESM)

(11)

Produce the Needed Core Services

• We can classify services in many ways and

following 2 charts are one way; slightly changed

from proposal as NCOW and our work changed a

little.

• Green

is “in hand”; our team leaders

• Orange

is “in hand” with outside but available

solutions

• Red

has problems –

Security

does not have

industry consensus while current

Scheduling

(12)

The Core

F

eatures/

S

ervice Areas I

ECS

WS9

FS10: Policy

CIM

NCES1

GS6

WS8

FS9: Management

Globus MDS

Semantic Grid, WS-Context

WS7

FS8: System Metadata &

State

UDDI

NCES4

WS6

FS7: Discovery

Grid-Shib, Permis Liberty Alliance ...

NCES2

GS7

WS5

FS6 : Security

Grid Programming

NCES5

WS4

FS5 Workflow

JMS, MQSeries.

NCES3

WS3

FS4: Notification

Streams/Sensors. Team

NCES3

WS2

FS3: Service Internet,

Messaging

B: Core Services

Distinctive Strategy

for legacy subsystems and

modular architecture

FS2: Grid of Grids

Core Service Architecture

, Build Grids on Web

Services. Industry best practice

WS1

FS1: Use SOA: Service

Oriented Arch.

A: Broad Principles

Comments

NCES

(DoD)

GS-*

(13)

The Core

F

eature/

S

ervice Areas II

Current work only addresses scheduling “batch

jobs”. Need networks and services

GS3

FS18: Scheduling and

matching of Services and

Resources

XGSP, Shared Web Service ports

NCES6

GS7

FS17: Collaboration and

Virtual Organizations

Ad-hoc networks

GS5

FS16: Resources and

Infrastructure

Standalone Services

Proxies for jobs

NCES9

GS2

FS15: Applications and User

Services

JBI for DoD, WFS for OGC

GS4

FS14: Information

NCOW Data Strategy

Federation at data/information layer major

research area; CGL leading role

NCES8

GS4

FS13: Data and Storage

GS3

FS12: Computing

Portlets JSR168, NCES Capability Interfaces

NCES7

WS10

FS11: Portals and User

assistance

B: Core Services (Continued)

Comments

NCES

GS-*

WS-*

(14)

Additional Services

• Sensors

have low level support listed as FS3; higher

level integration using SensorML and Filters well

understood. Some work in phase I

• GIS Grid services

pioneered by team and already

shown in phase I

• Mediation (Interoperability) Services

needed to link

Grids (defined as a collection of ≥ 1 Services)

–

Need to generalize existing solutions for Sensor Grids

and for MQSeries-SOAP Mediation

(15)

Grid of Grids Building Tool

• This will provide a graphical interface to build grids

from existing libraries of Services and Grids

• Meta-data (provenance) needs to be specified

• Grids (services) need to be linked

• This will be built by extending an existing workflow

engine which is aimed at a more tightly couple

version of the builder problem

–

We will consider HPSearch (CGL), Taverna (Open

source from UK OMII), BPEL with user interface (OMII

or IU LEAD project)

–

Semantic Grid provenance “add-ons”

(16)

Applications

• Indiana University will apply to

Earthquake Science

and

Emergency Response

• Ball Aerospace will apply to

Matrix

• Anabas will work with Technical Advisory Board to

identify commercialization plan and possible

prototype applications in first 12 months

(17)

Out of Scope for Phase II

• Many areas are still evolving significantly

–

Mediation/Interoperation

–

Security

–

Scheduling of non-compute Resources

–

Data/Information Federation

–

Semantic Grid and management

• We will not test scalability on large number of services,

sensors and component Grids

• Integrating legacy systems not addressed

(18)

Research Tasks

The R&D Effort is divided into five major tasks:

• Task 1: Implementation of Collaboration Grid Middleware

• Task 2: Enhanced NCOW Core Enterprise Services (NCES)

with Enterprise Control Services and Metadata Services

• Task 3: Design and implementation of Grid of Grids

mediation

algorithms and NCOW services

• Task 4: Design and Implementation of Net-Centric

Collaboration Grid Builder Tool

(19)

Task Organization

•Requirements

•Analysis

•Design

•Implement

(20)

Task Assignments

Task Anabas Ball CGL

Requirements

Collaboration Grid Middleware (NCGGCM) (P) (S) (S)

NCOW, GiG (P) (S)

CES+ (11 service layers) (P) (S) (S)

GWSA (P) (S) (S)

Grid of Grids (GoG) (P) (S) (S)

Mediation Algorithms (P) (S) (S)

K2_service _(S) _(P)

NCOW services (e.g., JBI publish, etc.) (S) (P)

Net-Centric Collaboration Grid Builder Tool (P) (S) (S)

CES+ (P) (S) (S)

Wizards for each service (P) (S) (S) Collaboration and Messaging (P) (S) (S) GIS, Sensor and additional services (P) (S) (S)

GIS and sensor (P) (S) (S)

C2IEDM service (S) (P)

(21)

Task Assignments

Research and Concept Analysis

(22)

Task Assignments

Design

Net-Centric Collaboration Grid Builder Tool (P) (S) (S) GIS, Sensor and additional services (P) (S) (S)

C2IEDM Services (S) (P)

(23)

Task Assignments

Implement

(24)

Task Assignments

Testing & Evaluation

(25)

Task Assignments

Refine and Refactor

Collaboration Grid Middleware (P) (S) (S)

(26)