OGF eScience and OGF Software

(1)

Geoffrey Fox

GFSG Meeting

Kilburn Building

Manchester University

May 6 2007

(2)

Role of Software and Standards in

Grid Adoption I

• Grids are increasingly being deployed around the world using a

variety of open source and commercial software platforms,

modules and capabilities – many of which are based on

proprietary interfaces with support for only the most broadly

adopted standards.

• Users primarily adopt software solutions

, NOT standards whether

from OASIS, W3C or OGF

• Standards come to attention of software providers when the lack

of standards creates a highly visible “pain point” or barrier to

effective utilization.

• Software providers are often skeptical of all but the most

broadly

adopted standards

that have already proven their desirability

within the market.

• These providers are encouraged by the market to focus on

practical, user driven demands and must continually tradeoff: (1)

fixing bugs; (2) adding new features; (3) adopting standards – thus

setting a

high bar for incorporating standards

within their

(3)

Role of Software and Standards in

Grid Adoption II

• Looking at deployed Grids, one finds the simple Web service standards (WSDL, SOAP, XML) are critical but the more complex standards such as WS-RM and even BPEL and WS-Security have limited adoption.

• Rather software modules such as Condor and SRB set de facto interface points.

(http://grids.ucs.indiana.edu/ptliupages/publications/ReviewofServicesand Workflow-IU-Aug2006B.pdf)

• Web 2.0 shows the same trend in more extreme fashion with Google maps exemplifying a hugely popular module with no adherence to either

Web Service or Open Geospatial Consortium standards.

• HTTP is the Web 2.0 universal standard and the field suggests that one generates broader adoption by focusing on simplicity and ease of entry for developers rather than by specified interoperability.

• Why can Web 2.0 get away with such disdain for standards. It is trading off advantages of rich protocols used in WS-*/Grids versus advantages of low

barrier to entry.

• Presumably the approach that leads to the “best” user experience will be adopted and this could imply different technology choices in different

(4)

Role of Software and Standards in Grid

Adoption III

• The OGF supports broader exploitation and deployment of Grids with a mix of standard setting and community activities.

• Often OGF processes are aimed at driving user requirements all the way to relevant standard specifications.

• Other activities such as workshops and software forums promote

presentations and discussions that both inform about and advance the state of the art.

• Both community and standards activities advance OGF’s broad goal but their different capabilities and roles are not well understood and integrated.

• For example software providers should be interested in both standards and interacting with users but it is not clear that user-standards

interactions are especially fruitful.

• These observations suggest that it be useful to debate the roles of

standards and software and their interaction together with positioning of user, software provider and standards communities.

• How should OGF position its activities to maximize the “broader exploitation

(5)

Web 2.0 APIs



http://www.programmab

leweb.com/apis

has

(April 17 2007) 415 Web

2.0 APIs with

GoogleMaps the most

often used in Mashups



This site acts as a

(6)

The List of

Web 2.0

API’s



Each site has

API and its

features



Divided into

broad

Only a few used

a lot (

39 API’s

used in more

than

10 mashups

)



RSS feed of new

(7)

(8)

8

Four major types of eScience Activities

• Group activities

such as those in GIN (Interoperability)

• Groups perform long term activities in focused areas

• Typically does not get much high level visibility except for GIN

• Note two newish groups at OGF19 in Education and Reliability

• Typically one or more distinct 90 minute sessions

• Timely community activities

arranged in the two-five months

before meeting and including panels, tutorials and short

workshops

• Current submissions to community program

• Typically one or two 90 minute sessions per submission

• Long lead time

single track workshops

with invited and

contributed presentations in topics of broad interest to OGF.

• Typically 4 or 5 90 minute sessions

• Refereeing similar to high quality conferences

• Software Development Track

starting at OGF19

(9)

Community Program

• Community

oriented activities with relatively

light weight

approval

process with call 3-5 months before meeting and

decisions 2-3 months before

• Joint between Enterprise and eScience

• Exploratory

(Birds of a Feather) sessions that could leads to

groups or full one-day workshops

• Tutorials

but not well done in OGF as no easy thoughtful

coordination and don’t easily attract

Grid users (as opposed to Grid builders)

• Need to establish a track with a uniform audience like we propos

for software development

• Community workshops

often small (2 sessions) and led by

groups such as OGF20

Astro-RG/PE-RG/GIN The Astronomical Virtual Observatory - Building Operational Services on Pervasive Grids: Standards in Use9

Grid Services

Grid Subsystems

Users

(10)

Software Development Sessions

• Grids are built from services which hopefully respect standards

• However there are several building blocks or subsystems like Globus or SRB which are used in many Grid projects

• TeraGrid EGEE Geon LEAD MyGrid China National Grid Naregi are Grid Projects

• Form initially at least de facto standards

• What is OGSA in the “non Green Field” of existing subsystems?

• OGF will offer “user group” sessions in “Grid Subsystems” in a set of

consecutive sessions aimed at those building Grids and not at people using Grids

• OGF can provide

one-stop

shopping

so don’t need to

attend XYZweek for all XYZ

• GIN-Standards Interaction?

10

Grid Services

Grid Subsystems

Users

(11)

OGF19 Software Development Track

• Clarens

Grid Portal Toolkit

• Condor

Scheduling system

• Genesis II

OGSA Grid Infrastructure

• Globus

core Grid Infrastructure

• Grid Federated Identity

(GridShib, GAARDS, MyProxy)

• GridSphere

portlet container for portals

• Ninf-G

core Grid RPC Infrastructure

• NWS and BQP

Network/Queuing Tools

• OGCE

Open Grid Computing Environments collection of portlets (for

Science Gateways)

• OMII

core Grid infrastructure (includes OGSA-DAI and Taverna)

• SRB

data Grid infrastructure

• Unicore

core Grid infrastructure

(12)

Future of Software Development Track

• Not clear how many at OGF20 but not very many!

• Less widely adopted software could be showcased in community program

• Suggested additional software from call – no response to request to select from these

• A Introduce/GAARDS: Suggested by Stephen A. Langella [email protected] (part of development team) Action: GAARDS already part of track in federated identity session organized by Von Welch

• B: Nimrod Tools for Distributed Parametric Modeling: Suggested by Geoffrey Fox to broaden geographical scope

• C: CGSP China Grid Supporting Platform: Suggested by Geoffrey Fox to broaden geographical scope

• D: GOS China National Grid Software Environment: Suggested by Fox to broaden geographical scope

• E: European Grid Software: Suggested by Mirco Mazzucato (INFN Padua, Italy) and others

• EGEE core grid Infrastructure , EGEE high level services , SRM based MSS , INFN Grid policy and accounting framework

• F: Nordugrid core grid infarstructure: Suggested by Jean-Pierre Prost (IBM, France) and Balazs Konya (Lund University, Sweden)

• G: ProActive (http://www-sop.inria.fr/oasis/proactive/) and its associated grid component model: Suggested by Jean-Pierre Prost (IBM, France)

• H: GAT The Grid Application Toolkit: Suggested by Thilo Kielmann

• I: SAGA: Suggested by Ed Seidel

• J: PyGridware Python Grid infrastructure: Suggested by Mary Thompson (LBL)

• K: Broader Coverage in Scheduling and Workflow: Suggested by Lennart Johnsson (Houston)

• L Commercial Systems: Suggested by Dave Berry (Edinburgh)

• M: Altair: Suggested by Bill Nitzberg but said he didn’t want to talk!

(13)

13

Current

Workshop Topics

• Federated Identity at OGF19 organized by Ken Klingenstein and Satoshi

Matsuoka

• Semantic Web 2.0 at OGF19 organized by Dave de Roure

• 2nd International Workshop on Campus and Community Grids, continuing

interoperability at OGF20 organized by Wolfgang Gentzsch, Laura

McGinnis, and David Wallom

• Commercial Web 2.0 at OGF21 organized by Charlie Catlett

• Possible workshop on Implications of Web2.0 on eScience, OGSA (Grids),

Enterprise, Digital Libraries/repositories

• Preferred organization of one –day workshops

• Invited and Contributed Talks

• Panel aimed at summarizing topic as relevant to Grids or Grid technology

• After meeting one would

• Post presentations

• Convert panel discussion into a “review” “synopsis” or “Best Practice” for area covered

(14)

BP: Best Practice?

• There are many grids and experience building and using them.

These use “standard” software packages supplemented by

more or less “local” work

• BP’s, Status reports, Summaries, Reviews presumably capture

current understanding of research, practice and experience

• These consist of

• Integrated reviews and/or

• Broad collection of articles such as sets of papers at a focused meeting

• Provide a set of links organized by say OGSA categories if

possible which link to BKM’s which may or may not be OGF

Generated

• Portals, Workflow, Science Gateways were covered in OGF workshops

• We suggest OGF web pages populated with some existing

surveys and solicit additions

(15)

Call for OGF21 Software Provider Forums

• A feature of OGF19 was a multi-day track devoted to a Grid Software

Providers track. The track was motivated by the OGF GIN Grid

Interoperability work and featured the developers of core software

that has broad use or interest among Grid system builders. The

general idea is recorded below and although the sessions were

effective, the

audience was not as expected

experienced users but

rather those interested in a broad survey of the software modules.

• For OGF21, we will repeat the experiment but with some changes to

encourage a broader audience and greater impact on other important

OGF functions. We are interested in Grid Software Providers who

wish to present at OGF21 and

• a) Have a plausible

plan to attract a good audience

with preferably

“new” OGF attendees. This could be by advertising their session to

their existing users.

• b) Are willing to

participate in interoperability brainstorming sessions

with the OGF GIN activity and other OGF Standards and Enterprise

representatives.

(16)

OGF19 Software Provider Forum Expectation

How should we change for OGF21?

• The

audience

will typically be those that are

building Grids

(including software engineers and systems administrators) and

aims to present in one place a

set of user forums

on the

software used to build either broad national Grids or those

aimed at particular application domains. We expect interchange

between providers and users and between the users

exchanging experiences among themselves.

• We expect the sessions

not to be sales pitches

but

interchanges between providers and users already at least

broadly familiar with their product.

• Although organized by the OGF eScience function, we expect

the track to be

important for Enterprise Grids

. The software in

this track should/could be

major consumers of OGF standards

and so the sessions could be interesting from this perspective.

• We understand that we will only cover some of the critical and

innovative Grid software and that we are

omitting software

(17)

OGF21 Workshops

• At OGF19 we proposed an OGF21 Program:

• Day 1: Commercial Web 2.0

• One-day Web2.0 and the Internet Industry Workshop

(Charlie)

• All invited (Amazon, Google, eBay, Microsoft)?

• Day 2 (and perhaps 3): Web 2.0 and Grids

• - Implications of Web2.0 on eScience

• e.g. MyExperiment

• - Implications of Web2.0 on OGSA (Grids)

• e.g. Mashups v workflow and REST v SOAP

• - Implications of Web2.0 on Enterprise

• e.,g. Adoption of Enterprise 2.0

• - Implications of Web2.0 on Digital Libraries/repositories

(18)

Draft Web 2.0 and Grids Workshop CFP

• Collaboration is being revolutionized by the increasing power of communication infrastructure and the Internet which allow both new modes of collaboration and new technologies to support existing approaches. Grids are enabling eScience with scientific collaboratories that will be essential for managing the deluge of information coming from sensors and instruments from the tiniest environmental monitor to distributed high throughput biological devices and the mammoth CERN LHC and shared international satellites. Social or community collaborative

networks are being created by intelligent bookmarking tools like del.icio.us and linked back to scientific grids by projects like Connotea. Further Wikis and

collaborative collections of MP3 files point to other models of collaborative

resource sharing. Web 2.0 encompasses social networking and collaboration sites such as MySpace, Facebook, Flickr and YouTube. There are simple but pervasive compute and storage services from Amazon and online office products from

Google. Web 2.0 tends to use mashups, JavaScript, Ajax and REST; Grids use workflow, BPEL, WSDL and SOAP. Further Web 2.0 focuses on usability and enabling all users to create Internet content.

• We invite contributions from researchers in academia and industry on the technology practice and user experience on the interface of Web 2.0, Grids, Enterprise systems, Digital Libraries or Repositories and e-Science.

• These contributions can be white papers, suggested presentations, panels or