Overview of Grid Computing Environments
Proposed GGF Information Documen G.Fox, D. Gannon, M. Pierce, M. Thomas
PTLIU Laboratory for Community Grids
Geoffrey Fox
Computer Science, Informatics, Physics Indiana University, Bloomington IN 47404
Source of Report
• Last Half of 2001: Call from GCE RG for Papers for Journal special issues – 28 papers submitted and reviewed
• Published in Concurrency and Computation: Practice and Experience Vol. 14, Grid Computing
Environments Special Issue 13-15, Fall 2002
• http://aspen.ucs.indiana.edu/gce/gce2001index.html
• Augmented by chapters (about 14) in Fran Berman, Geoffrey Fox and Tony Hey, ‘Grid Computing:
Making the Global Infrastructure a Reality’, ISBN 0-470-85319-0, John Wiley & Sons Ltd, Chichester, 2003. See http://www.grid2002.org
Raw (HPC) Resources Database Aggregatio n Portal Syste Service s Syste Service s Syste Service s Application Service Portal Service s Porta Service s Gri Computin Environments Use Services “Core Grid Application Service
OGSA (OGSI) Interfaces
Two Major Areas
• “Programming the User Side of the Grid”
which involves discussing computing model
for Grid
• Controlling user interaction – rendering any
output and allowing user input in some (web)
page.
– This includes aggregation of multiple data sources in a single portal page (Jetspeed). – Has natural overlaps with information
Classification of GCE Papers i
Programming the User Side of the Grid
• Technology for building GCE systems -Interface with
backend Infrastructure e.g. Community Grid Kits, GPDK
• Problem Solving Environments
– Domain specific collection of tools and user interface. E.g. XCAT, Polder, SCIRun, Astrophysics Collaboratory
• GCE Tools
– Support parameter sweep, visualization, job status, files, security, workflow ..
• GCE Shell Portals providing a general interface to many Grid capabilities
– Analogous (not usually command line) to role of UNIX shell providing access to UNIX tools and user programs, files …
– Note UNIX has core system and higher level tools accessed by Shell – E.g. Unicore, Hotpage, Mississippi Portal
Aspects of Programming Model I
• See review of programming model by GGF APM RG
– Chapter 21 of Wiley Book and web page
• Handling of the basic components of a distributed computing system – files, computing and data resources, programs, and accounts.
– The GCE will typically interface with an environment like Globus or a batch scheduler like PBS to actually handle the back-end resources. – However the GCE will present the user interfaces to handle these
resources.
• We can follow the lead of UNIX (and Legion) and define a basic
GCEShell providing access to the core distributed computing functions.
– JXTA also builds some Grid-like capabilities with a modification of UNIX shell model.
Implications of 3-Tier Model
• The 3-tier model implies that any given capability (say run a matrix inversion program) can appear at multiple levels.
– Maybe there is a backend parallel computer running an MPI job; this is front-ended perhaps as a service by some middle-tier component running on a totally different computer, which could even be in a different security domain.
– One can “interact” with this service at either level; a high performance I/O transfer at the parallel computing level and/or by a slower middle-tier protocol like SOAP at the service level.
– These two (or more) calls (component interactions) can represent different functions or the middle tier call can be coupled with a high performance mirror.
Raw (HPC) Resources Database Aggregatio n Portal Syste Service s Syste Service s Syste Service s Application Service Portal Service s Porta Service s Gri Computin Environments Use Services “Core Grid Application Service Application Metadata Actual Application
OGSA (OGSI) Interfaces
Technology for building GCE Systems
• CoG Kits for Java Perl Python CORBA ….
• GPDK extending CoG kits with JavaBean
and JSP middleware
• Grid Portal Toolkit
• Other such interfaces with also C and XML
tools
GCEShell Portals and PSE’s
• One can divide GCE capabilities into
generic (copy file) or domain specific
(generate the multigrid solver suitable to
simulate Middle Earth)
• Correspondingly one finds portal
frameworks or GCEShell Portals presenting
general capabilities
• Grid or web-based Problem Solving
Environments optimized for a particular
domain
Typical Layered Architecture
GCEShell Portals
Globus GT2 Service Manage backend
resources
Interface Middleware wit backend resources Core Middleware Services
Application Services Problem SolvinEnvironments User
Interface-Client
GCE Tools
• Data Management
– File manipulation in all tiers including client, middleware and back end
• Security
– In all tiers and providing interfaces to core Grid Security
• Workflow or “Programming the Grid”
• Grid versions of MPI
• Higher-level tools include visualization or
support of parameter sweep applications
More on Programming Models
• Basic 2-level programming model is assumed by most projects
– First you use classic (parallel) programming to produce “simulation” nuggets (wrapped as application web
services)
– JDBC / OGSA-DAI to package data resources as objects or services
• Then you need to compose (orchestrate) the control and dataflow between nuggets
– Many different models for how this is to be done and can call this workflow – Next thrust of GCE RG?
Research in “2-level” Programming Models
• Basic user interface to middle tier proxies controlling backend (software) resources
• Component Models like ICENI (Imperial College) or DoE Common Component Architecture
• Commercial workflow engines as in BPEL4WS • Scripting front-ends as in Matlab or Python
• Network server model as in NetSolve or Ninf • Computational Economies
• Semantic Grid technologies (ontology based
integration of resources) as in MyGrid or DiscoveryNet
Programming Infrastructure (Hosting Environment)
• Different approachs assume different run-time (implementation) and user programming “ether” • BPEL4WS thinks about specifying interactions
between components; Matlab interface invokes components from a script
• Java Interface to OGSI WG emphasizes that we don’t have an established implementation even if you pick a language
– Jini
– JXTA
– Servlet
– Enterprise JavaBeans EJB
– Or perhaps some future SJB (Scientific JavaBean)
Web Services as a Portlet
• Each Web Service naturally has a user interface specified as “just another port”
– Customizable for universal access
• This gives each Web Service a
Portlet view specified (in XML as always) by WSRP (Web services for Remote Portals) • So component model for
resources “automatically” gives a component model for user
interfaces
– When you build your
application, you define portle
at same time
Application o Content source WSD L Web Service S R W P
Application as a WS
General Application Port Interface with other We Services
User Face o Web Servic
WSRP Ports define
WS as a Portlet
Web Services have other ports (Grid Service) to be
Online Knowledge Center built from Portlets
• Web Services
provide a
component model
for the middleware (see large “
common
component architecture
” effort in Dept. of
Energy)
• Should match each WSDL component
with a corresponding user interface
component
• Thus one “must use” a
component model
for the portal
with again an XML
specification (
portalML
) of portal
component
Content Provider WSD L Web Service F I U O F I R O Portal Aggregat WS-User Facing Fragments Render Other W User Facin Ports Other WS Resource o Service-facin
Ports User-facinPorts
Use Profile Application o Content source WSD L Web Service F I U O F I R O Render Porta (Aggregator ) Selecto r Filte r Control Channel Customized View Selectio View Control Channel Customized View
Customize
User-Facin
Ports
Portlet Portlet Portlet Portlet
XML
RSS, OCS, or other Local or remote
HTML
Local files
JSP or VM
Local templates WebPageRemote HTML
Portlet
Portlets
User
implemented using Portal API
Portlets
Data
PortletControlle
r PortletController Screen Manager HTML PSML PortletContro l EC S JSP template EC
S ECS EC
S ECS
EC
S ECS ECS
ECS Root to HTML
EC S
Turbine Servlet
Jetspeed
Portlets and Portal Stacks
• User interfaces to Portal services (Code
Submission, Job Monitoring, File
Management for Host X) are all managed as
portlets.
• Users, administrators can customize their portal
interfaces to just precisely the services they want.
Core Grid Services User facing Web
Service Ports
Jetspeed Computing Portal: Choose Portlets
4 available portlet
Choose Portlet Layout
Choose 1-column Layout
Lists user files on selected host, noahsark.
File operations include Upload, download, Copy, rename, crossload
Tabs indicate available
portlet interfaces.
File
Sample page with several portlets:
Provide information about application
and
host parameters
Select application to edit
