OGCE
Consortiu m
Reusable Grid Portal
Components
Outline
•
The Portal Challenge
•
The Solution:
–
The Open Grid Computing Environments
Consortium (OGCE)
•
The OGCE Portal Architecture
•
Application Portal Examples
OGCE
Consortiu m
Port al User Int erf ace Grid Resource Broker Service Grid and Web Protocol s Information and Data Services Database Service Database HPC or Compute Cluster Grid Information Services, SRB Portal Client Stub Portal Client Stub Portal Client Stub JDBC, Local, or Remote Connectio n
Three-Tiered Architecture
OGCE
Consortiu m
Portal are Gateways to Grid
•
Early Grid Portals were great success:
–
Applications oriented,
–
Provided scientists with some useful tools
–
Encapsulated some complexities.
•
But they were:
–
All “stovepipe” solutions.
•
Very little reusable code and infrastructure
•
Nothing could be shared between portals
–
Many ignored emerging architecture of the Grid
Impact of Distributed Grid and Web
Services
Resource layer
1000s of PCs ->massive supercomputers
Grid Portals
Launch, configure
And control Application Interface
Workflow service App Instanc e App Instanc e App Instanc e Security Data Management Service Accounting Service Logging Event/Mesg Service Policy Administration & Monitoring Grid Orchestration Registries and Name binding Reservations And Scheduling
Open Grid Service Architecture Layer
OGCE
Consortiu m
Advantage of Portlet Architecture
•
Portlet/component approach very compatible with
Web/Grid services model
•
Each Grid service can be associated with a unique
portlet
– Very easy to add new services
– Many different Groups can contribute portlets which can be
plugged into a portal.
• Currently Indiana, Argonne, Michigan, NCSA,Texas + GridSphere portal group following this model. Most likely others as well.
– Jetspeed is Apache standard. Basis for IBM WebSphere
portal. Also Oracle, Sun and others.
• Now a Java Standard (waiting for JSR 168)
•
User can select and configure portlets he/she wishes
to use
The Next Generation Grid Portal
• Building on Standard Technologies
– Portlet Design (JSR-168) IBM, Oracle, Sun, BEA, Apache
– Grid standards: Java CoG, Web/Grid Services
– Web server: JetSpeed (open source)
• User configurable, Service Oriented
– Philosophy: The Portal is a gateway to distributed Grid and Web Services
• With common API, portlets can be exchanged, interoperate
Portal Server
Directory Applicatio
n Factory
ServicesMessaging and group collaboratio
n Event
OGCE
Consortiu m
User Requirements
Have Not Changed
•
Users & Clients are:
– Portal clients/users
– Portal developers
•
Security
– Single sign-on, authentication to all grid resources
• Certificate based / uses MyProxy
– VO management
•
File Management
– Grid FTP
– File staging
– Metadata management
•
Remote Job Management
– Job submission, scheduling, and tracking. Log and output
mgmt.
User Requirements
•
Information Services
– Grid resource and status information directories
– Data and metadata catalogs
– Personal and VO metadata and service index/directories
– Database Interfaces
•
Application Interfaces
– Configure and launch workflow from domain specific interface
•
Access to Collaboration Tools
– VO collaboration. Synchronous and asynchronous
– Access Grid venues
OGCE
Consortiu m
The Solution
The Open Grid Computing
Open Grid Computing Environments
(OGCE) Consortium
•
Motivation:
–
Emerging technologies impacting portal
developers:
• Grid/Web services; ubiquitous database use, XMl
–
Lessons learned from GCE-RG and other projects
(GridPort/HotPage, Alliance Portal, DOE
Components, NEESGrid, Gateway, GRaDS, others)
–
Multiple projects repeating similar tasks, work,
development;
•
OGCE established in Fall 2003 to foster
OGCE
Consortiu m
OGCE
•
Funded by:
–
NSF NMI: software releases will be part of
NMI and synchronized with GRIDS releases
–
Also: NCSA Alliance, NPACI, DOE, NASA,
others
•
Release Schedule:
–
First Release: 11-15-03
–
Next Release: April, 2004
OGCE Tasks/Goals
•
Development of interoperable Grid portal
architecture
–
Define standards for common API’s between portlet
service and remote services
•
Establishment of a Grid Portal Collaboratory
–
Repository of portlet and portal service components
•
An online forum for developers of Grid Portals
•
The building of reusable portal components:
–
Can be integrated in a common portal container
system
•
Contribute to standards organizations:
–
NSF NMI program
software releases
OGCE
Consortiu m
OGCE Initial Architecture
Po rtal Local Portlets Teamlets Proxy Portlets Jetspeed Internal Services Jav aCO GAPI Java Co GKi t Grid Services Grid Protocols GRAM, MDS-LDAD MyProx y Service API ServicesCHEF
Remote Interface s
CoG Stub s HTTP Grid Services Other Services SOA P
OGCE
Consortiu m
OGCE Initial Arch:
Features & Capabilities
•
OGCE approach encompasses multiple portal
development systems:
– Preserving development investments
•
Central portal system (Jetspeed) acts as a central
portal container:
– Portal responsible for managing the lifecycle of pluggable portal components, or portlets.
•
Grid/computational portlets must provide services on
the behalf of the user:
– Not just a collection of links/static pages
Supports Aggregation of
Diverse Tasks
•
Portlets provide a uniform management interface to
the portal (on the left) connect to heterogeneous set of
APIs on the right.
– First horizontal thread
•
Portlets interact with web interfaces written in other
languages such as Perl, Python, PHP, ColdFusion, etc.
– Architecture for bridged support is depicted in the second horizontal thread
•
Examples:
– Locally running Myproxy Grid portlet uses Java CoG kit to
interact with remote Grid services
– Job Scheduler portlet uses GridPort Toolkit to submit job via Community Scheduling Frameworks (CSF) service
– Teamlets (sharable portlets) share collaboration and Grid
OGCE
Consortiu m
OGCE - Revised Architecture
• Revised portal architecture provides a unified interface for portal services.
• Supports aggregation of services in backend
Po rtal Portlets and Teamlets Jetspeed Internal Services Service API Grid Protocols Grid Service Stubs Remote Content Services Remote Content Servers HTTP Grid
Services Java CoG Kit
Features
•
All services will employ a single (generic)
service API that allows them to be deployed
and accessed in a uniform fashion.
•
Developer then chooses (during portal
configuration) the implementations of
particular services.
•
Services may be local, may access remote
sites via Grid protocols, or may access remote
content, such as web interfaces written in Perl.
•
OGCE
Twist
:
–
Entire portal will auto-install and include all required
OGCE
Consortiu m
Key Technologies????
•
Core Portal Framework:
–
CHEF/JetSpeed (U. Mich)
•
Core Grid Technologies:
–
JavaCoG (provides Globus interface)
OGCE
Consortiu m
TeraGrid Portal
• Access Point for TeraGrid Services
• Version 0: Collecting Initial Services
– Public Information about Resources
– Private Information for the developers.
• Version 1: User centered portal
– HotPage/GridPort style access to user accounts, credentials, job submission &
management.
• Version 2: Portals for Science Collaborations (Q3 2004)
– Shared spaces, whiteboards, AG access, group
TeraGrid Integrated Architecture
Diagram demonstrates how existing software projects (such as GridPort) can
Po rtal Portlets and Teamlets Jetspeed Internal Services Grid Service Stubs Remote Content Services Remote Content Servers HTTP Grid
Services CoG KitJava
Local Portal Services Service
API
OGCE
Consortiu m Create job descripti on Store and record results Generate and store derived products monitoring myMEAD personal information space Broker Service Teragrid data service Teragrid Globus Toolkit 3.0GridFTP MetaCatalog
Service(MCS) NEESgrid Meta-Catalog service Storage ResourceBroker (SRB)
Gather data files, resources Execute applicatio n Need ‘x’ put to ‘l’ by time y:00
Need to store ‘y’ and ‘z’
Provides context to resolve ‘x’, ‘y’, ’z’ Retrieve customized run description template from data management
services Monitoring results captured in event services persistent channel. myMEAD records address of service and channel
Delayed invocation of GridFTP
Teragrid Data Service-single point of contact to server-side data servers
DOE Fusion Grid Portal
•
Portal Development on
the DOE SciDAC Grid
– Initially, Fusion Grid
•
Multi-site collaborative
project:
– TACC, Indiana, SDSC, General Atomics
•
Developing interoperable
portal and Web services
OGCE
Consortiu m
CHEF - NEESGrid
•
NEES Tools
– Data Viewers (Live/Stored)
– Telepresense
– Video as Data
– Electronic Notebook
– Data/Metadata Repository
•
Grid Technologies
– Grid sign on using myproxy
– Grid computational portal
– GridFTP
OGCE
Consortiu m
OGCE Portal System
System Integration
OGSA Globus
COG
CHEF Toolkit
Gridport
Alliance
Portal Testing
OGCE
Consortiu m
NMI Features List (April 04 Release)
•
Content Management:
– Resources: Users and groups can upload and share
documents and URLs.
– News: Several news feeds are available, including XMLHack
and BBC News.
– Grid Context: Store arbitrary web objects (movies, web pages,
audio files) in a customizable, annotated directory tree.
•
Collaborative:
– Chat: Engage other members of your group in online
discussion.
– Discussion: Participate in discussions with other members of
your group
– Calendar Program:schedule appointments and attach
associated documents and URLs. Calendars can be used to schedule both user and group activities.
– Anabas Impromptu: system for real time shared display, audio,
NMI Features List (April 04 Release)
• GT 3.0/Interactive:
– Proxy Manager and Job Submit: Obtain and manage one or more Grid
credentials that can be used to access Grid resources through your browser.
– GridFTP: Use Grid credential to browse remote directories and to upload and download files between your desktop and a remote host.
– Application Manager
– Application Factory Manager
• Remote Job Management
– GridAnt for workflow
– Job submission portal
– Community Scheduling Framework (CSF): submit jobs to scheduler
• Information Services:
– LDAP Browser:
– Navigate the LDAP server of your choice.
– GridPort Information Repository Portlets (GPIR): Portlets are available for browsing Machine Summaries, System Status, and Total Grid Capabilities.
• Application Portlets:
OGCE
Consortiu m
Key Feature –
Download a Portal & Install
• Prerequisite Software
– Java 2 Standard Edition version 1.4.x or higher.
– Apache Ant version 1.5.x or higher.
• Preparing the Installation
– Optionallly, create a "portal" account on your web server.
– Download the portal bundle
– Put the download bundle into the home directory of "portal"
account.
– Untar or unzip the portal bundle.
– Globus: installation process will create the directories .globus and .globus/certificates in the
• Software automatically installed includes:
– Globus/Java CoG
– CHEF
• Install the Portal
– Move into the portal/chef-1.0.7 directory.
– Edit file "build.properites"
– Type "ant deploy" to build in and install the portal.
– Start the Tomcat web server.
– Point your browser at http://<your host
name>:10081/ogce and create an account.
• OS/Machines Tested:
– MAC OSX
– PC/Windows XP
– Linux
Current Status
•
SC03 Demonstration portal and software download
available
– Beta1
•
Updated Release available now
– Bug fixes
– Now supporting GT3.x with J2SDK 1.4.2
•
Working on NMI April 2004 release
– Including new features to support science portal applications
• Open GCE Runtime Engine
(
OGRE) workflow services and portlets• Sample template portlets for simple science applications
– Based on aggregation of existing projects
OGCE
Consortiu m
Future Directions
•
Moving towards new portlet standard
– JSR 168
– Should allow compatibility with
• GridSphere
• Academic education and business portals Oncourse, OneStart, CHEF (of course), and uPortal
• Major commercial products offerings from Sun, IBM, Oracle, etc.
•
Incorporate new features:
– Grid Services (CSF, OGSA-DAI portlets, Job Sequencer)
– Access Grid, Polycom and other AV Portlets
•
Collaborations and Contributions
– DOE SciDAC projects (such as CMCS)
•
Expand online repository
Future Directions
•
Moving towards new portlet standard
– JSR 168
– Should allow compatibility with
• GridSphere
• Academic education and business portals Oncourse, OneStart, CHEF (of course), and uPortal
• Major commercial products offerings from Sun, IBM, Oracle, etc.
•
Incorporate new features:
– Grid Services (CSF, OGSA-DAI portlets, Job Sequencer)
– Access Grid, Polycom and other AV Portlets
•
Collaborations and Contributions
– DOE SciDAC projects (such as CMCS)
•
Expand online repository
OGCE
Consortiu m
Participants
Indiana University
University of Michigan University of Texas
at Austin
Argonne National Labs
JavaCoG
National Center for
Supercomputing Applications
