Portals and Portlets
Marlon Pierce
What a Science Portal Is/Is Not
n It is
• A tool for aggregating and managing web content
• A user customizable view of these Web content pieces.
n You see what you want/can see. n But you must log in.
n The portal recognizes you.
• Implemented on top of standard services
n Like login, authorization, customization.
n May include collaboration, etc, that depend on login.
• A way to accomplish tasks through browsers:
n Launch, monitor jobs n Move files
n Run science applications based on these services.
• Compatible with emerging standards and best practices (such as
portlets, JSR 168 and WSRP).
n It is not (just)
• A web page
Things I Hate About Portals
n If you involved in portal efforts, be aware of the following: n Browsers have limited interactivity.
• Desktop GUIs provide much better interactivity but have other problems.
• Applets are a solution, but they don’t interact with other parts of the browser very
well.
• Solution: Service Oriented portals let you use services through both portals and
grid desktops.
n Developing really useful user interfaces to your set of services is a time
consuming, non-scaling process.
• Get users involved early in design.
• If you can get users directly involved, even better.
n Browsers notoriously have incompatible features.
• Things don’t work the same on IE, Mozilla, Konqueror, etc. • Same browsers on Macs, Windows don’t work the same. • No substitute for lots of testing.
n Portals depend upon backend resources and security
• But these things are not under portal developers’ control.
Interoperable Portal
Services (ET-03-011)
Marlon Pierce
What 3
rd
Party Technologies Did
We Use?
n
Tomcat 3.x and 4.x, Jetspeed 1.4, JSP
2.x, JavaBeans, Apache Axis 1.x
n
We also used kerberized CORBA in
previous projects
•
Unfashionable these days
n
If I were starting today, I would use
•
Gridsphere (for JSR168), JavaServer Faces,
Project: ET011
n
Goals of the project are to demonstrate
interoperability between Portal/PSE projects
• Mary Thomas (PI), TACC: HotPage
• Tomasz Haupt, MSU: DMEFS
• Marlon Pierce, IU: Gateway
n
We investigated building interoperability at two
levels:
• Web services provide standard interfaces
• Portlets provide component-based interfaces
n
I was responsible for deployment at ASC and
ARL.
Portal Security
n We are building off
Gateway’s approach for Web-based security for DOD portals.
• Approved for ARL and ASC
n Users kinit to a web server
to get a ticket.
• SSL, MD’d sessions,
Certificates maintain secure connection.
• Web server typically
located in “DMZ”
n Web server manages
session IDs, invokes backend requests with Kerberos client utilities.
n Probably would break
Portlets and Containers
n
One of the problems of previous portal
development is that there is no good way to
share interface components.
• How do developers share web interfaces?
n
Also, how can we avoid constantly reinventing
things like login services, customization
services, page organization, access controls.
n
Answer: use portlets and containers.
• Becoming a recognized best practice for portal
What Is a Portlet?
n A portlet is a piece of Java code that runs in a Web
server inside a container servlet.
n Portlets can do two things:
• Perform non-visual operations such as make connections to remote hosts, perform operations.
n Example: get a list of local files.
• Create their display
n The portlet passes its display to its parent, which is
responsible for constructing the entire display.
• Typically this is HTML, with tables uses to organize component displays.
Portal Services
n
We had several services that we portletizing as
part of this project:
• Job submission
• File Transfer
• Job Monitoring
n
We develope DOD versions of TACC’s GPIR
services
n
We extended Jetspeed login to support web
Job Submission
n
Primarily based at ARL
• Support Fluent, ANSYS, ABAQUS
• Services construct GRD scripts, allow users to run
and archive jobs.
n
We are extending this to support ANSYS at
ASC, DMEFS codes at ARL.
n
We need to extend script generators for other
queuing systems.
File Management
n
File management services allow you to
• Upload, download files between desktop and remote
HPC
• Download entire directories as zipped files.
• Delete remote files.
• Navigate remote directories.
• Unzip/untar remotely.
n
Targetting ASC and ARL initially
Job Monitoring
n
We have web interfaces that will allow to monitor
your jobs on various hosts.
• Constructs an HTML table of your running jobs in a
unified format.
• Allows you to stop jobs
n
We support GRD in production portal at ARL.
nHave ported this to PBS, LSF, and LoadLeveler
Access to Portlets
n
Obviously not all users have accounts at all
centers.
• An ASC file browser should be accessible only to
users with an ASC account.
n
Jetspeed has role based accessed control to
portlets.
n
Each user can be assigned to one or more user
roles (“ERDC”, “ASC”, etc).
n
This controls which portlets a user can add to his
GridPort Information Repository
(GPIR)
n
Developed by TACC group for NPACI
resources.
n
Porting this to DOD.
n
Aim is to aggregate and cache grid and
portal related data from multiple sources in
a uniform way.
GPIR Approach
n GPIR is implemented as a set of Java Web Services,
one to handle the input of GPIR data (Ingester WS) and another to facilitate the querying of that data (Query WS)
n The Ingester WS accepts or "ingests" several types of
XML documents and stores them in a relational database (currently MySQL, Postgres).
• These documents are created by a variety of means, including Java Clients that exist on the resources themselves, http "web scraping" of machine-specific flat-file formats, and queries of additional information providers such and MDS, GMS (Grid Monitor Service), and NWS (Network Weather Service).
n Persistently stored data can then be queried via the
Query Web Service which uses the same XML
resources used by the Ingester, in addition to some
GPIR Schema Types
n Static: static data for a machine.
n Load: load data for a machine.
n Status: machine status (up, down, unavailable).
n Downtime: downtime data for a machine.
n Jobs: job data for a machine.
n MOTD: Message of the Day data for a machine.
n Nodes: Nodes data for a machine.
n Services: represents the status of grid software running
on a system.
n NWS: This returns bandwidth and latency
Where Are We Today?
n
JSR 168 is an important new standard for Java
portals.
• Standardizes portlet containers
• Commercial products available from Sun, IBM,
Oracle, BEA
• Open source implementations include uPortal,
GridSphere, eXo, Jetspeed2
n
Web Services for Remote Portlets (WSRP) also
available.
• Compatible standard to JSR 168 that uses SOAP and
WSDL to communicate between portlet container and portlet.
• Potentially allows containers and portlets to be from
Some Portlet Projects
EU project that develops both portlet containers and Grid portlets.
GridSphere
Education-focused portal tools. UMich’s CHEF, uPortal, IU’s OneStart, etc.
Sakai
Portlet component providers Open Grid Computing
Access context services for managing metadata GridContext Portlets
Access to Anabas shared display applets Anabas
View, interact with HPC status, job, etc information. GPIR Portlets
Run simple executables on remote hosts GRAM Job Submission
Live chat services and interfaces Chat
Persistent topic-based discussion for groups Discussion
Interactive individual and group calendars Schedule
WEBDav based document system for group file sharing Document managers
Post topics to newsgroup, manage group references and citations with access controls
Newsgroups and citation portlets
Upload, download, crossload remote files. GridFTP
Basic Globus MDS browsing and navigating MDS/LDAP Browsers
Get MyProxy certs after logging in. Grid Proxy Certificate Manager
Description Portal Capabilities
Set up and run task graphs using the Java CoG CoG Workflow demonstration portlet
The backend for the Grid Context portlet XDirectory Services
Combine GridFTP and GRAM into application wizard forms.
Application Management
Interact with Condor through browser. Condor Portlets
Download and install server side of the OGCE newsgroup system.
Newsgroup Services
Manage complicated grid tasks through an extensible, Apache Ant like task list.
OGRE Job Management Services
Schedule sequences of jobs on several hosts using Community Scheduling Framework.
Job Scheduling and Sequencing
Description
Why Is This Important?
n HPCMP developers generate extremely sophisticated
web interfaces and services to science and engineering applications.
• These should be portlets
n Things like login, user display layouts managers, access
controls to content, etc., should not be reinvented.
• There are plenty of portal projects that have done this.
n These are portlet containers.
• You should not reinvent this.
n By adopting the portlet/container approach, you can also
create portals that combine HPCMP specific content as well as third party portlets.
What is JSR 168?
n Defines the (Java) standard for vendor
container-independent portlet components.
• Portlets can be developed independently of the container.
n Many implementations:
• Gridsphere, uPortal, WebSphere, Jetspeed2, ….
n From the portlet development point of view, it is really
very simple:
• You write a java class that extends GenericPortlet.
• You override/implement several methods inherited from GenericPortlet.
• You use some supporting classes/interfaces
n Many are analogous to their servlet equivalents
n Some (portletsession) actually seem to be trivial wrappers
The Infamous Big Picture
n As a portlet developer, the previous set of classes are
all you normally touch.
n The portlet container (such as Pluto or Gridsphere) is
responsible for running your portlets.
• Init, invoke methods, destroy.
n Portlets have a very limited way of interacting with the
container.
• It is a black box.
Some Suggestions
n Java-based systems should adopt JSR 168 based
portlet containers for Java based systems.
n All components should be at least Web Service “ready”
• Use WSDL to define APIs for services (and bind later to specific programming implementations)
n Make a clean separation between portals and services.
• “Service oriented approach” will allow both desktop GUIs and portals to use the same services.
More Suggestions
n
Portal/service teams have to be first-class
members of the MSRC infrastructure.
•Need to know about HPC turnover well in
advance.
•
Command line login changes need to be
accompanied by simultaneously approved
Web logins.
n
Need more uniformity in MSRCs
•
TeraGrid’s CTSS “common software stack” as
Telescoping Portal Architecture
Jo
b
Su
bm
it
File
M
an
ag
e
Se
cu
