Application Web Service
Toolkit
Geoffrey Fox, Marlon Pierce, Ozgur
Balsoy
Motivating Concerns
• Grid infrastructure software tends to be geared toward low level services:
– Job submission and monitoring – Information services
– Distributed security
• Typical end users are really interested in running specific applications, not using low level grid tools.
– GCE computing portals build user-centric tools out of Grid infrastructure tools. – The grid infrastructure is a great foundation for this.
• Not all HPC resources use Grid infrastructure technologies in production but still see the value of portals
– DoD MSRCs are good examples
– We need service definitions that are independent of implementation
• There are no well defined general ways for
– Describing scientific applications
– Specifying the Grid services these applications require
Some Stakes in the Ground (and a
rough outline)
• We need interfaces to describe real scientific applications and their execution life cycles.
• Application ‘bundles’ are composed of core, general purpose Web services.
– Need composition languages
• Application interfaces define some related Web services useful to portals
– Description, archiving
• Application bundles can also be composed to form larger applications.
• Application interfaces can be used to automatically generate browser interfaces.
Application Web Services Toolkit
• We are building a portal infrastructure to implement the ideas on the previous slide.
• Our goal is to be application-centric
– By “application” we mean some scientific or engineering code (finite element codes, quantum chemistry codes, etc).
– Applications are added to the portal in well defined ways.
• The application interface is defined by the developer (or application expert) through well defined XML schema.
• The application is bound to various core service interfaces.
– Core service interfaces are in WSDL, will be in OGSA
– Applications can be statically bound to core services (and specific resources), as we describe here.
Application Lifecycles
• Abstract State: describes potential uses of the application.
– Analogous to a.out on a file system
• Ready State: a specific application instance has been set up but not run
• Submitted State: Application instance is running
– Analogy: sh a.out
• Archived State: The job is completed and metadata about the instance is stored.
– Metadata can be queried later as a service
– Archived instances can be used to create new instances.
Application Web Service Bundles
• An application is composed of several core services.
– Application description, batch script generation, job
submission, job monitoring, file transfer.
• These should be deployable on a specific compute
resource.
• These become services on a host resource.
• The application should have two parts:
– Describe how to invoke
– Describe ‘workflow’ of how the core services interact
Application Composition
• Application services on specific resources can be combined to create
aggregate applications. • That is ‘Run code 1 on
machine 1, use output for code 2 on machine 2, and use visualization service on machine 3’
• We are currently trying to decide the best way to do this workflow.
AWS 2 AWS 3 AWS 1
Application Web Service Schemas
• From the proceeding, we have two sets of schema:
– First set defines the abstract state of the application
• What are my options for invoking disloc? • Dub these to be “abstract descriptors”
– Second set defines a specific instance of the application
• I want to use disloc with input1.dat on solar.uits.indiana.edu. • Dub these to be “instance descriptors”.
• Each descriptor group consists of
– Application descriptor schema
– Host (resource) descriptor schema
Container Structure for Descriptors
•
Applications contain hosts, which contain
execution environments.
•
Each of these are independent schemas that get
included in the parent with a “binding” tag that
uses <xsd:anyType>
– We were inspired by WSDL bindings here.
•
This keeps schema scope manageable, makes
schema pluggable
Application Schema Elements
•
The host and environment descriptors are the
usual stuff, so we won’t go over that here.
•
Abstract descriptors describe possible
invocations of the application.
– Edited by application deployers, used to generate user interfaces
•
Instance descriptors describe particular
invocations of the application.
Application Element
•
Each application is defined by a number of fields
– Name and version tags are strings
– Flag tag lists code flags that can be set and how to use.
– Input, output, and error ports describe how the code handles I/O and error.
– ApplicationParameter tag provides a general purpose place to put arbitrary name/value pairs if you need
more descriptions.
InputPort Explanation
• You need one input port for each piece of input that the code expects.
– Typically just an input file.
• InputHandle is a useful short name for referring to this input port.
• InputDescription is a longer string that allows the
application deployer to give a much longer description of what this input port is used for.
• InputMechanism describes how the input for the code is aquired.
– Right now, just a file from local disk.
– Will support services to load input from specified resource (local file, URL, database, etc.).
Application Descriptor Services
•
After defining the schema, next step is to cast
into language bindings
– We used Castor to unmarshal XML to Java
•
Ultimately, we will make this a Web service
– Get/set and query methods for the Application are
obvious candidates for turning into a WSDL interface.
Application Instances
• Host and execution environment (queue)
descriptors are familiar.
• Application Instance descriptors are
superficially similar to the previous
abstract descriptors.
–
Contain name, version, input/output/error
ports, host bindings, etc.
Application Instance Schema
Services
•
Again, we just cast the schema into java code.
•
We then implement JavaBeans/JSPs for
manipulating the schema.
– Applications can be deployed in the portal
•
Methods for working with schema instances/java
code will be used to define a Web service
– The instances are stored in a repository logically separate from the user interface server.
– The repository might be a file system or an XML
Automatic Interface Generations
with Schema Wizards
•
Gateway schema pages are currently one shot
development efforts.
– We map HTML forms to a specific schema.
– Form actions update schema instances through Castor generated classes.
•
More generally, we want to be able to develop
general purpose schema elements to GUI
widgets (HTML or otherwise).
XML Schema Schema Processor Castor Castor SOM Javabeans Velocity TemplatesVelocity TemplatesVelocity Templates Velocity TemplatesVelocity TemplatesJSP and HTML forms
Schema Wizard Explanation
• A schema is read in to create an in-memory
representation (SOM) and also to create Java files.
– SOM=Castor’s Schema Object Model
• Each schema element is mapped to a self-contained JSP nugget.
• JSP nuggets are generated from templates.
– One template for each element type (simple, complex, enumerated, unbounded,….).
– Velocity is used for convenient scripting of JSP.
• The final JSP page is an aggregate of the JSP nuggets files (using <%@:include>).
• Complex schema elements are mapped to JavaBeans generated from the schema with Castor.
Where Are We Really?
•
Many core Web service implementations
developed.
•
Application schema are available and have been
implemented in a demo portal.
•
Schema wizard is still in development.
•
Lots of work on remote portlets for Jetspeed
– Navigable, session maintaining, form parameter passing portlets developed.
– Still need to work out security.
