D ESIGN P HASE. Activity: Lead Partner: Document classification:

(1)

DESIGN PHASE BGII-DJRA1.2-v1.0-PSNC-DesignPhase.doc PUBLIC Page 1 of 24

D

E S I G N

P

H A S E

I N T E R O P E R A B I L I T Y A N D D E F I N I T I O N O F C O O P E R A T I O N W I T H O T H E R A C T I V I T I E S

:

S A 1 ,

N A 3 ,

S A 3

A N D P L A N F O R E D U C A T I O N T R A I N I N G S I N C O

-

O P E R A T I O N W I T H

N A 2

Document Filename: BGII-DJRA1.2-v1.0-PSNC-DesignPhase.doc

Activity: JRA

Partner(s): PSNC, VGTU, VU

Lead Partner: PSNC

Document classification: PUBLIC

Abstract:

This document gives a detailed overview of works planned within Joint Research Activity as well as relations of this activity to other activities of the project.

JRA activity focus on enhancements of three main components: Migrating Desktop Platform, Gridcom and visualization tools to support everyday work of BalticGrid-II users. Document provides description of these components, and current state of their available functionality. Based on experiences gained during BalticGrid-I project as well as discussions (internal and with representatives of other activities) required directions of development were designed and shown in details.

The last part of the document shows plans for interoperability and cooperation between JRA and other activities. JRA activity has relations with SA3 (providing user support regarding JRA components), and NA2 (participation in educational and disseminative activities), as well as associations with NA3 and SA1 activities. It makes efficient collaboration between activities a key issue for successful results of the project.

(2)

DESIGN PHASE

BGII-DJRA1.2-v1.0-PSNC-DesignPhase.doc

PUBLIC Page 2 of 24

Document review and moderation

Name Partner Date Signature

Released for moderation to Approved for delivery by

Document Log

Version Date Summary of changes Author

0.1 28/8/2008 Draft version Bartek Palak

0.2 15/9/2008 Description of visualization tools Arnas Kačeniauskas 0.3 16/9/2008 Corrections after 1st_{internal review} _{Bartek Palak}

0.4 17/9/2008 Corrections after 2nd_{internal review} _{Arnas Kačeniauskas}

0.5 18/9/2008 Description of Gridcom groupware Algimantas Juozapavicius

0.6 19/9/2008 Editorial changes Arnas Kačeniauskas

0.7 28/9/2008 Corrections after 3rd_{internal review} _{Arnas Kačeniauskas}

0.8 29/9/2008 Corrections after 3rd internal review Algimantas Juozapavicius

(3)

DESIGN PHASE BGII-DJRA1.2-v1.0-PSNC-DesignPhase.doc PUBLIC Page 3 of 24

3. DESIGN OF COMPONENTS DEVELOPMENT ... 8

3.1.MIGRATING DESKTOP PLATFORM... 8

3.1.1. Component description and current state ... 8

3.1.2. Software development design ... 10

3.2.GRIDCOM... 12

3.2.2. Software development design ... 13

3.3.VISUALIZATION TOOLS... 13

3.3.2. Software development design and deployment issues ... 16

4. INTEROPERABILITY AND COOPERATION WITH OTHER ACTIVITIES ... 20

4.1.NA2“EDUCATION,TRAINING,DISSEMINATION AND OUTREACH”... 20

4.2.NA3“APPLICATION IDENTIFICATION AND COLLABORATION” ... 21

4.3.SA1“GRID OPERATIONS”... 21

4.4.SA3“APPLICATION INTEGRATION AND SUPPORT”... 21

5. SUMMARY AND CONCLUSIONS... 23

(4)

DESIGN PHASE

LIST OF ACRONYMS

AHM All-Hands Meeting

ALs Activity Leaders

BG-I BalticGrid

BG-II BalticGrid-II, BalticGrid Second Phase CPU Central Processing Unit

EC European Commission

EGEE Enabling Grids for E-sciencE, a project co-financed by the EC

EGI European Grid Infrastructure

ETDO Education, Training, Dissemination and Outreach (NA2 Activity of BG-II)

EU European Union

FP7 EU Framework Program 7

GPU Graphics Processing Unit

GUI Graphical User Interface

HDF Hierarchical Data Format

ICT Information and Communication Technologies

IFJPAN Instytut Fizyki Jadrowej, im. Henryka Niewodniczanskiego, PAN, Krakow, Poland JRA1 Joint Research Activity 1

LOD Level-Of-Detail

MD Migrating Desktop

MPI Message Passing Interface

PC Personal Computer

PD Project Director

PMB Project Management Board

SE Storage Element

SIGs Special Interest Groups within the BG-II project

UIIP NASB United Institute of Informatics Problems of National Academy of Sciences of Belarus VGTU Vilnius Gediminas Technical University

VU Vilnius University

VTK Visualization ToolKit

WN Worker Node

X3D ISO standard XML-based file format for representing 3D computer graphics

(5)

DESIGN PHASE

1. INTRODUCTION

The research-development works of the activity are focused on development and integration

of e-Infrastructure components and services designed for grid applications as well as tools

supported everyday work of the user in grid infrastructure.

Three main components are developed within activity:

• Migrating Desktop Platform – intuitive interface to project resources - will be

enhanced about support for advanced data and complex application management as

well as features enabling interoperations between various computing infrastructures

• Gridcom – web-based innovative groupware for grid technology - extension of

mechanisms for cooperation of user groups and applications,

• Tools for visualization of specific computing results - development of special

mechanisms for visualization of application results in distributed grid environment

Detailed design of planned components’ enhancements is an important matter to accomplish

activity goals. The other issue is definition of collaboration framework to ensure efficient

interoperability and cooperation between activities.

(6)

DESIGN PHASE

2. JOINT RESEARCH ACTIVITY

2.1. GOALS OF THE ACTIVITY

The experiences gained during BalticGrid-I project (and other projects related to grid

technologies) show that tools providing intuitive access to grid resources and enabling group

work could attract the grid users and make their work much more convenient. The need of

advanced services that help users in their everyday work will be fulfilled by extending an user

interface to grid (Migrating Desktop Platform – a user-friendly environment to access and

manage resources) and providing tools that supports users cooperation (Gridcom

web-based innovative groupware for grid technology

). The JRA aims also at development mechanism

enabling visualization of computing results in grid environment.

Activity goals will be achieved by:

•

Extending the e-Infrastructure with advanced services necessary from the user,

application and administrator point of view - the GUI level

Analysing and designing of MD Platform development priorities ensuring fast

adaptation of work performed within other activities of the project following NA3 and

SA3 activities requirements and recommendations. Based on the analysis required

functionalities will be designer and implemented

• User-level support for Grid advanced data management

Research, design and implementation of user friendly data management mechanisms

handled by grid services. Providing intuitive tools for accessing and managing this data

via Migrating Desktop Platform.

• Adaptation of mechanisms used for definition and management of task flow

Analysis of existing mechanisms used for handling of application based on steering

flow between jobs. Adaptation of chosen mechanism for BG-II application purposes

and integration within MD Platform

• Ensuring a proper way of further developments by following the existing interface and service

standards

Ensuring compatibilities with standards and ‘de facto’ standards in case of their

changes

.

• Support of the education process organised by NA2

Close collaboration with NA2 for supporting the education process organised via BalticGrid-II tutorials, summer schools and local workshops

• Providing graphical data visualization tools for scientific computing in Grid

In grid environment jobs outcome are stored in distributed storages, so usage of standard data visualization programs is not possible. In this case the transmission and interpretation of results become crucial. Moreover, development of special mechanisms for remote visualization of application results is necessary.

• Development of Gridcom - a web-based innovative groupware for grid technology

Groupware functionality of Gridcom – a user-friendly environment used for cooperation of user groups and applications – will be enhanced.

(7)

DESIGN PHASE

2.2. RESPONSIBILITIES AND ORGANIZATION OF WORK

Within the JRA activity three independent components will be developed by consortium partners: • Poznan Supercomputing and Networking Center (Coordinator of the activity) - Extension of

Migrating Desktop Platform

• Vilnius University - Development of Gridcom – web-based innovative groupware for grid technology:

• Vilnius Gediminas Technical University - Visualisation of specific computing results

At the beginning of the project, very important task was to establish efficient activity team, defining effective internal collaborations and communications rules. As a result of internal discussions during BalticGrid-II Kick-off Conference the collaboration framework, and detailed activity work plan regarding to milestones and deliverables as well as other organisational issues were established. Due to small number of partners participating in activity works, and because of the independence of individual tasks we try to keep organizational overhead as small as it is possible. However regular teleconferences and face-to-face meetings will be scheduled where progress and possible issues are to be discussed and coordinated.

(8)

DESIGN PHASE

3. DESIGN OF COMPONENTS DEVELOPMENT

3.1. MIGRATING DESKTOP PLATFORM

3.1.1. Component description and current state

The Migrating Desktop Platform is a powerful and flexible user interface to Grid resources

that gives a transparent user work environment and easy access to resources and network file

systems independently of the system version and hardware. It allows the user to run

applications and tools, manage data files, and store personal settings independently of the

location or the terminal type. It is an advanced graphical user interface similar to a

window-based operating system that hides the complexity of the grid middleware and makes access to

the grid resources easy and transparent.

Fig. 1 Migrating Desktop main window

The key feature of the Migrating Desktop is the possibility of easy adding various tools,

applications and supporting different visualization formats. The Migrating Desktop offers a

framework that can be easily extended on the basis of a set of well-defined plug-ins used for:

accessing data, defining job parameters, pre-processing job parameters, and visualization of

job results. Open architecture of the Migrating Desktop speeds up the application integration

process and makes that product significantly more flexible than specialized tools (e.g. portals)

designed only for a specific application.

(9)

DESIGN PHASE

Fig. 2 Job defining and visualisation of results (examples of Migrating Desktop plug-ins

The integration of many applications, come from various projects and science areas, proved

correctness of platform integration mechanisms and procedures.

Current state

Migrating Desktop was created in The EU CrossGrid Project (http://www.crossgrid.org/) and

developed in The EU Interactive European Grid Project (http://www.interactive-grid.eu/),

where functionality for handling interactive grid applications was implemented. It also proved

its usefulness in everyday work of BalticGrid users. Currently it is developed in EU The

BalticGrid Second Phase project (http://www.balticgrid.org/).

Current version of Migrating Desktop offers following functionality: •

Single sign-on mechanism

Single sign-on mechanism for authentication and authorization to all supported

infrastructures using VOMS proxy – once authenticated user can access resources of

various grids

• User profile management

User profile data are stored on network server, so personalized working environment is

available independently of the user location.

• Support for various computing infrastructures

Open architecture of Migrating Desktop and mechanism of well defined plug-ins and interfaces makes available interoperations of various computing architectures in terms of data handling, job submission and monitoring (currently supported infrastructures: gLite, i2g).

• Job handling

Migrating Desktop Platform handles the whole job’s life-cycle from job defining and submission (using wizards - general and dedicated to specific application),

job state monitoring and

v

isualization of job results. It provides also mechanism (based on concept of plug-ins) that allows easy integration of applications within Migrating Desktop

(10)

DESIGN PHASE

• File management

Possibility of performing basic operations on files (file downloading, uploading,

renaming, removing, etc) stored on various data management systems (gLite SEs,

GridFTP servers ,etc) and file transfer between different storages.

During BalticGrid (First Phase) the Migrating Desktop Platform was enhanced to ensure

compatibility with BalticGrid environment and successfully deployed within project

infrastructure. The platform framework was tuned for quick and simple integration with

BalticGrid applications. Integration procedures were precisely described. To proved

correctness of integration mechanisms and procedures, pilot application (GAMESS,

SYNTSPEC and SemtiKamols) were integrated within Migrating Desktop Framework. In

everyday work MD proved its usefulness as a tool that makes grid application usage much

easier and more intuitive which is essential for the encouraging potential beneficiaries to reap

profits from using grid infrastructure for compute- and data-intensive applications.

3.1.2. Software development design

The Migrating Desktop Platform is an evolving environment, so there are some features

recognized as important from users (or applications developers) point of view, which are still

not supported. Based on experiences gained during BalticGrid-I and taking into consideration

needs of users and application developers, existing functionality will be extend with special

emphasis put on:

• Support for advanced data management • Support for advanced applications

• Empowering GUI with the new features requested by other project activities • Support for interoperation between computing infrastructures

Support for advanced data management

• Separating GridCommander as an independent network application

A lack of intuitive tools supporting management of data in grid environments could be noticed. To fill this gap we decide to separate GridCommander as an independent network application. Grid Commander is one of tools constituting Migrating Desktop Platform. It is a graphical tool based on the idea of well-known applications like Total Commander or Midnight Commander dedicated to file management. Providing this tool as independent, lightweight application that could be loaded from any location (Java applet) could be useful for the users which don’t required other functionality of MD. The separation itself will be done using idea of well defined (and already used) MD plugins based on OSGi mechanisms, to ensure further seamless cooperation between GridCommander and Migrating Desktop.

• Mechanism for easy integration of various file systems

Available version of Migrating Desktop Platform supports several file systems: local, FTP, GridFTP, SRMv2.2, LFN. Currently Migrating Desktop offers well defined interface that have to be implemented for adding new file system. Unfortunately code changes and deployment of new version of MD is required to make support for new data management systems available. To make

(11)

DESIGN PHASE

integration of various file systems easier, plug-in mechanism will be designed and implemented. Thanks to open architecture developers will be able to add their own implementation of interface between particular file system and MD to integrate it within MD framework.

• Support for data access mechanisms used within Grid infrastructure

Some of the data access mechanisms (e.g. Unicore data management mechanism and sFTP protocol) were identified as useful for the grid user but not supported by MD. As one of the JRA tasks support for these protocols will be implemented using existing libraries and plug-in mechanism described above.

Advanced data management functionality will be provided as a part of the prototype DJRA1.3

“First Prototype” and delivered until M9 (9th_{month of the project)}

Support for advanced job management

• Handling “multiply jobs”

Possibility of submitting of set of jobs differing only by values of input parameters (so called “multiply job”) was recognized as one of features required by users. Implementation of such functionality will be closely related to next feature: “adaptation of mechanisms used for definition and management of task flow”.

• Adaptation of mechanisms used for definition and management of task flow

Current version of Migrating Desktop allows only to submit separate, independent applications. Lack of possibility of defining compound applications based on flow of results between separate jobs was one of gaps identified during BalticGrid-I works. Implementation of such complex mechanism “from scratch” significantly exceeds BalticGrid -II JRA resources, so an existing mechanism used for definition and management of task flow will be search and adopted. This work will be divided into two parts: adaptation of existing graphical tool for workflow defining and preparation of server-side mechanism

• Empowering GUI with the new features requested by other project activities

The Joint Research Activity closely cooperates with other activities of the project. Strict collaboration with NA2 and SA3 activities provides information about the needs of new applications and user communities. Existing MD framework will be extended (according to available JRA resources) to follow NA3 and SA3 activities requirements and recommendations. Support for complex jobs will be provided as a part of the prototype DJRA1.4 “Second Prototype” and delivered until M18 (18th month of the project)

Support for interoperation between computing infrastructures

• Features for interoperation support

Development of features for interoperation between various computing infrastructures was not foreseen as one of the tasks of JRA activity. However requirements coming from SA3 and NA3 shown usefulness of integration with ARC and Unicore infrastructures, so works on that functionality is planned.

Currently available functionality allows Migrating Desktop Platform to act as a bridge between various computing infrastructures providing an user transparent and intuitive access to heterogeneous grid resources. MD framework will be extended to make process of adding functionality, that intermediates between MD and particular infrastructure, easier and not so time consuming.

(12)

DESIGN PHASE

A plug-in that acts as an interface between Migrating Desktop and ARC will be implemented to ntegrate ARC middleware in terms of job submission, monitoring and data handling.

• Integration of Migrating Desktop Platform with Unicore infrastructure

Integration of Migrating Desktop Platform with Unicore infrastructure requires similar action as those taken for integration with ARC

Support for interoperations between computing infrastructures was not planned while preparing proposal, so works on this issues will be performed “in background” to not interfere with other tasks.

3.2. GRIDCOM

3.2.1. Component description and current state Description

Gridcom is a simple web interface for launching applications on grid. The main differences between gridcom and similar systems are:

1. Gridcom is used to launch applications, not single jobs. Each application can use as many jobs as it is needed. Application here means the specific research area, where group of researchers work or group of related jobs for this area.

2. Gridcom allows group access to the same working space (so representing groupware properties).

While using gridcom, user’s application is being formulated as a special gridcom package. In addition to job code it contains control code, for control launched application instance—scatter input data into intervals, generate jobs, collect job outputs, provide interface for gridcom users, provide result for visualization and reporting. The system submits application generated jobs into grid, it controls their states, resubmits aborted jobs, receives the results. While process time user may turn off his computer as all application controlling code is executed on a dedicated gridcom machine and don’t require any attention. User has only to initiate application execution (specify parameters, input files).

Group access feature can be used for launching/controlling an application for a group of users. Because of users launch only specially written gridcom applications, they cannot harm grid. The security is arranged on a level of operating system (Linux). Group of users have access to jobs and computing resources via login, password and certificate.

In order to make potential users to be more familiar with Gridcom functionality, an example of group access is given, by a test gridcom instance (http://sig.balticgrid.org/gridcom/test/). In this example everybody can access it without any password, launch installed applications, get the results. However, users cannot upload their own applications and, for example, specify large submitted job count.

Group access allows watching execution process of one launched application by a group of users. Upon complete, group member can download the results, view reports and data visualizations.

Group access also allows easier user support, because in case something goes wrong, administrator just opens Gridcom instance of the user and sees all launched applications with log files. There is no need for a user to collect log files and to send them to grid administrators

(13)

DESIGN PHASE

Current state:

At the present time Gridcom is used for some applications. It allows to launch applications and to access working space by a group of people. This group can watch application execution, use the results, reports, result visualizations. However, these features require some enhancements. Gridcom also requires some new features.

3.2.2. Software development design

The enhancement of Gridcom will be done in following directions:

• Referential application will be modernized to print failed gLite UI command output. Now it just prints the fact that an operation has failed and tries to repeat the operation. To find the reason user has to look log files.

• Mobile access for watching the process of executing an application will be implemented. Mobile access makes sense when an application is executed for days and a user waits for it to be

completed.

• An interface for managing group users will be implemented. Now only Gridcom administrator can create users and modify passwords.

• A simple web interface for group access to LFN grid file system will be implemented. This can be useful for users who wish to store large files in grid but do not want or do not know how to use gLite UI commands for that. Gridcom will allow uploading and downloading files, creating and deleting directories by using web browser. This can be used to share files on grid.

3.3. VISUALIZATION TOOLS

Considering the results of performance analysis presented in DJRA1.1, three visualization tools are considered in this chapter:

• ParaView software • E-Service VisLitG

• Visualization tool for Particle Systems

Deployment of ParaView software and e-Service VisLitG in BalticGrid-II provide for numerous users enchanted application services aimed for visualization of computed results. Development of special purpose visualization tool targeted for visualization of particle systems and crack propagation in building structures is very important for scientist using the Discrete Element Method and solving industrial applications. Special application area and complex algorithms present challenge to developers and add scientific value to performed research.

3.3.1. Component description and current state 3.3.1.1. ParaView software

ParaView [1] is an open-source application designed to visualize large datasets. ParaView supports hardware-accelerated parallel rendering and achieves interactive rendering performance via level-of-detail (LOD) techniques. ParaView runs on distributed and shared memory parallel machines as well as single processor PC and has been successfully tested on Windows, Mac OS X, Linux and various

(14)

DESIGN PHASE

Unix workstations, clusters and supercomputers. Under the hood, ParaView uses the VTK [2] as the data processing and rendering engine and has a user interface written using Qt [3].

ParaView is designed as layered architecture. The foundation is VTK, which provides data representations, visualization algorithms, and a mechanism to connect these representations and algorithms together to form a working program. The second layer is the parallel extensions to the VTK. This layer extended VTK to support data streaming and parallel computing. These extensions are currently part of the toolkit. The third layer is ParaView itself. ParaView is designed as a three-tier client-server architecture. The three logical units of ParaView are as follows:

• Data Server is the unit responsible for data reading, filtering, and writing. All of the pipeline objects seen in the pipeline browser are contained in the data server. The data server can be parallel.

• Render Server is the unit responsible for rendering. The render server can also be parallel, in which case built in parallel rendering is also enabled.

• Client is the unit responsible for establishing visualization. The client controls the object creation, execution, and destruction in the servers, but does not contain any of the data (thus allowing the servers to scale without bottlenecking on the client). If there is a GUI, that is also in the client. The client is always a serial application.

These logical units need not by physically separated. Logical units are often embedded in the same application, removing the need for any communication between them. The client-server mode (Fig. 3) is the most suitable for BalticGrid-II users. The pvserver program is executed on the considered grid cluster. The server connects to the Paraview client via the established socket in the reverse connection mode. The pvserver program has both the data server and render server embedded in it, so both data processing and rendering take place there. A socket is assumed to be a relatively slow mode of communication, so data transfer over this socket should be minimized.

Fig. 3 Client – server mode of ParaView software deployed for BalticGrid-II users

In standalone mode, the client, data server, and render server are all combined into a single serial application. It is the most suitable for local rendering of small datasets. In client-render server-data server mode, all three logical units are running in separate programs. As before, the client is connected to the render server via a single socket connection. The render server and data server are connected by many socket connections, one for each process in the render server. Data transfer over the sockets is minimized.

ParaView supports large data visualization via techniques that include data streaming, LOD rendering and parallel computing. ParaView supports parallelism using either shared memory processes via threads or distributed memory via MPI. ParaView uses a parallel rendering library called IceT. It uses a sort-last algorithm for parallel rendering. This parallel rendering algorithm allows each process to independently render its partition of the geometry and then composites the partial images together to

(15)

DESIGN PHASE

form the final image. The wonderful thing about sort-last parallel rendering is that its efficiency is completely insensitive to the amount of data being rendered. This makes it a very scalable algorithm and well suited to large data.

3.3.1.2. E-Service VisLitG

Visualization e-Service VizLitG is being developed in GirdTechno project supported by the Lithuanian State Science and Studies Foundation [4]. It is designed for convenient access and efficient visualization of results produced by scientific computations in LitGRID [5]. The client-server architecture of the system is based on web standards. The developed visualization system is targeted for datasets of medium size. A visualization pipeline is distributed between the client and the server. Data filtering and mapping is performed on the server, therefore, enough powerful server is required. Generated geometry can be transferred to the client and rendered locally. For the most applications modest network bandwidth is sufficient. Version of the system transferring image to the client is under development. The server runs on the special UI named UIG (User Interface for Graphics). Thus, natural access to the grid resources is available.

Data is stored in HDF5 [6] files that can be downloaded from different locations. Several possibilities are available:

• Data files reside locally on UIG, • Full data files are transferred from SE,

• Interactively chosen parts of datasets are transferred from SE (under development), • Full data files are transferred from WN,

• Interactively chosen parts of datasets are transferred from WN (this advanced option of computational steering is under development).

The visualization engine of the VizLitG is based on VTK toolkit. VTK modules are enwrapped by Java programming language and build the service running on the server. Java EE 5 platform [7] provides tools for remote client-server communication. The visualization service is implemented in GlassFish [8] application server developed by Sun Microsystems. GlassFish is integrated in Netbeans 6.0 IDE. Services are developed by using convenient tools such as web service designer and web service tester. WSDL [9] interface, client stub and server skeleton are generated automatically. Governing parameters from the client to the server are transferred by high level SOAP protocol Contrariwise, the geometry written to X3D files can be transferred from the server to the client by SCP. This server-client communication is implemented by jsch-0.1.37 library [10], which is a pure Java implementation of SSH2.

(16)

DESIGN PHASE

Fig. 4 Service implementation and data-flow control in GalssFish application server

The client implemented as Java application consists of GUI and Viewer. Tabular design of GUI is developed by using Java Swing [11]. It allows interactive reading selected datasets from HDF5 file. Time dependent and time independent data is clearly separated and processed differently. User can employ basic VTK filters and choose appropriate visualization strategy. Currently viewer is based on the Xj3D browser [12], which reads transferred X3D files and renders them on screen. Rendering is completely performed on the client, therefore, sufficient interactivity level is provided.

3.3.1.3. Visualization tool for Particle Systems

Particle systems have no permanent connections or usual mesh that can be applied for spatial discretization or visualization purposes. Usually particles are represented by glyphs that can be colored by investigated scalar values or oriented by the examined vector values. Discrete element computations are based on particle positions, forces acting between particles and Newton’s laws. Initial defects are identified between pairs of neighbouring particles, that can only form 1D connections in 2D or 3D space. The resulting physical phenomenon has no continuous structure. 1D connections are not suitable for reliable interpolation and standard visualization. Later defects form propagating cracks, that can be represented by 1D curves (line strips) or 2D surfaces (triangle strips) in 3D space. The usual 3D grid might be necessary for detection of cracks and construction of these line strips or triangle strips. During the latest stage particle groups can separate from each other and form 3D volumetric cracks in 3D space.

In order to visualize particle systems on grid, the correponding VTK application based on glyph filter has been written in C++ language. Developed VTK application has been implemented in grid environment by using GVid sowtware as video-streaming module [13]. As a result developed VTK application runs on WN while the compressed video stream is efficiently transferred through the network and displayed on client. Remote user has full interactivity provided by VTK application. GVid can be treated as middleware independent software and has been run on gLite based grid.

(17)

DESIGN PHASE

3.3.2.1. ParaView software

In DJRA1.1 presented benchmark tests reveal that ParaView software is well designed for parallel visualization and parallel rendering of large datasets. It can be very useful for BalticGrid users performing large parallel computations of actual industrial problems like sediment transport, nano-powders, compacting, mixing, hopper discharge and crack propagation in building constructions. Considering results of performance analysis, deployed software should be targeted for CPU rendering, but available GPU rendering would be considered as an advantage. It raises requirements for BalticGrid testbed necessary for validation and testing of developed and deployed visualization tools. The ParaView version 3.2.1 should be deployed in BalticGrid-II testbed for visualization until M08 (8-th mon(8-th of (8-the project). The first prototype will be described in DJRA1.3.

In DJRA1.1 presented performance analysis reveals that parallel sped-up is not increased reading data by automatic readers. The highest efficiency can by achieved by special purpose readers that are adapted to the nature of a parallel application. Parallel software packages considered in BalticGrid-II application group named Engineering Modelling Tasks produce multiple HDF5 files. Each process works on data partition generated considering the whole load balance. Process produce separate file, which can be efficiently stored on the file system of local node. Developed unstructured HDF5 reader can process these data files in parallel. In this case high speed-up will be achieved automatically. Development of HDF5 readers for unstructured datasets is crucial for parallel performance of considered visualization. This work will be accomplished by JRA team until M14 (14-th month of the project). The most suitable version of ParaView software with developed readers should be deployed in BalticGrid-II testbed for visualization until M18 (14-th month of the project). The results will be described in DJRA1.4 (the second prototype). Final assessment of the software performance will be made until M23 (23-th month of the project) and will be described in DJRA1.5.

3.3.2.2. E-Service VisLitG

VizLitG has flexible environment and remote instrumentation of e-Service provided by Java EE and GlassFish. It can be employed by BalticGrid users for convenient access and visualization of results produced by scientific computations in grid. The sequential Java code of VizLitG system is targeted for visualization of small and medium datasets. Thus, application area of VizLitG is quite different than that of ParaView. Moreover, remote instrumentation of e-Service provides for users flexible access of remote data files located in grid. Transfer of interactively chosen parts of datasets located in SE instead of the whole data files can save significant amount of visualization time [15]. Computational steering also belongs to advanced functionalities of visualization.

However, the client-server architecture delivering geometry to client for local rendering is not suitable for the pilot application of poly-dispersed particle systems. In order to be employed for visualization of particle systems e-Service VizLitG should be redesigned for image delivery support. Currently this feature of VizLitG is under development. The version supporting image delivery to the client is expected to appear at the end of this year. The e-Service VizLitG will be deployed in BalticGrid by joint JRA and SA1 team of VGTU until M09 (9-th month of the project). The first prototype of the e-Service will be described in DJRA1.3. VizLitG is being developed in the other project, therefore, it is not planned to be enchanted by JRA developers in BalticGrid. However, deployment of LitVizG in BalticGrid-II encourages constructive cooperation with NGI.

3.3.2.3. Visualization tool for Particle Systems

Currently, only initial version of visulization tool for particle systems is developed. It is based on VTK toolkit, therefore, the newest VKT version 5.2 usefull for successful development of advanced features should be deployed in BalticGrid-II testbed for visualization until M08 (8-th month of the project). Sequential application for glyph generation was mainly targeted for performing benchmark tests. Advanced tool needs extended functionality including special features:

(18)

DESIGN PHASE

• Visualization of forces acting between particles and other pheasical features, • Visualization of initial defects identified between pairs of neighbouring particles, • Identification and visualization of propagating cracks.

Visualisation of forces and initial defects between pairs of neighbouring particles needs additional data from the post-processing stage of Discrete Element Method. Marked edges can be extracted from the employed representation and plotted by using vtkTubeFilter. This functionality will be added to the tool until M15 (15-th month of the project).

In later stages, initial defects form propagating cracks that can be represented by 1D curves or 2D surfaces in 3D space. Detection of cracks and construction of their appropriate geometric representation by line strips or triangle strips presents main challenge for development of advanced visualization tool. Particle systems handled by Discrete Element Method (DEM) have no permanent connections. The usual 3D mesh might be required for detection of cracks and construction of their polygonal representation. If necessary, 3D connections will be generated by vtkDelaunay3D filter, which performance analysis is provider in DJRA1.1.

Parallel version of the visualization tool suitable for efficient distributed rendering of large number of particles will be developed by JRA team of VGTU until M17 (17-th month of the project). Resulting application will be based on data parallel model of VTK. Final image will be collected on the zero node of MPI based visualization tool. Then compressed video stream is efficiently transferred through the network and displayed on client (Fig. 6).

Fig. 5 SDL output client of GVid software. Visualization of mono-dispersed particles

As the initial application, parallel software will be implemented in grid environment by using GVid software or related i2glogin software [16]. They should be deployed in BalticGrid-II testbed for visualization until M08 (8-th month of the project). GVid classes responsible for rendering and interaction will be revised. In order to sucsessfully employ GVid for development of special purpose visualization tools based on VTK, the software should be renewed to support VTK 5.x versions. The

(19)

DESIGN PHASE

last GVid development was targeted for VTK 4.4.2. In general, VTK 5.x versions have significant architectural changes that innovated even final view of user applications. It is necessary to check how significant changes have been made in two most importand clases of GVid. If corresponding GVid classes need significant redevelopment, XWin-GrabServer of GVid or i2glogin can be employed to achieve the same goals.

Currently, only sequential version of the tool including minimal functionality is developed. The advanced functionality will be added and the final version of the visualization tool for particle systems will be deployed in BalticGrid-II testbed for visualization until M18 (18-th month of the project). The results will be described in DJRA1.4 (the second prototype). Final assessment of the parallel performance will be made until M23 (23-th month of the project) and will be described in DJRA1.5.

3.3.2.4. Integration of visualization tools in grid environment

Visualization can be treated as the final stage of numerical analysis or any data processing. Usually grid users prepare their data, transfer data files, perform computations and, finally, visualize obtained results. Very often these basic steps are enclosed into the loop of scientific analysis. In order to perform research efficiently, all steps can be performed by using graphical grid environments. Visualization tools have their own graphical interfaces, but usually they lack extended functionality necessary for the whole loop of numerical analysis or data processing.

Migrating desktop is a good candidate for the discussed purpose, because it provides extended functionality for efficient and convenient work on grid. All developed and deployed visualization tools inherit the client-server architecture. Thin visualization clients run on the user PC as well as the MD client does. MD client can run any executable like ParaView client or GVid SDL client on the user PC. Java application serving as a client of LitVizG can be implemented in MD as a plug-in. In the later stage MD can submit the visualization jobs like ParaView server to grid by using its tools. The final image from WN can be transferred to the client by GVid software. JRA team of PSNC has necessary know-how and valuable experience performing such implementation. Joint JRA team of PSNC and VGTU will integrate visualization tools with MD until M20 (20-th month of the project).

(20)

DESIGN PHASE

4. INTEROPERABILITY AND COOPERATION WITH OTHER ACTIVITIES

The Joint Research Activity works are strictly related to tasks performed within other activities of the project so the interoperability and cooperation between activities is the key issue. JRA closely cooperates with the SA3 providing necessary user support regarding JRA components, and actively participate in NA2 educational and disseminative activities. JRA supports also SA1 and NA3 activities, because tools, developed within JRA will be commonly used in everyday work of application users.

4.1. NA2 “EDUCATION, TRAINING, DISSEMINATION AND OUTREACH”

The NA2 “Education, training, dissemination and outreach” activity is aimed at “ensuring expertise sharing and active dissemination of the benefits of Grid-empowered e-Infrastructures (especially of BalticGrid-II) and to carry out the necessary education to build active and successful user communities in the Baltic States and Belarus”1_.

Very important aspect of JRA works is wide dissemination of the results in scientific community and show the usefulness from the user point of view. This task is executed in strict co-operation with NA2 activity. The conferences, summer school and other events organized by NA2 activity are the good opportunity for the application users and developers to familiarize with functionality (services) delivered for the research community by JRA.

Other actions include:

• Providing all necessary information required by NA2 for preparing dissemination materials (project presentations, brochures, posters, and other promotional items).

• Active support of NA2 in preparation of BG-II demos shown during conferences, workshops and other dissemination events. The latest demo “Using the BalticGrid-II infrastructure: SemtiKamols - a linguistic analyser of Latvian language” was shown during EGEE’08 in Istanbul

• Preparation, as a part of BalticGrid-II portal, web pages describing JRA activity, its research and components developed. These pages will contain information on the role of the given partner and its achievements within the Project.

• Organisation of local seminars

• Wide dissemination of functionality (services) delivered for the research community by JRA • Promoting visualization tools within grid users from different application areas. For example, ParaView is well known and widely used for parallel visualisation of large datasets. NA2 team of VGTU joined together with people from JRA and NA3 is going to prepare brochure on developed and deployed visualization tools at the end of the BalticGrid-II project M22 (22-th month of the project). Show cases of Visualization tool for Particle Systems, ParaView software and E-Service VisLitG might be interesting for different communities from special interest groups. Considered show cases will be prepared by NA2 team and JRA team of VGTU until M20 (20-th month of the project).

Plan for education trainings in co-operation with NA2

JRA will actively support the education process organised by NA2 activity. The most important task for JRA is providing lecturers to tutorials, summer schools and local workshops to teach direct use of results achieved by JRA as well as advertising the use of the JRA services.

(21)

DESIGN PHASE

The plan of education and dissemination activities and events is described in details in deliverable DNA2.2 “Dissemination Roadmap”, so to avoid redundancy it is not repeated in this chapter.

4.2. NA3 “APPLICATION IDENTIFICATION AND COLLABORATION”

The NA3 activity - “Application Identification and Collaboration” – is focused on identification of applications that could benefit from usage of grid infrastructure, supports their development and ensures effective collaboration between their users and developers.

Experience from previous project shows that user-friendly and intuitive access to Grid environment is a very important to attract user to use grid technology. So, JRA activity is especially important from NA3 point of view, because tools, developed within JRA will support the everyday work of application users providing intuitive graphical interface to project resources and enabling effective collaboration between scientists. From the other side the user expectations and requirements give the driving forces to JRA works.

Visualization tool for Particle Systems will be developed for the particle technology applications that belong to the BalticGrid-II application group named Engineering Modelling Tasks. Particle systems have been chosen as a pilot application for the visualization tool, because of large number of particles that are employed in discrete element computations of actual industrial applications. The most interesting examples include nano-powders, compacting, mixing, hopper discharge and crack propagation in building constructions.

ParaView software is well designed for parallel visualization and parallel rendering of large datasets that are common in Computational Fluid Dynamics. Applications of convective transport, viscous incompressible flows and moving interfaces belong to the BalticGrid-II application group named Engineering Modelling Tasks. Sluice gates, dam break flows, tank sloshing, sediment and pollution transport has been modelled by the developed engineering software. Deployment of ParaView package in BalticGrid will be very useful for fast and efficient visualisation of datasets resulting from applications of Engineering Modelling Tasks.

4.3. SA1 “GRID OPERATIONS”

The main task of the SA1 “Grid Operatons” activity is “to provide a sustained, high quality Grid infrastructure in the Baltic States and Belarus”.

Because SA1 is providing services residing on low tier of project architecture in opposite to JRA which focuses on “high level” tools (JRA is developing tools and components that are running on the infrastructure provided by SA1 activity), only very loosely cooperation between activities is foreseen. Examples of tools that need testbed suitable for interoperability investigation and benchmarks, are ParaView software and visualization tool for particle systems. Considering results of performance analysis, deployed software should be targeted for CPU rendering, but available GPU rendering would be considered as an advantage. Several MPI implementations should be available in BalticGrid testbed for visualization, because both tools employ parallel visualization and parallel rendering. SA1 team of VGTU is going to build the testbed satisfying described requirements for validation and testing of developed and deployed visualization tools until M07 (7-th month of the project). Joint SA1 and JRA team of VGTU will deploy considered visualization tools in the BalticGrid-II testbed for visualization.

4.4. SA3 “APPLICATION INTEGRATION AND SUPPORT”

The main goal of SA3 activity “Application integration and support” is “to support scientists using the BalticGrid infrastructure and to foster the use of Grid by new research communities”.

(22)

DESIGN PHASE

The services developed within JRA foster the new research communities supported by SA3 - one of the activity objectives strictly related to JRA works is deployment on BalticGrid infrastructure advanced services and tools developed by JRA1 to support user work in grid environment

• Preparation of procedures of BG-II applications procedures of integration with components developed within JRA (user friendly grid interface – Migrating Desktop, workgroup tools – GridCom, visualization tools). Procedures will base on documents and experiences achieved during BalticGrid-I project

• Support for integration of BG-II applications with components developed within JRA; • Preparing manuals and user guides concerning JRA tools

• Training the support personnel: integration of application with JRA tools, usage of JRA tools

• Publishing manuals and procedures descriptions on BalticGrid-II User pages and the BalticGrid-II web page.

(23)

DESIGN PHASE

5. SUMMARY AND CONCLUSIONS

The Joint Research Activity “Enhanced Application Services on Sustainable

e-Infrastructure” aimed at extending the project infrastructure with advanced services

through enhancement of: Migrating Desktop Platform (intuitive interface to project

resources), Gridcom (web-based groupware) and special tools for visualization of application

results in distributed grid environment

Within this document these main components were described. Keeping in mind goals of the

Joint Research activity, the directions of development and detailed plans for enhancement

were designed for every component.

Internal discussions as well as discussions with representatives of other activities lead to

design of detailed plans of interoperability and cooperation between activities and shows

efficient collaboration as an important factor that guarantees successful project results.

(24)

DESIGN PHASE

6. REFERENCES

[1] Squillacote, A. ParaView Guide. Version 3. Kitware, Inc., (2008). [2] VTK User's Guide Version 5, Inc. Kitware. 5th Edition, (2006). [3] Qt: http://trolltech.com/products/qt/

[4] VMSF: http://www.vmsfondas.lt/index.php?lang=en [5] LitGRID: http://www.litgrid.lt/

[6] HDF5: http://hdf.ncsa.uiuc.edu/products/hdf5/ [7] Java EE: http://java.sun.com/javaee/

[8] GlassFish: https://glassfish.dev.java.net/ [9] WSDL: http://www.w3.org/TR/wsdl [10] JSCH: http://www.jcraft.com/jsch/

[11] Fischer,P. Introduction to Graphical User Interfaces with Java Swing. Addison Wesley, (2005). [12] Xj3D: http://www.xj3d.org/

[13] GVid: http://www.gup.jku.at/gvid/

[14] http://www.teragrid.org/userinfo/data/vis/pv_overview.php

[15] DJRA1.1: Report on Performance analysis of sequential and parallel visualization in grid environment, August 2008.

D ESIGN P HASE. Activity: Lead Partner: Document classification:

Document Filename: BGII-DJRA1.2-v1.0-PSNC-DesignPhase.doc

DESIGN PHASE

Document review and moderation

Contents

2.1.GOALS OF THE ACTIVITY... 6

3. DESIGN OF COMPONENTS DEVELOPMENT ... 8

3.1.MIGRATING DESKTOP PLATFORM... 8

4. INTEROPERABILITY AND COOPERATION WITH OTHER ACTIVITIES ... 20

4.1.NA2“EDUCATION,TRAINING,DISSEMINATION AND OUTREACH”... 20

5. SUMMARY AND CONCLUSIONS... 23

DESIGN PHASE

Three main components are developed within activity:

mechanisms for cooperation of user groups and applications,

interoperability and cooperation between activities.

work could attract the grid users and make their work much more convenient. The need of

enabling visualization of computing results in grid environment.

SA3 activities requirements and recommendations. Based on the analysis required

via Migrating Desktop Platform.

Ensuring compatibilities with standards and ‘de facto’ standards in case of their

DESIGN PHASE

DESIGN PHASE

3.1.1. Component description and current state

The Migrating Desktop Platform is a powerful and flexible user interface to Grid resources

window-based operating system that hides the complexity of the grid middleware and makes access to

Fig. 1 Migrating Desktop main window

The key feature of the Migrating Desktop is the possibility of easy adding various tools,

process and makes that product significantly more flexible than specialized tools (e.g. portals)

Fig. 2 Job defining and visualisation of results (examples of Migrating Desktop plug-ins

The integration of many applications, come from various projects and science areas, proved

Current state

Migrating Desktop was created in The EU CrossGrid Project (http://www.crossgrid.org/) and

Single sign-on mechanism

available independently of the user location.

job state monitoring and

During BalticGrid (First Phase) the Migrating Desktop Platform was enhanced to ensure

SYNTSPEC and SemtiKamols) were integrated within Migrating Desktop Framework. In

3.1.2. Software development design

The Migrating Desktop Platform is an evolving environment, so there are some features

Support for advanced data management

DESIGN PHASE

Support for advanced job management

Support for interoperation between computing infrastructures

DESIGN PHASE

3.2. GRIDCOM

3.2.1. Component description and current state Description

DESIGN PHASE

Current state:

3.3. VISUALIZATION TOOLS

3.3.1. Component description and current state 3.3.1.1. ParaView software

DESIGN PHASE

Fig. 3 Client – server mode of ParaView software deployed for BalticGrid-II users

DESIGN PHASE

3.3.1.2. E-Service VisLitG

DESIGN PHASE

Fig. 4 Service implementation and data-flow control in GalssFish application server

DESIGN PHASE

3.3.2.1. ParaView software

3.3.2.3. Visualization tool for Particle Systems

DESIGN PHASE

Fig. 5 SDL output client of GVid software. Visualization of mono-dispersed particles

DESIGN PHASE

3.3.2.4. Integration of visualization tools in grid environment

DESIGN PHASE

4.1. NA2 “EDUCATION, TRAINING, DISSEMINATION AND OUTREACH”

Plan for education trainings in co-operation with NA2

DESIGN PHASE

4.3. SA1 “GRID OPERATIONS”

DESIGN PHASE

DESIGN PHASE

resources), Gridcom (web-based groupware) and special tools for visualization of application

Internal discussions as well as discussions with representatives of other activities lead to