• No results found

The evolution of DAVE

N/A
N/A
Protected

Academic year: 2021

Share "The evolution of DAVE"

Copied!
25
0
0

Loading.... (view fulltext now)

Full text

(1)

The evolution of DAVE

The evolution of a large IDL application

Data Analysis and Visualization Environment (DAVE) project.

Richard Tumanjong Azuah

Larry Kneller

Yiming Qui

John Copley

Robert M Dimeo

Craig Brown

Philip Tregenna-Piggott

David Fanning

Craig B. Markwardt

(2)

NIST Center for Neutron Research

What is the NCNR?

 A neutron scattering facility

 Used by roughly 2000 scientists a year  20 neutron scattering instruments available

 For materials research – eg atomic structure, dynamics and magnetism.

How it works.

 Users apply for beam time  Proposals are peer review

 Limited beam time (a few days) awarded to best ones  User brings own sample and works with a staff scientists

 Data is generated – counts as a function of angle, time, energy, temperature, magnetic field, etc

 Raw data is instrument dependent – instruments are uniquely optimized to cover different regimes or

(3)

Data Treatment Process

The 3 main wishes

D

ata Reduction – convert raw data to a universal

scattering function, S(Q,E).

Q → atomic structure E → atomic motion

Data reduction is instrument dependent

 Visualization of raw or reduced data can

provide useful information

 Analyse reduced data – extract parameters

that determine physical quantities

atomic structure, magnetic moments, vibrational motion, etc.

Analysis involves a mix of basic data operations, data modelling, visualization, etc.

(4)

Origins of DAVE Project

Motivation

Seven years ago at the NCNR:

there were few or non-existent software tools for basic data reduction and analysis.

traditional approach of user-written software was time-consuming.

complexity steadily increasing with improvements in spectrometer designs. Software

(5)

Origins of DAVE Project

A pragmatic solution

DAVE project created in 2001 to address shortcomings.

Goals:

 User-friendly tool which can accommodate the software requirements for a range of instruments at

the NCNR.

 Full availability at no cost to users.

 Instrument scientist should play key role in development, particularly data reduction aspects.

IDL chosen as development platform

 Full support on several OS platforms

 Comprehensive support for scientific programming and optimized for large array operations.  Built-in widget toolkit for providing GUIs

 Some IDL programming expertise

 License agreement with RSI Inc allows free and unlimited distribution of DAVE executable with an

(6)

The DAVE suite

Main Application

 Defines and maintains application wide user preferences (eg data directories, useful symbols, etc)  Define an internal data structure for storing experimental data

 Basic infrastructure to launch modules and pass data and to display PDF based help files.

Main Parts

 Data Reduction  Data Visualization  Data Analysis -  least squares fitting  Planning and other  Miscellaneous Tools

(7)

The DAVE suite

Data Reduction

10 supported instruments

Performing data reduction with the FANS

Data Reduction module.

(8)

The DAVE suite

Data Visualization

Line, contour, image and surface plots

The Data Browser interface showing data and graphic containers. Also shown is

an image view of a 2D dataset as well as a line plot of multiple 1D datasets.

(9)

The DAVE suite

Data Analysis

Dataset manipulations (merging, scaling, combining, etc) Fit data to model functions using least-squares fit

Single crystal analysis (projections, cuts, etc)

(10)

The DAVE suite

Miscellaneous Tools

More than a dozen tools

Eg Neutron cross-section tables and Calculations for compounds

(11)

DAVE Limitations

Limitations of DAVE

 Had evolved organically (little common API, too much duplication of effort)  Was too modular with little communication between components

 Was becoming harder to maintain and even more difficult to extend

(12)

The arrival of iTools

Experimented with iTools (iPlot, iSurface, etc) in 2004

Contained many powerful features

Undo/Redo capability,

Messaging system for sending notification messages between components, highly customizable visualization graphics especially annotations,

saving application state, and many more

Was very impressed.

(13)

What are iTools

Has two parts:

1 The iTool Framework – a set of classes with well defined API.

It is a comprehensive component library consisting of dozens of classes.

2 The Intelligent Tools or iTools are a set of 7 flagship applications built on the framework. They are end-user applications

(iPlot, iImage, iContour, iSurface, iVolume, iVector, iMap).

For maximum flexibility, the framework must be used to develop own applications if they deviate too much from the example iTools.

(14)

Inadequate Documentation

Inadequate Documentation

Many undocumented classes.

Lack of complete tutorials even in iTool Developer's Guide book.

Took me 6 months to reach sufficient level of understanding.

Some improvements since the early days for example the addition of the object model

inheritance diagram.

(15)

Data-centric Interface

Data-centric UI

Example iTools present image processing interface – dominated by graphics window.

DAVE primarily manipulates experimental data

 Therefore it needs a data processing interface.

 Experimental dataset must be presented and maintained within a single entity.  These datasets have many components besides the plottable data.

Graphics and tree widgets are only viable UI elements for user interaction with data.

• Can't use graphics because dataset is more than just the plottable data. • Adopt the iTool Visualization Editor widget interface.

• But add a custom DAVE Data Manager for storing experimental datasets.

Design application Tool Class (DAVETool which subclasses from IDLitTool) that

 Maintains a separate data manager folder from the default system data manager to prevent automatic

modification of experimental data.

 Activate or Deactivate menus based on selected data types from the Data Manager

Design appropriate data hierarchies for all supported file formats.

(16)

Data-centric Interface

The main interface.

Consists of: 1 Data Browser

shows contents of the Data Manager 2 A property sheet for for viewing details of selected data items 3 Menus

4 A Property sheet for selecting visualizations and associating data with the visualization Parameters.

Note: Note that some menus are deactivated because those operations cannot be applied to the selected data item, S(Q,w)

(17)

Using built-in Visualization classes

Extend Visualization Classes

IDLitVis* classes eg IDLitVisPlot

 Accommodate metadata – eg axes titles and units (OnDataChangeUpdate)  Handle histogram or point mode data – the mode is also specified as metadata.  Automatically remove supporting data created

 during a visualization when no longer required  eg color palettes. Set IDLitData's auto_delete  property for such data.

Also extend the visualization tool class so thatit is dependent on the DAVE application tool  (DAVETool) class.

(18)

AxisType property Meaning in a visualization

0 First indep. axis data (x-axis)

1 Second indep. axis data (y-axis)

2 Dependent axis data (z-axis)

3 Error in dependent axis data

Simplify visualization creation

Automate Visualization Creation

Automatically associate experimental data with visualization parameters for all supported visualizations. Extended IDLitData class with an axisType property.

It specifies how each data element is to be used in a visualization.

(19)

Replacing a Default Component

“ASCII Text” Reader component

Cannot successfully unregister/register the same

component name using different class names

without first deleting the IDL settings file

(itools_prefs.sav).

Modified ASCII text Reader

Default ASCII text Reader

(20)

What has been accomplished with iTools

Project management – users can now save and resume data + visualizations + application

state in a DAVE project file.

Data Manager. Easily handle neutron scattering datasets.

Visualizations. Adapt built-in IDLitVis* visualizations to work well with DAVE datasets.

Support for various data file formats (file readers) – DAVE and ASCII (SPE, column, group)

Support for various data file formats (file writers) – DAVE and ASCII (SPE, column, group)

Basic analysis functionality – data operations.

(21)
(22)

An operation component

Operations retrieve data from the Data Manager

User customizes operations through UI Service components

For most Operations, the Property Sheet widget can be used to

provide the UI Service dialog – no need to waste time designing custom dialogs!

With careful thought, it is possible to check user input for consistency.

E.g. The “Nos of datasets to sum” field can not be more than the nos of datasets in the Data Manager.

(23)

What is Good

What is good about iTool Framework

Really like the Operations component API especially the undo/redo buffer - excellent for a

data-processing application

Use of Property Sheet widget and other built-in dialogs give consistent look and feel to a large

application.

The data classes are comprehensive and can be used to create sophisticated data

hierarchies.

(24)

Suggestions for Improvement

Widget toolkit is dated – needs to be replaced!

New UI Elements such as worksheets are needed. Rich text support will be wonderful.

Cut and paste should work on all platforms.

Font support needs significant enhancement.

ħ and Å symbols are a must for scientific applications.

Better control over which components are registered by default (by the system object).

Without having to examine IDLitSystem class code to determine names of components to unregister.

Update the iTool Developer's Guide with more meaningful tutorials. Also complete the class

reference documentation. For example the excellent metadata interface in IDLitData is still

undocumented.

(25)

Conclusion

The iTool Framework architecture has introduced several new

concepts that will result in a fresh approach to application

development in IDL.

However, there are key problem areas such as documentation, a

dated widget toolkit and inadequate font support.

References

Related documents

quadratic functions, and simple Grade Level Problem name Standards for Mathematical CA CCSSM CEE National Standard Jump$tart Standard Practice Grade 10 Credit Card

• The recommendation as noted previously is to have part time lab personnel who could provide additional laboratory hours for students to utilize the lab and equipment as

(P K Maulik PhD), The George Institute for Global Health, New Delhi, India (S Bhaumik MBBS, Prof V Jha MD); Istituto di Ricerche Farmacologiche Mario Negri IRCCS, Ranica, Italy

Pairwise variation analysis recommended four and two reference genes for optimal normalization of the expression data of canine OS tumors and cell lines, respectively..

study was designed to document the commonly isolated gram-negative bacilli from different clinical specimens and their susceptibility patterns in tertiary health care hospital

• Before filling the claim, you must ask your doctor to fill in the doctor’s statement on the claim form, stating the period of absenceA. To whom is this

Perry (2009) estimates that over the period 1970–2005 about 44% of the ‘excess’ output volatility of developing countries, defined as the standard deviation of GDP per capita from

• Class 1, Division 1 – LED and high-performance lighting, explosionproof fittings and enclosures, cable tray, strut and explosionproof visual signals from Cooper