A
THENA
U
ser’s
G
uide
Bruce Ravel
http://cars9.uchicago.edu/~ravel/software/exafs/
Document version 1.5
for
athena
version 0.8.56
July 1, 2009
Just as Phaeacian men excel the world at sailing, driving their swift ships on the open seas, so the women excel at all the arts of weaving. That is Athena’s gift to them beyond all others – a genius for lovely work, and a fine mind too.
This document is copyright c 2007–2008 Bruce Ravel.
This work is licensed under the Creative Commons Attribution-ShareAlike License. To view a copy of this license, visit
http://creativecommons.org/licenses/by-sa/3.0/
or send a letter to Creative Commons, 559 Nathan Abbott Way, Stanford, California 94305, USA. You are free:
• to Share – to copy, distribute, and transmit the work • to Remix – to adapt the work
Under the following conditions:
• Attribution. You must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work).
• Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license.
• Any of these conditions can be waived if you get permission from the author.
• For any reuse or distribution, you must make clear to others the license terms of this work. • Any of the above conditions can be waived if you get permission from the copyright holder. • Nothing in this license impairs or restricts the author’s moral rights.
Your fair dealing and other rights are in no way affected by the above. This is a human-readable summary of the Legal Code (the full license).
Contents
1 Forward 9
1.1 Layout and typesetting conventions. . . 9
1.2 Acknowledgements . . . 10
1.3 Data citations. . . 11
1.4 Installing Athena on your computer . . . 11
1.5 Building this document from source . . . 12
2 Introduction to Athena 15 2.1 First Look at ATHENA . . . 15
2.2 Getting help. . . 16
3 Data import 19 3.1 Column selection dialog . . . 21
3.1.1 Data types and energy units . . . 21
3.1.2 Multi-element detector data. . . 23
3.2 Project selection dialog . . . 25
3.3 Multiple data set import . . . 26
3.4 Reference channel. . . 27
3.5 Preprocessing data . . . 28
3.5.1 Rebinning quick scan data. . . 29
3.5.2 Other pre-processing chores . . . 30
3.6 File type plugins . . . 31
3.6.1 Overview of how plugins work. . . 31
3.6.2 Example plugin. . . 32
3.6.3 Namespace . . . 33
3.6.4 Required methods and variables . . . 33
3.6.5 Athena’s plugin registry . . . 34
3.6.6 Reformatting and data processing . . . 35
3.6.7 System plugins and user plugins . . . 36
3.6.8 Miscellaneous advice on plugins. . . 36
4 Normalization and the AUTOBK Algorithm 37 4.1 Normalization. . . 37
4.1.1 The normalization algorithm . . . 38
4.1.2 The flattening algorithm. . . 39
4.1.3 Getting the post-edge right . . . 41
4.1.4 Getting the pre-edge right . . . 41
4.1.5 Measuring and normalizing XANES data . . . 42
4.2 Understanding Fourier transforms . . . 43
CONTENTS
4.4 Spline clamps and k-weight in background removal . . . 49
4.4.1 Spline clamps . . . 49
4.4.2 The effect of k-weight on background removal . . . 50
4.4.3 The interaction between spline clamps and k-weight . . . 50
4.5 Spline range in background removal . . . 52
5 Plotting Your Data in ATHENA 53 5.1 Plotting space tabs . . . 54
5.1.1 Plotting in energy . . . 54
5.1.2 Plotting in k-space . . . 55
5.1.3 Plotting in R-space. . . 55
5.1.4 Plotting in q-space . . . 55
5.2 Spectral resolution and k-range . . . 56
5.3 Stacked plots . . . 56
5.4 Indicators . . . 56
5.5 The point finder . . . 58
5.6 Group specific plot parameters . . . 58
5.7 Other plotting features. . . 58
5.7.1 Zooming. . . 59
5.7.2 Cursor . . . 59
5.7.3 Plotting I0 . . . 59
5.7.4 Merged groups and the standard deviation . . . 59
5.7.5 The collapse button . . . 60
6 Athena’s User Interface 63 6.1 Using the group list . . . 64
6.1.1 Copying groups . . . 64
6.1.2 Reorganizing the group list . . . 64
6.1.3 Information about items in the group list . . . 64
6.1.4 Using the mouse in the group list. . . 67
6.1.5 Detector groups and background groups . . . 68
6.2 Marking groups . . . 68
6.2.1 Using regular expressions to mark groups . . . 69
6.3 Pluck buttons . . . 70
6.4 Plot styles . . . 71
6.5 Using different k-weights . . . 71
6.6 Frozen groups . . . 73
6.7 Palettes . . . 74
6.8 Setting user preferences . . . 76
7 Setting parameters in Athena 77 7.1 Constraining parameters between data groups . . . 77
7.1.1 Constraining individual parameters. . . 77
7.1.2 Constraining groups of parameters . . . 78
7.1.3 Constraining all parameters . . . 79
7.2 Setting E0 . . . 79
7.2.1 Other ways of setting e0 . . . 80
8 Data export 83
8.1 Column output files . . . 83
8.2 Project files . . . 87
8.2.1 The project file format and compatibility with older versions . . . 88
8.3 Parameter report files . . . 89
9 Data processing 91 9.1 Calibrating data groups . . . 92
9.2 Aligning data groups . . . 92
9.3 Dispersive XAS . . . 95
9.3.1 The dispersive data processing algorithm . . . 95
9.3.2 Using the dispersive data correction . . . 96
9.4 Deglitching data . . . 98
9.5 Truncating Data . . . 100
9.6 Rebinning data groups . . . 102
9.7 Smoothing data. . . 102
9.8 Convoluting data groups . . . 104
9.9 Deconvoluting data groups. . . 104
9.10 Self-absorption approximations . . . 106
9.10.1 Correcting XANES data . . . 107
9.10.2 Correcting EXAFS data . . . 107
9.10.3 Information depth . . . 109
9.10.4 Algorithm references . . . 110
9.11 Removing multi-electron excitations . . . 110
9.12 Merging data groups . . . 112
10 Data analysis in Athena 115 10.1 Linear combination fitting . . . 115
10.1.1 Fitting a single data group . . . 116
10.1.2 Constraints and modifications to the fit . . . 117
10.1.3 Fitting, statistics, reports . . . 118
10.1.4 Constraining linear combination fit parameters between groups . . . 119
10.1.5 Batch processing . . . 119
10.1.6 Combinatorial fitting using many standards . . . 119
10.2 Peak fitting . . . 121
10.3 Log-ratio/phase-difference analysis . . . 122
10.4 Principle component analysis . . . 124
10.5 Difference spectra. . . 125
11 Worked examples 129 11.1 Basic data processing . . . 129
11.2 A hard background removal problem . . . 136
11.2.1 Getting started . . . 136
11.2.2 Examine the theory . . . 136
11.2.3 A simple fit to the first coordination shell . . . 139
11.2.4 Using the fit as a background removal standard . . . 139
11.2.5 Understanding . . . 141
11.3 Linear combination analysis . . . 141
11.3.1 Examining the data . . . 142
11.3.2 Improving the fit . . . 143
11.3.3 Understanding the fit . . . 144
CONTENTS
11.3.5 Analyzing the data series . . . 146
12 Hephaestus 149 12.1 Description . . . 149
12.2 Periodic Table of Absorption Data . . . 150
12.3 Absorption Lengths of Compounds . . . 150
12.4 Periodic Table of Chemical Data . . . 152
12.5 Absorption of Ion Chambers. . . 152
12.6 Emission Line Transitions . . . 154
12.7 Edge and Line Finders . . . 154
12.8 Complex Scattering Factors for the Elements . . . 155
12.9 Initialization file . . . 155
12.10Keyboard shortcuts. . . 156
12.11Bugs and Missing Features . . . 156
Chapter 1
Forward
The best way to learn how to use athenais to use athena. Poke at the buttons, poke at the menus, try things just to see what happens. And above all, remember the physical and mathematical meanings of your data and of the data analysis techniques and think about how actions in athena relate to those meanings.
athena is a powerful and versatile program capable of supporting almost all of your common (and
not-so-common) XAS data processing chores. It is not, however, a particularly intuitive program. I doubt that any XAS program could be intuitive. On top of that,athenahas accumulated lots of features over the years. Many of these features are necessary for high-quality data processing, others are bells and whistles intended to make data processing more convenient or more fun.
This document attempts to be a comprehensive overview of all of athena’s features. There are lots of words, but also lots of pictures. Feel free to jump around and to focus on the parts most directly relevant to your immediate needs. I hope you find this document and the program helpful.
1.1
Layout and typesetting conventions
Here is a summary of fonts, colors, and symbols used to denote different kinds of text. Note that some of these may appear the same in certain presentation media.
• Filenames look‘C:like\this’.
• The names of parameters for data processing look “like this ”. • Emphasized text looks like this.
• Bold text looks like this. • Links to web sites looklike this. • Internal links look like this (Sec. 1.1).
• Keyboard shortcuts look like this: Ctrl - q . This example means to hit the q key while holding the Control (or Ctrl) key.
• Program names from the ifeffitfamily look like this: athena.
• References toathena’s preferences are written like this: § Bkg → fraction. To modify this preferences, open the “bkg” section of the preferences dialog (Sec.6.8on page76) and then click on “fraction”.
1.2. ACKNOWLEDGEMENTS
Points that require special attention are written inside of attention-grabbing
boxes.
This symbol indicates a section describing one of athena’s features that I consider especially powerful and central to the effective use of the program.
This symbol indicates a section with difficult information that newcomers toathenamight pass over on their first reading of this document.
The html version of this document makes use of HTML 4.1 character entities (mostly Greek symbols) and will not display correctly in very old browsers.
POD – the native documentation for Perl and the format used internally in athena – is not able to display graphics. Everywhere that a graphics file is displayed in the other document formats, a bit of text showing the name of the image file and the caption of the figure is displayed in the POD. Here is what it looks like when one of the pod documents is displayed by a pod viewer:
(Image file: athena_main.png) The parts of the ATHENA.
In most cases, this will convey sufficient information given that you will haveathenaopen in front of you. If you need to see the picture, you can seek out the image file by name or look at the corresponding part of the html or PDF versions of the document.
1.2
Acknowledgements
I have to thank Matt Newville, of course. Withoutifeffitthere wouldn’t be anathena. One afternoon over coffee, Julie Cross and Shelly Kelly lit the spark that eventually lead to this document. Some content of this document was inspired by an upcoming XAS review article by Shelly Kelly and Dean Hesterberg, which I have had the pleasure of editing (and, apparently, swiping from). I have a huge debt of gratitude to all the folks on the ifeffitmailing list. Without the incredible support and wonderful feedback that I’ve received in the last six yearsathenawould be a shadow of what it is today.
The following great software tools were used to create this document:
• The Template Toolkit, a really fun, really powerful templating system that was used to create this entire document
• The Perl programming languageandSyntax::Highlight::Perlfromthe CPAN repository
• TheEmacsandXEmacstext editors along withtt-modeand the simply wonderfulEmacs Code Browser • The GIMPimage editor with frequent use of the Xach effect shadowing plugin
• Various tools from the KDE desktop, including KColorChooser andKSnapshot
All screenshots were made of athena or the PGPLOT window on my KDE desktop, mostly using some version of theOrangio themefordeKorator. The screenshots of spreadsheets made from a report file (Sec.8.7on page90) and an LCF fit report (Sec. 11.22on page147) are displayed inOpenOffice.
The images of the Tholos temple on the front page and the Klimt painting Pallas Athena in the navigation box of the html document are fromhttp://www.artchive.com.
The image used as theathenaprogram icon is from “Heracles and Athena. Tondo of an Attic red-figure kylix, 480-470 BC, from Vulci” from the Staatliche Antikensammlungen in Munich, Germany. The image is in the public domain and was found onWikimedia Commons. The image used as thehephaestusprogram icon is from Reubens’ “Vulcan forging Jupiter’s thunderbolts” from the Museo Del Prado, Madrid Spain. This image is also in the public domain and also fromWikimedia Commons.
1.3
Data citations
• The copper foil data shown here and there are the data that Matt Newville, Yanjun Zhang, and I measured one day back in 1992 that has, inscrutably, become the copper foil data shown and referenced in a large fraction of the XAS theory literature.
• The platinum catalyst data shown in the difference spectrum section (Sec.10.5on page125) and MoO3
data shown in the worked examples section (Sec.11.2on page136) were donated by Simon Bare. • The MoO3 worked example (Sec.11.2on page136) was based on a tutorial written by Shelly Kelly.
• The gold edge data shown in many places throughout this document are taken from measurements published as M. Lengke, et al., Environ. Sci. Tech., 40:20, (2006) p. 6304-6309.
• The gold oxide data shown in the smoothing section (Sec.9.7on page102) were donated by Norbert Weiher.
• The iron foil data shown in the convolution section (Sec.9.8on page104) and elsewhere were measured by me while I was commissioning NSLS beamline X11B in 2004.
• The sulphate data shown in the self-absorption section (Sec.9.10on page106) were donated by Zhang Ghong and come with Daniel Haskel’s Fluo program. The copper data shown in the same section (Sec.9.10on page106) come with Corwin Booth’s RSXAP program.
• The data in the dispersive XAS section (Sec. 9.3 on page 95) were provided by Giuliana Aquilanti, beamline scientist at ESRF’s ID-24.
• Data on a hydrated uranyl phosphate that appear in several places are the U LIII standard used by
my former research group. Spectra from this standard have appeared in many publications from that group. The U3O8 sample shown in the the deglitching section (Sec. 9.4 on page 98) are from the
group’s standards library.
• Tin edge data which appear in several places are from C. Impellitteri, O. Evans, B. Ravel, J. Environ. Monit., 4, (2007) p. 358-365.
• Data on PbTiO3, BaTiO3, and EuTiO3are taken frommy own PhD thesis.
1.4
Installing Athena on your computer
Linux, BSD, and other unixes
It is not especially hard to build athena from source code. The procedure is explained in detail on this web page: http://cars9.uchicago.edu/iffwiki/BruceRavel/EvolvingSoftware.
Debian and debian-based Linux
Add the repositories listed on this page: http://wiki.mr.aps.anl.gov/ObtainingSoftware. This can be done either by editing ‘/etc/apt/sources.list’ directly or by adding repositories using the system package manager (e.g. Adept on ubuntu). Once these are included as repositories, the “horae” package, which contains athena, can be installed.
Windows
An MSI installer package is available at http://cars9.uchicago.edu/iffwiki/Downloads. Installation is very simple. Just double click on the installer and follow the instructions. If you are installing onto a machine for which you do not have administrator privileges, you should choose a part of the disk that belongs to you when asked about the installation location.
For those with ActiveState Perlinstalled on their windows machines, the “horae” source code can be installed using the Perl Package Manager. In the Perl Package Manager, select “Preferences” from the Edit menu, then click on the repositories tab. Add http://cars9.uchicago.edu/ ravel/ppm/ as a new repository. The “horae” package should now show up as an installable package.
1.5. BUILDING THIS DOCUMENT FROM SOURCE Macintosh
An installer package is available at http://cars9.uchicago.edu/iffwiki/Downloads. Installation is very simple. Just double click on the installer and follow the instructions.
1.5
Building this document from source
Obtaining the document source
The source files and all images files for this document can be downloaded by SVN. To grab the source, you will need anSVN clienton your computer. This command checks a copy of the source out and downloads it onto your computer:
svn co http://cars9.uchicago.edu/svn/aug/
This document is written using The Template Toolkit. It requires the perl interpreter and a fairly complete installation of version 2 of The Template Toolkit to build. If TT2 is not available as a package for your system (it is available as a pre-compiled package for many versions of Linux;a ppm file for ActivePerl on Windows exists; a Fink package for OSX exists) it can be downloaded from its website and installed by hand or downloaded using perl’s CPAN utility. You will also need to install the Image::Size, and Syn-tax::Highlight::Perl modules. Compiling the LATEX version of the document will require a fairly complete
LATEX installation as I make use of many styles, including amsmath, amsfonts, floatflt, fancybox, fancyhdr,
keystroke, varioref, and hyperref. (teTeX compiles the document without any problem. The texlive package may complain about not being able to generate two postscript font sets. Those two warning can be safely ignored by simply hitting ←- when pdflatex pauses to complain. As far as I can tell, ignoring this problem has no impact on the document. I have no experience building the PDF document on any system other than linux.)
Once TT2 and the other modules are installed, building the document should be quite simple. TT2’s ttree program is used to recurse the through the directory structure containing the templates. The
‘bin/build’, ‘bin/tex’, and ‘bin/pod’ scripts are wrappers around ttree. They invokes a number of important command line options and pass any further command line options to ttree.
TT2 was chosen for this project because it is an excellent templating tool. A templating tool was chosen because the strong separation of format and content was attractive to me. The template source is used to generate html and PDF versions of the document as well as the pod format used by athena’s internal document viewer.
Contributions to the document are extremely welcome. The very best sort of contribution would be to directly edit the source templates and commit your changes to the SVN repository. The second best sort would be a patch file against the templates in the repository. If TT2 is more than you want to deal with, but you have corrections to suggest, I’d cheerfully accept almost any other format for the contribution. (Although I have to discourage using an html editing tool like FrontPage to edit the html directly. Tools like that tend to insert lots of additional html tags into the text, making it more difficult for me to incorporate your changes into the source.)
Building the html document
After downloading and unpacking the source for this document, you must configure it to build correctly on your computer. This is simple:
./configure
To build the entire document as html
Individual pages can be built by specifying them on the command line:
./bin/build bkg/norm.tt forward.tt
Building the LaTeX document The LATEX document is built by
./bin/tex -a cd tex/
pdflatex athena.ltx pdflatex athena.ltx
You need to run pdflatex two or three times to get all of the section numbering and cross referencing correct. The varioref package, used to handle cross-referencing, is sometimes a little fragile. If you see the following error message: simply hit return. The message should disappear when you recompile the document.
! Package varioref Error: vref at page boundary 142-143 (may loop).
A version of the PDF document suitable for printing on a grayscale (i.e. without color) printer can be made using this sequence of commands:
cd images/ ./images/convert_to_gray cd ../ ./bin/texbw -a cd texbw/ pdflatex athena.ltx pdflatex athena.ltx
This first bit converts all of the images used in the document to grayscale using ImageMagick. The “texbw” version of the document is nearly identical to the normal LATEX document, except that the style
file and several templates are changed to remove all color from the document. Compiling the version in the texbw/ directory results in a PDF file that is suitable for a non-color printer.
This output target was created for the 2008 APS summer school. The cost saving of printing the
athenadocument in black-and-white rather than color for each of the students was quite significant.
Building the pod document
The pod document,athena’s internal document format, is built by
./bin/pod -a
Individual pages can be built by specifying them on the command line:
./bin/pod bkg/norm.tt forward.tt
Using the document with Athena
The pod and html document files can be used by athena, but they must be installed in the correct location. To do this, run the install script that comes with the document package. This will use an
athenamodule to determine the correct location, then copy over all relevant files. Once installedathena
will display the html document in either a web browser or a pod viewer depending on the values of the § Doc → prefer and § Doc → browser preferences.
Chapter 2
Introduction to Athena
athenais an interactive graphical utility for processing EXAFS data. It handles most of the common
data handling chores of interest at the beamline or for preparing your data to begin analysis. athenais a graphical front end to Matt Newville’s ifeffit library written entirely in the Perl programming language and using the PGPLOT graphical library for data display. It is being developed on Linux and tested on Windows XP, but should work on any unix-like, Windows, or Mac operating system.
athenais designed to provide high quality analysis with a highly usable interface. It allows very fine
grained control over the processing and plotting of individual data sets while still enabling the processing and plotting of large quantities of data.
Amongathena’s many, many features, you will find: • Convert raw data to µ(E)
• File import plugins for reading arbitrary data files • Process and plot multiple data files simultaneously • Merge data in energy, k-, R-, or back-transform k-space • Energy calibration
• Align data scans with or without a reference channel • Deglitch µ(E) data
• Self-absorption corrections for fluorescence spectra • Compute difference spectra
• Fit linear combinations of standards to XANES or EXAFS data • Fit peak functions to XANES data
• Log-ratio/phase-difference analysis
• Background removal using the AUTOBK algorithm
• Normalization of XANES data to the Cromer-Liberman calculations • Forward and backward Fourier transforms
• Save data as µ(E), normalized µ(E), χ(k), χ(R), or back-transformed χ(k) • Save project files, allowing you to return to your analysis later
• ... and much, much more!
2.1
First Look at ATHENA
Whenathenafirst starts, something like Figure2.1on the next page appears on your computer screen. Theathenawindow is divided into several parts. We will discuss each of these parts and give each a name.
2.2. GETTING HELP
Figure 2.1: The parts of theathena.
At the top of the window is a menu bar. This works much like the menu bar in any program. Much of the functionality inathenais accessed through those menus.
The largest part is the main window, highlighted in the picture above with a red border. The red border does not happen in the real program – it’s there simply to clarify this discussion. The main window is divided into six parts. The top box identifies the file name of the current project file (Sec.8.2on page87). Below that, are various parameters identifying the current data group.
The next three boxes are used to define the parameters associated with normalization and background removal, forward Fourier transforms, and reverse Fourier transforms. At the bottom of the main window are a couple of parameters associated with plotting.
At the bottom of the screen is the echo area. This very important space is used byathenato commu-nicate with you, the user. This space is used to display informational messages whileathenais working on your data, error messages when it runs into trouble, and other kinds of messages. Whenathenastarts, it displays a hint from it’s built-in list of hints. A new hint can be seen at any time by pressing Ctrl - h .
The large blank area on the right is the group list area (Sec. 6.1on page 64). As data are imported into athena, they will be listed in this space. Access to the data already imported is made by clicking in this space.
Above the group list area are the mark buttons (Sec.6.2on page68) The blank space next to the mark buttons is the modified project indicator. This indicator shows when your project has been modified and needs to be saved.
Below the group list area are the plot buttons. Below that are the buttons used to set the k-weighting for use when plotting in k-space or when making a forward Fourier transform. Below that are various other plotting controls (Sec.6.2on page68) in the plotting options section.
After importing several data files, each is made into a data group and listed in the group list. The label and the check button next to it are the main controls for interacting with data inathena.
Figure 2.2: athena, after importing some data.
There is quite a bit of help built right into athena. Typing Ctrl - m or selecting “Document” from the Help menu will displayathena’s document in a web browser or in the built-in document viewer. The “Document sections” submenu allows you to jump directly to a particular topic. Also, many parts of the program have a button which will take you directly to the part of the document that describes that part of the program.
You can select the format in which the documentation is displayed by setting the § Doc → prefer
preference. Because it has all the images, the html document is probably the better choice. Users on Windows and the Mac will have the html document displayed in whatever browser is the system default. For Linux, BSD, and other unix users, athenaattempts to find a suitable browser. You can explicitly set the browser from a list of common (and not so common) options by setting the § Doc → browserpreference. This list includes every GUI browser option that Bruce was aware at the time this paragraph was written, including several options from the Mozilla ecosystem.
Another feature that you can use to explore different parts of the program are the demo files. Demo files are normal project files (Sec.8.2on page 87) that get installed when you install this software on your computer. Each demo file demonstrates a particular feature of the program. When you select “Import a demo project” from the Help menu, a file selection dialog opens to the location of the demo projects. When you select one, its project journal (Sec. 6.7 on page 74) is opened after its data is displayed. The journal explains how to use the data to learn about the topic covered by that demo.
Chapter 3
Data import
athena is very versatile in how she reads in data files. Pretty much any data in the form of columns
of numbers can be successfully read. With a few exceptions, athena relies upon ifeffit’s read data() command to handle the details of data import. ifeffit is clever about recognizing which part of a file is columns of numbers and which part is not. In the following, I’ll explain how the read data() command interprets files, explain the limits on its andathena’s abilities to interpret a data file, and discuss the kinds of manipulations of data that can and cannot be performed byathenaas data are imported.
athena expects data of one of a few types. Column data in which the columns represent such things as the energy grid and the scalars measured during the experiment are the most common sort of data that most people use import into athena. athena’s column selection dialog is used to convert the raw scalars into µ(E) data. Other common kinds of data files that might be read into athena are files that contain µ(E) or chi(k) data in columns or the output files from Feff,‘xmu.dat’and‘chi.dat’.
Here is an example of a data file that will make athenaas happy as can be. There are some header lines, followed by a line of dashes, followed by a line of column labels, followed by lines containing columns of data.
# X15B project: MT 9/23/04 # original file: STD1.001
# unpacked from original data as a sequence of 4-byte floats #
---# energy I0 narrow wide
2400.0020 60183.3008 38.5000 83.0000
2401.5088 60241.0508 41.5000 82.0000
2403.0078 60347.5508 40.0000 83.7500
2404.5039 60531.0508 42.2500 78.2500
... etc ...
In this example of a perfectly formatted file, the header lines, the line of dashes, and the column labels line are all preceded by a hash (#) mark. ifeffit is thus able to recognize these as header lines. Since ifeffitrecognizes them as such,athenawill store them in the project along with the data. Because there
is a line of dashes and because it is followed immediately by column labels,athenais able to use these labels in the column selection dialog. A few other common US keyboard symbols, such as also be understood as marking header lines.
The numbers in the columns can be integers, floats (such as 1.234, -1.234, .1234 or -.1234), or exponentials (such as 1.23e45 or -1.23E-45). Anything interpretable as a number in the C programming language will be interpretable in this context. The columns of numbers go to the end of the file. There is no text following the data.
When data is recorded as described above, it will be fully utilized by athena. The headers will be recorded, the column labels will be used, the data will be interpreted. athenacan, however, accommodate significant deviation from the format described above.
• If the header lines are not marked by a # or some other recognizable marking character, ifeffitwill not be able to recognize headers or column labels. As long as no text follows the data, the columns will still be understood as columns of data and the data can be imported by Athena.
• If the line of dashes is missing, again the headers and column labels will not be recognized, but the columns of data will be.
• If no headers are in the file, the columns of data will still be understood as data.
athenaexpects that the data are recorded as a function of energy and that one of the columns contains
energy values. The assumption is that the first column is the energy column, but that can be changed in the column selection dialog. athena works in eV. If data are recorded in keV, there is a menu in the column selection dialog that must be set accordingly.
Here are some operations that can be performed as data is imported.
1. Data can be imported as a function of pixel position on an area or linear detector. 2. Data from a multi-element detector can be summed on the fly.
3. Data from a multi-element detector can be imported such that each detector channel is imported into its own data group.
4. Data can be negated, i.e. multiplied by -1.
5. A reference channel can be read from the the same file.
6. Data can be preprocessed. That is, data can be truncated, deglitched, aligned to a standard, and have its parameters constrained to a standard
Here are some operations that can be handled using the Plugin architecture (Sec.3.6on page31). 1. Conversion from wavelength to energy.
2. Conversion from encoder reading or motor steps to energy. 3. Conversion of data in a binary format
4. Dead-time corrections using columns from the data file.
5. Any math expression more complicated than sums of columns in the numerator and denominator, e.g. plugins allow you to multiply the If column by 7 and divide by the sine of the I0 column, if that’s what you want.
If some of the criteria for the data file format are not met, for example if there is text following the data columns or if you need to perform one of the operations not yet supported, you will need to process you data before trying to import intoathena.
There are examples of data files thatathenawill process before sending off toifeffitfor import. An example is the data file format from beamline X10C at NSLS. Files from that beamline cannot be imported as written byifeffit’s read data() command. athenawill recognize such a file and process it as needed before importing it. This can be done with other beamlines. You should contact Bruce if you are the beamline scientist or a frequent user of some beamline which writes data in a way that read data() cannot import.
As a final comment, I would encourage beamline scientists and the authors of data acquisition software to consider their users when designing data file formats. While I certainly will not say that beamlines should be required to accommodateathenaor even that beamline staff have any obligation to recommendathena
to their users, the truth is thatathenais becoming an increasingly common tool in the EXAFS community. The format that best serves athenais actually a fine format that can be imported by a very wide variety of EXAFS software, plotting software, spreadsheets, and other programs. It’s a good format and your users would be well served by your adopting it.
3.1
Column selection dialog
To import a data file, select Open file from the File menu or type Ctrl - o . A file selection dialog opens. On my Linux computer, it looks like Figure3.1
Figure 3.1: The file selection dialog on a Linux computer.
It looks somewhat different on Windows, but behaves the same. It allows you to navigate your disk to find the file you want to import. Once you find that file, click on it then click on the Open button.
Once you have selected a file to import the column selection dialog, shown in Figure3.2on the following page, appears.
On the right side of this dialog, the contents of the data file are displayed. This allows you to examine the file to help you figure out which columns should be imported to turn into the µ(E) data.
On the left are various control for specifying which columns contain the energy values and which contain the signals from the various detectors. Typically, the signals from the detectors are saved to disk as columns of numbers. These columns need to be combined depending on the nature of the experiment. For a transmission experiment, the incident channel is divided by transmission channel and the natural log is taken at each point. For fluorescence data, the fluorescence channel is divided by the incidence channel. Electron yield data is like fluorescence data – the yield channel is divided by the incident channel.
The controls in the tabs at the bottom left are the discussed in later sections.
In the example shown, the incident channel is, for some reason, called “mcs3”. Since this is transmission data, I have checked the “mcs3” button for the numerator. The transmission channel is called “mcs4” and its button is checked for the denominator.
As you check the buttons, a couple of helpful things happen. The first is that equation for how the columns combine to form µ(E) is displayed in the box below the column selection buttons. Also as you check buttons, the data are plotted. If you have selected the correct columns and chosen the numerator and denominator correctly, the plot will look like XAS data. If the plot is upside-down, then you need to switch the numerator and denominator. If the plot doesn’t look like XAS at all, you need to try some of the other channels.
I chose this example because the columns are labeled somewhat confusingly. Often the columns will be labeled in the file more obviously with names like “I0” or “It”. In this case, we either need to know what the columns mean or patiently click through the buttons to figure it out. As a last resort, you may need to ask the beamline scientist!
3.1.1 Data types and energy units
Figure 3.2: The column selection dialog.
Occasionally,athenaneeds a bit more information to correctly interpret your data correctly. The data types menu is shown in Figure3.3. The default is for data to be imported as µ(E).
Figure 3.3: Data types in the column selection dialog.
The other choices are:
• norm(E) : µ(E) data that have already been normalized in some other way. These data will not be normalized by athena
• xanes(E) : µ(E) data measured over a limited data range and for which you do not need to look at the χ(k)
• chi(k) : χ(k) data, that is data that have already been background subtracted from µ(E) • detector : data that are not µ(E) but which you may want to plot in energy
• chi.dat : the‘chi.dat’file from feff
If you make a mistake and import your data as the wrong data type, you can change between any of the energy-valued (µ(E), normalized µ(E), XANES, or detector) record types at any time by selecting “Change record type” from the Group menu and selecting the correct choice from the popup dialog, Figure3.4. This dialog cannot, however, be used to change χ(k) data to an energy-value type or vice-versa, nor to change one of thefefftypes to a non- fefftype.
Figure 3.4: Changing the data type after the data are imported.
athenauses electron volts as its energy unit. It uses a simple heuristic to figure out if an input file is
in eV or keV. In caseathenagets it wrong, you can specify the energy unit with the “Energy units” menu. Dispersive XAS (Sec.9.3on page95), i.e. data which is a function of pixel index, requires special treatment.
3.1.2
Multi-element detector data
athena’s column selection dialog has some special features for dealing with multi-element detectors.
You can select all the channels of the MED as elements of the numerator, as shown in Figure3.5.
Figure 3.5: Importing multi-element data in the column selection dialog.
Importing the data will then add up the channels on the fly and put a group containing the summation of the channels in the group list.
You have the option of clicking the button that says “Save each channel as a group”, as in Figure3.7 on the next page.
3.1.2 Multi-element detector data
Figure 3.6: After adding columns of multi-element data on the fly inathena.
Then, instead of adding the channels to make one group, each channel will be imported as an individual group and given its own entry in the group list. This is handy for examining the channels and discarding any that are not of usable quality.
Figure 3.8: After importing the channels of multi-element data as individual groups.
3.2
Project selection dialog
Project files (Sec.8.2on page87) areathena’s mechanism for saving the state of an analysis project. The project selection dialog, shown in Figure3.9, provides a way of selectively importing part or all of the contents of the project file.
3.3. MULTIPLE DATA SET IMPORT
Project files, like any other files are imported using the file selection dialog (Sec. 3.1 on page 21).
athenawill recognize a project file and present the project selection dialog.
The contents of the project file are listed on the left side of the dialog. The contents of the journal (Sec.6.7 on page 74) are listed in the box at the top right. When you click on one of the data groups in the list on the left, it gets plotted and its title lines are inserted into the box at the lower right. Using these parts of the dialog, you can examine the contents of your project file before importing them intoathena.
By selecting some of the projects from the list, you can import a subset of the project file. The group listing uses extended selection:
• Click on an item in the list to select one group
• Control-click (i.e. hold down the control key while clicking) on an item in the list to add it to the selection
• Shift-click to on a group to select all groups between it and the previously selected group. • Click and drag to select all groups that you drag over.
Additionally the buttons labeled “All”, and “None” can be used to select all groups or to clear the selection. The “Invert” button will invert the selection of each group.
Once you have selected the groups you want to import, click the “Import” button. If no groups are selected then all the groups will be imported.
athenachecks for project files that have become damaged. When it recognizes a damaged file, it will
extract what information is salvageable and insert those groups in the group list. It will then issue an error message with a brief hint about the problem. That error message is inserted into the titles box in scary red text.
3.3
Multiple data set import
You can import multiple data sets in the same manner that was explained in the last section. Select Open file from the File menu or type Ctrl - o . When the file selection dialog opens, you can select more than one data file by clicking on file names while holding down the Ctrl key. On my Linux computer, it looks like Figure3.10.
Figure 3.10: Importing multiple data sets with the file selection dialog.
Note that three files are highlighted in the file listing and that those three files are listed below in the “File name” box. Another way of selecting multiple files is to click on a file in the listing then click on
another file while holding down the Shift ⇑ key. When you do this, all files between the two you clicked on will be selected.
When you click the OK button, all of the selected files will be imported. If all of the files are of the same type, athena will import them all with only one interaction of the column selection dialog. Thus, if you select several files that were measured one after the other, they will all be imported using the same column selections as well as the same parameters for the reference channel, rebinning, and preprocessing (all of which will be described in the following sections). If, however, a file is found that appears to be of a different format, the column selection dialog will reappear as needed. athenaconsiders two files to be the same if they have the same number of columns and those columns have the same labels.
Each file imported in this way will be listed in the group list, shown in Figure3.11
Figure 3.11: athenaafter importing multiple data sets.
When you import multiple project files, the project selection dialog (Sec. 3.2on page 25) will appear for the first one in the list. If you import the entire contents of the project file, then the entire contents of all remaining project files will also be imported. If, however, you import only a subset, the project selection dialog will appear for the next project file. As soon as you import an entire project, all subsequent projects will be imported without having to interact with the dialog.
3.4
Reference channel
The column selection dialog offers several other features related to data import. In this section we will see how to import a reference channel. It is common to place a third ionization chamber in line after the transmission chamber and to place good transmission standard between the two. The point of measuring the standard is that it is measured in parallel with your real sample. This standard can then be used to align the actual data using the data alignment dialog (Sec.9.2on page92).
The standard often is a zero-valent foil, but the most important thing is that it is an excellent standard which will yield consistently high-quality data. The other most important thing is that you always use
3.5. PREPROCESSING DATA
the same sample as your reference standard for ever experiment you make at a particular edge. To this end, it is wise to make a library of standards and carry them with you to the synchrotron. By using the same standard for every experiment at an edge, you can align any data, even data measured years apart at different synchrotrons.
Figure 3.12: Importing a reference channel with the column selection dialog.
The reference channel selection works almost the same as column selection for the data except that you don’t need to specify the energy column again – the same column is used. When a reference channel after the transmission channel is used, you should use the transmission channel as the numerator and the reference channel as the denominator. Another common solution to measuring a reference channel is measure elastically scattered radiation through the standard with a PIN diode. In that case, I0 is the numerator and the diode is the denominator.
You can plot the reference channel to make sure that you have selected the correct channels with the “Plot reference” button. If your reference standard is of an element with a nearby edge energy, uncheck the button labeled “Same element”. If you use some other kind of reference measurement that is not a transmission measurement, you can uncheck the “Natural log” button.
When you click the OK button, the data are imported and inserted into the group list. The reference channel is placed in the group list below its data. The reference channel is just like any other group, with one distinction. The data and its reference channel are tied together in the sense that the values for their “eshift ”parameters will always be the same.
This relationship is shown visually by the change in color of the text in the box for “eshift ”, as seen in in Figure3.13on the facing page.
When you change the value of“eshift ”for the reference, the value of“eshift ”for the data changes as well. (And vice versa!) This feature of reference channels is put to good use in data alignment (Sec.9.2 on page92).
Occasionally, it is useful to tie two data groups together in this way. This can be done by marking the two groups you want to tie together as data and reference, then selecting “Tie reference channel” from the Values menu.
Figure 3.13: Data imported with a reference channel inathena.
There are a number of other operations that athena can perform on your data as it is imported. These are found in the other tabs on the bottom left of the column selection dialog. Note that all of the pre-processing chores discussed here can also be performed on data after it has been imported. See the data processing chapter (Sec.9 on page91). for more details.
3.5.1
Rebinning quick scan data
Some beamlines offer the option of collecting data as a quick scan. In that scanning mode, the monochro-mator is slewed continuous from the beginning of the scan to the end. The detectors are read continuously and integrations of the detector signals are stored in intervals. The length of these intervals and the slewing speed of the monochromator determine the energy width of each measurement bin. Typically these mea-surement parameters are chosen to provide adequate resolution through the edge – typically a third or a half of an electron volt. This results in data that are vastly over-sampled in the EXAFS region. To improve the statistics in the EXAFS region and to make the data arrays smaller, it is useful to rebin the data. This process uses a boxcar averaging to place the evenly spaced quick scan data onto a typical EXAFS grid. That grid is usually something like 10 eV in the pre-edge, 0.5 eV through the edge, and 0.05 ˚A−1 in the EXAFS region.
In Figure3.14on the next page, I have imported some data on a uranyl compound measured in quick scan mode:
At this stage, athena has not examined the data closely enough to have guessed what edge you are measuring, so you must specify the element symbol of the absorbing atom. The remaining numeric parame-ters define the grid onto which these data will be rebinned. The first two numbers define the boundaries of the edge region in energy, the third defines the size of the grid in the pre-edge, the fourth defines the size of the grid through the edge, and the last defines the grid in wavenumber in the EXAFS region.
athena will remember the values of these parameters between data sets. However, the default is to
3.5.2 Other pre-processing chores
Figure 3.14: Rebinning data on the fly as it is imported using the column selection dialog.
you import, and click on the “Perform rebinning” button. When you import multiple data sets, though, rebinning will be performed on each one without intervention according to the normal rules of multiple data set import.
3.5.2
Other pre-processing chores
This tab provides controls for a number of other things that can be done with your data as it is imported. The first three – marking, truncating, and deglitching – can be performed even on the first data set imported. The other two require that a standard be specified. The menu at the top of the tab contains every item form the group list. The one specified in that menu is the standard.
Here are descriptions of each of the pre-processing chores, seen in in Figure3.15on the facing page: Mark each data set
If this is selected, each data set will be marked (Sec. 6.2on page68) as it is imported. Note that the reference spectrum is not marked. Also note that, unlike the other four pre-processing options, this one is always deselected when new data is imported.
Truncate each data set
If this is selected, the data before or after the specified value will be removed from the data (Sec.9.5 on page100). Note that only the data in athena’s memory is truncated, the data file is not altered. Deglitch each data set
If this is selected, the data are deglitched using the batch deglitching algorithm (Sec.9.4on page98). Note that only the data inathena’s memory is truncated, the data file is not altered.
Align to the standard
If this is selected, the data are aligned to the specified standard using the auto-alignment algorithm (Sec. 9.2 on page 92). If both the data and standard have reference channels, those are used in the auto-alignment.
Set parameters to the standard
If this is selected, all parameters (except for“eshift ”) will be set to the values of the standard (Sec.7.1 on page77).
The pre-processing tab is one of athena’s genuine power features. With a bit of forethought, most of your data processing can be performed automatically. I typically import one data file and carefully calibrate it and set its various parameters. Having done that, the remaining data gets well processed simply by reading it in. This kind of time saver is of particular value at the beam line.
3.6
File type plugins
athenausesifeffit’sread data()function to import data. This means thatathena’s notion of what is an acceptable data format is completely identical toifeffit’s notion. The contrapositive is also true – if
ifeffitcan read a data file, so canathena.
In practice, this works great. ifeffit is able to read the data files generated by many of the world’s XAS beamlines. And so, consequently, isathena. Sadly, there are many beamlines that use a format that confounds ifeffit and athena. There are three reasonable ways that I could deal with deal data from those beamline:
1. Refuse to deal with them and require the user to transform the data into a form that ifeffit can handle.
2. Hard-wire code intoathenato deal with each new data format as I become aware of it.
3. Create a plugin architecture that allows athenato be extended to deal well with new data formats without having to change the underlying code.
For a long time, athenarelied on a combination of 1 and 2 from that list. Eventually I decided to adopt number 3. This page documents the plugin architecture so that athena’s users can begin writing their own file type plugins.
3.6.1
Overview of how plugins work
In simple language, a perl module is a short file containing special perl code placed in a special location.
athena uses the code contained in that file to recognize and pre-process data files so that they can be
3.6.2 Example plugin
In somewhat more technical language plugin is just a perl moduleplaced on your computer in a place where it can be found. This file is used when athena starts and its methods are available when data is imported.
When a plugin is available for use, it is invoked every time a file is imported into athena using the Open file function. The new file is checked using one of the plugin’s methods to ascertain if the file is of the sort serviced by the plugin. If the file is recognized, another method in the plugin transforms the original data file into a form that is readable byifeffit. This transformation is done in a way that leaves the original data file unchanged.
If the transformation is successful, the user is presented withathena’s column selection dialog and can import data in the normal manner. Ideally, a plugin is written in a way that makes the import of the data intoathenaa completely transparent process for the user.
3.6.2
Example plugin
Here is a complete example of a functional plugin taken from the horae distribution. This plugin allowsathena to import files from NSLS beamline X10C. As you can see, the plugin is quite short. The following sections of this page will explain this example in detail.
1
2 packageIfeffit::Plugins::Filetype::Athena::X10C; 3
4 usevars qw(@ISA @EXPORT @EXPORT_OK); 5 useExporter;
6 useFile::Basename; 7 useFile::Copy;
8 @ISA = qw(Exporter AutoLoader); 9 @EXPORT_OK = qw();
10
11 ## define the required variables 12 usevars qw($is_binary $description); 13 $is_binary= 0;
14 $description= "Read files from NSLS beamline X10C."; 15
16 ## this method recognizes a file from NSLS X10C 17 subis {
18 shift;
19 my $data= shift;
20 open D,$data or die "could not open $dataas data (X10C)\n"; 21 my $first = <D>;
22 close D, return 0 unless (uc($first) =~ /^EXAFS/); 23 my $lines = 0;
24 while(<D>) {
25 close D, return 1 if (uc($first) =~ /^\s+DATA START/); 26 ++$lines; 27 }; 28 close D; 29 return 0; 30 }; 31
32 ## this method transforms a file from NSLS X10C 33 subfix{
34 shift;
35 my ($data,$stash_dir,$top,$r_hash) = @_; 36 my ($nme, $pth,$suffix) =fileparse($data); 37 my $new= File::Spec->catfile($stash_dir,$nme);
38 ($new= File::Spec->catfile($stash_dir, "toss"))if (length($new) > 127); 39 open D,$data or die "could not open $dataas data (fix in X10C)\n"; 40 open N, ">".$newor die "could not write to $new(fix in X10C)\n"; 41 my $header= 1;
42 my $null=chr(0).’+’; 43 while(<D>) {
44 $_ =~ s/$null//g; # clean up nulls
45 print N "#" .$_ if $header; # comment headers 46 ($header= 0),next if (uc($_) =~ /^\s+DATA START/); 47 next if ($header);
48 $_ =~ s/([eE][-+]\d{1,2})-/$1 -/g;# clean up 5th column 49 print N $_;
50 }; 51 close N; 52 close D; 53 return $new; 54 }; 55 56 1; 57 __END__
3.6.3
Namespace
The module must be in a particular namespace. The namespace is defined by the package function on line 2 of the example. The convention for the namespace used by the plugin is slightly unwieldy, but was chosen for a good reason. The package must be in the Ifeffit::Plugins::Filetype::Athena namespace and should have a name that is descriptive of what format it is made for. in the case of the example, the plugin is intended to transform X10C files, so the full namespace of the module is Ifeffit::Plugins::Filetype::Athena::X10C. Lines 4-9 include requisite boilerplate which will allow this module to work properly withathenaand call some modules that are almost always useful.
The reason I chose such an unwieldy namespace is to allow for the possibility of moving much func-tionality in bothathenaand artemisinto the form of plugins. With this choice, I will have considerable flexibility without having to rewrite any existing code.
3.6.4
Required methods and variables
The plugin must supply two “public” variables and two “public” methods. (I put public in quotes because, of course, perl modules constructed like our plugins do not provide true encapsulation of variables and methods. The sense in which I mean public is that athenarequires certain variables and methods in the namespace of the plugin. Without them, the plugin will fail noisily.)
required variables
Lines 12-14 define the two required variables in a way that allows them to be accessed outside the scope of this module.
$is binary
A boolean that tellsathenawhether the input file format is text or binary. athenahandles binary files slightly differently in the column selection dialog.
$description
A short text string describing the purpose of this plugin. This string will be displayed in the plugin registry – take a look at the amount of space available there, and make your string shorter than that.
the is method
Lines 17-30 show the is method. This method is called by athenato try to recognize an input data
file as being of a particular format. In the case of this example, the X10C file is recognized by some of the text in the first few lines of the files. When the file is recognized, this method returns a true value. If the test fails, it returns 0. Whenathenasees the true return value, it applies the fix method to transform the data file.
It is quite important that the is method be fast. It is possible that a data file will have to be tested against a large number of plugins. If the is method is slow, file import will be slow.
3.6.5 Athena’s plugin registry the fix method
Lines 33-54 show the fix method. This method is called when the is method returns true. In some manner it makes a copy of the original data file and transforms that copy into a form that can be read by
ifeffit. This method needs to follow a number of strict rules, however within those rules there is a lot of
flexibility about how the transformation is accomplished and the scope of what that transformation does to the data.
This method takes four scalars as inputs. $data
This is a string containing the fully resolved name of the input data file. $stash dir
This is a string with the location whereathenawill look for the transformed copy of the data. $top
This is a reference toathena’s main window. This allows you to create a GUI input dialog to collect information interactively from the user. See the Encoder plugin for an example of how this is used. $r hash
This is a reference to a hash that can contain information about how the data are transformed. This hash persists between invocations of the fix method, thus allowing you to reuse parameters about the transformation. Again, see the Encoder plugin for an example of how this is used.
The return value of this method is the fully resolved file name of the transformed file which must be located in the directory indicated in the $stash dir input scalar.
The basic work flow of the fix method is to open the original data file, perform some kind of operation on the data, and write the transformed data to the $stash dir. This can be done almost any way. Some plugins use ifeffitcommands (via the Ifeffit module) to operate on the data. Other plugins use pure perl to parse and transform the file.
In the example given on this page, the first thing the fix method does is to create a file name in the stash directory for the transformed file. Lines 36 and 37 tell athenato give the stash file the same name as the original file (that is the function of the fileparse command) but in the stash directory (the catfile method builds a fully resolved filename in a platform transparent manner). Line 38 checks the length of the fully resolved filename to avoid running into one ofifeffit’s internal limitations.
Three things are done to transform an X10C file. The header is stripped of null characters, the header is commented out by putting # characters in the first column, and a formatting problem in some files involving a lack of white space between columns is resolved. Each line of the original file is read, operated on, and written to the transformed file in the stash directory. The while loop starting at line 43 reads through the file line-by-line and performs the operations.
Lines 51 and 52 close the original and new file handles. The filter should always close the file handles. This is not such a huge issue under unix, but Windows places a lock on any open file handle. If you fail to close one, for as long asathenais running no other process will be able to do anything with that file.
At line 53, the method returns with the fully resolved name of the transformed file. At no point was the original file altered. Whenathenaexits, it will clean up the stash directory, thus avoiding a pile up of unnecessary data files.
The work flow in this example is a simple stream from one file to another. Other filters (‘Lambda.pm’
is an example) in thehoraedistribution use ifeffitto perform the transformation. ‘X15B.pm’uses perl’s pack functionto transform a binary file. ‘Encoder.pm’generates a simple GUI dialog to get data necessary for the transformation from the user. As you can see, the architecture of these plugins is quite flexible, allowing you to solve the transformation problem in whatever manner best suits the situation.
Because there might be a large number of file type plugins, it is possible for the user to turn the checks for the file types on and off. In the Settings menu, you will find the Plugin Registry. This is a simple list of all plugins found in the system and user directories. The check buttons enable and disable the plugins. The value of the variable is displayed in the list (so be sure to choose a suitable and suitably short value for that variable).
Figure 3.16: athena’s plugin registry.
Note that the order in which the plugins are displayed in Figure 3.16is the same order in which files are checked against the plugins. User plugins are checked before system plugins. After that the plugins are ordered alphabetically. If you want your system plugins to be checked against the data first, choose a name that comes early in the alphabetical sense.
3.6.6
Reformatting and data processing
When I originally conceived the concept of the file type plugins, the scope was for the problem of importing data in a format not recognized byifeffit. The plugins can also be used for some data processing. For instance, you might do deadtime corrections using the ICR and OCR columns of an MED file using a plugin. In that case, the fix method would perform the deadtime correction as the data is streamed between the original file and the stash directory.
Another example of pre-processing is the filter I wrote forMED files from APS Sector 10. We try to run in a mode where dead time is negligible. However, the large number of columns tends to use way to much ofifeffit’s memory, thus slowingathenaway down. A good solution to that problem was to use a plugin to strip all the unused columns, leaving only the ROI columns in file in the stash directory. Note that the Sector 10 plugin is an example of using a plugin to alter a file thatathenacould already read, but in a way that adds value to the user’s interaction withathena.
3.6.7 System plugins and user plugins
3.6.7
System plugins and user plugins
athena looks in two different places for these plugins. One place is inathena’s installation location
where it finds the plugins that come with the horae distribution. The other is in the user’s space (on Windows plugins are located in ‘C:\Program File\Ifeffit\horae\Ifeffit\Plugins\Filetype\Athena\’, on unix
‘$HOME/.horae/Ifeffit/Plugins/Filetype/Athena/’). In both places, it reads the contents of the plugin directory and attempts to import the files which end in‘.pm’.
3.6.8
Miscellaneous advice on plugins
1. Cut-n-paste is an excellent way to get started on a new plugin. Make a copy of a plugin for a file that is similar to your own file and use that as the basis for your new plugin.
2. ‘X15B.pm’ is an example of a plugin for a binary format.
3. ‘Encoder.pm’is an example of a plugin that uses both GUI elements and the persistent hash.
4. ‘Lambda.pm’ is an example of using ifeffit to perform the transformation. (Simply use Ifeffit qw(ifeffit); near line 5, then make use of the ifeffit function in the fix method.)
5. You can use any module that you need, thus you have all of CPAN available to you when designing your plugin. If you need to do any seriously heavy lifting, check out the Math::Pari module or the Perl Data Language
6. Although a well-tested, robust plugin should be your goal, one of the nice features of the plugin architecture is that a “good-enough” plugin is easy to write and can quickly get you over a hurdle.
Chapter 4
Normalization and the AUTOBK
Algorithm
The primary function of athenais to import and process XAS data. In the broadest sense, this can be broken up into three parts:
• Import raw data and convert it to µ(E).
• Normalize the data so that the measurement is independent of the details of the sample or the detector setup.
• Determine the background function and subtract it from the data to make χ(k).
Of course, there are many other details, such as calibration, alignment, and deglitching. Those will be discussed in detail in later sections of the document. In this section, we will cover the details of the normalization algorithm and the AUTOBK background removal algorithm. Special attention will be payed to the most important background removal parameters.
For many measured µ(E) spectra, athenawill do a good job of normalizing data and removing the background using its default parameters. In other situations – noisy data, data with large white lines, data which terminate in the appearance of another edge – user intervention is required. for those situations it is important that you understand well how the various parameters in the background removal section of the main window affect the data.
4.1
Normalization
Normalization is the process of regularizing your data with respect to variations in sample preparation, sample thickness, absorber concentration, detector and amplifier settings, and any other aspects of the measurement. Normalized data can be directly compared, regardless of the details of the experiment. Normalization of your data is essential for comparison to theory. The scale of the µ(E) and χ(k) spectra computed byfeffis chosen for comparison to normalized data.
The relationship between µ(E) and χ(k) is:
µ(E) = µ0(E) ∗ (1 + χ(E)) (4.1)
which means that
4.1.1 The normalization algorithm
The approximation of µ0(E) in an experimental spectrum is a topic that will be discussed shortly
(Sec.4.12on page47).
This equation is not, in fact, the equation that is commonly used to extract χ(k) from the measured spectrum. The reason that equation is problematic is the factor of µ0(E) in the denominator. In practice,
one cannot trust the µ0(E) to be sufficiently well behaved that it can be used as a multiplicative factor. An
example is shown in Figure4.1.
Figure 4.1: Au foil data which crosses the zero axis.
In the case of the gold spectrum, the detector setting were such that the spectrum crosses the zero-axis. Dividing these spectra by µ0(E) would be a disaster as the division would invert the phase of the extracted
χ(k) data at the point of the zero-crossing.
To address this problem, we typically avoid functional normalization and instead perform an edge step normalization. The formula is
χ(E) = (µ(E) − µ0(E))/µ0(E0) (4.3)
The difference is the term in the denominator. µ0(E0) is the value of the background function evaluated
at the edge energy. This addresses the problem of a poorly behaved µ0(E) function, but introduces another
issue. Because the true µ0(E) function should have some energy dependence, normalizing by µ0(E0)
intro-duces an attenuation into χ(k) that is roughly linear in energy. An attenuation that is linear in energy is quadratic in wavenumber. Consequently, the edge step normalization introduces an artificial σ2term to the
χ(k) data that adds to whatever thermal and static σ2 may exist in the data.
This artificial σ2 term is typically quite small and represents a much less severe problem than a
misbe-having functional normalization.
4.1.1
The normalization algorithm
The normalization of a spectrum is controlled by the value of the “e0 ”, “pre-edge range ”, and “normalization range ”parameters. These parameters are highlighted in Figure4.2on the next page.
The“pre-edge range ”and“normalization range ”parameters define two regions of the data – one before the edge and one after the edge. A line is regressed to the data in the “pre-edge range ” and a polynomial is regressed to the data in the “normalization range ”. By default, a three-term (quadratic) polynomial is used as the post-edge line, but its order can be controlled using the“normalization order ” parameter. Note that all of the data in the “pre-edge range ” and in the “normalization range ” are
Figure 4.2: Selecting the normalization parameters inathena.
used in the regressions, thus the regressions are relatively insensitive to the exact value of boundaries of those data ranges.
The criteria for good pre- and post-edge lines are a bit subjective. It is very easy to see that the parameters are well chosen for these copper foil data. Both lines on the left side of Figure 4.3 on the following page obviously pass through the middle of the data in their respective ranges.
Data can be plotted with the pre-edge and normalization lines using controls in the energy plot tabs (Sec.5.1.1on page54). It is a very good idea to visually inspect the pre-edge and normalization lines for at least some of your data to verify that your choice of normalization parameters is reasonable.
When plotting the pre- and post-edge lines, the positions of the“pre-edge range ”, and“normalization range ” parameters are shown by the little orange markers. (The upper bound of the “normalization range ”is off screen in the plot above of the copper foil.)
The normalization constant, µ0(E0) is evaluated by extrapolating the pre- and post-edge lines to“e0 ”
and subtracting the e0-crossing of the pre-edge line from the e0-crossing of the post-edge line. This difference is the value of the“edge step ”parameter.
The pre-edge line is extrapolated to all energies in the measurement range of the data and subtracted from µ(E). This has the effect of putting the pre-edge portion of the data on the y=0 axis. The pre-edge subtract