Image Processing of Atomic Resolution Transmission
Electron Microscope Images
Young-Min Kim, Jong-Man Jeong, Jin-Gyu Kim and Youn-Joong Kim∗
Division of Electron Microscopic Research, Korea Basic Science Institute, Daejeon 305-333
Young Soo Lim
Corporate R&D, LG Chem Research Park, Daejeon 305-380
(Received 17 October 2005)
A practical application of image processing to atomic resolution transmission electron microscope (ARTEM) images was introduced. The processing was achieved by using a fast Fourier transform (FFT) of the experimental ARTEM micrograph, Fourier mask filtering of the FFT pattern, and its inverse FFT. As a result of the process, by applying appropriate Fourier mask filters that limit and cut the spatial frequency in the diffractogram, we were able to significantly improve the image quality related to brightness and contrast. A complicated ARTEM image due to the complex phase interference was easily reconstructed by using a modified Fourier mask application. In addition, digital image processing provided effective visualizations available for a variety of presentations, which accentuated the results of the ARTEM work. Here, we present some examples of digitally manipulated ARTEM images processed by using a Fourier mask filtering technique.
PACS numbers: 42.30.Va, 61.16.Bg
Keywords: Image processing, Atomic resolution, ARTEM, Fast fourier transform (FFT), Fourier mask filtering
I. INTRODUCTION
Image processing by Fourier mask filtering in high resolution transmission electron microscope (HRTEM) images is widely employed in producing a contrast-enhanced image and is achievable through application of a suitable spatial frequency filter in the Fourier space of a digital electron micrograph. This processing technique also provides various information for material character-izations, such as phase-separated images, high-resolution dark-field images, and defect structure images, from the HRTEM results [1–3]. By virtue of recent advances in computer technology, the digital signal processing in the FFT algorithm breaks up a large computation into smaller computations of a continuous periodic function [4]. Many software packages are capable of FFT process-ing for HRTEM micrographs and are distributed for free or commercially so that one can select software to meet with one’s purposes [3]. Most recent HRTEMs equipped with charge-coupled device (CCD) camera systems have built-in software for the FFT algorithm for image pro-cessing, which makes a real time diffractogram possible. Some researchers have employed such real-time
diffrac-∗E-mail: [email protected]; Fax: +82-42-865-3939
togram technique by using the TV rate (1/30 sec) to do a dynamic study of the atomic-scale defects and to adjust the electron lens automatically [1,5].
HRTEM images are phase-contrast images and result from the interference of the transmitted and diffracted beams [1,2]. In the Fourier reciprocal space of the image converted by using the FFT process, a maximized image intensity, excluding background signals, can be achieved by using a Fourier mask filtering technique. The idea of Fourier mask filtering of the HRTEM image seems to be similar in a way to the image formation principle in a simplified TEM. By treating the electrons as waves and considering a simplified electron microscope as shown in Fig. 1, we see that there are three planes in the TEM: the image plane, the focal plane of the objective lens, and the specimen exit surface. When an electron beam passes through the specimen exit surface and into the objective lens, the transmitted and the diffracted beams satisfying the Bragg condition are focused at the back focal plane of the objective lens and form an electron diffraction pattern. In this plane, the space where the electron diffraction pattern can be observed is called the reciprocal space, which is mathematically given by the Fourier transform of the real space. Inserting a circu-lar objective aperture to limit the spatial frequency to
-250-Fig. 1. Simplified TEM (left) and the mathematical rela-tionship required for the image calculation (right). The three principal planes in the TEM consist of the specimen plane, the objective back-focal plane, and the image plane.
a certain value in the back focal plane will define the resolving range of the electron microscope. This oper-ation is analogous to Fourier mask filtering for digital processing in HRTEM images. Since all the processes have been achieved by computation, the Fourier mask filtering technique does provide more opportunities to select various filtering pattern types, such as twin oval, periodic, circular band-pass, and wedge shape patterns, which make it easy to extract and maximize the spe-cific structure information. If the transmitted and the diffracted beams interfere on the image plane, the recip-rocal space can be mathematically reconstructed again in real space to form a magnified image corresponding to its inverse Fourier transform. Therefore, the main task of the Fourier mask filtering technique is to select certain spatial frequencies in reciprocal space and then to reconstruct a direct specimen image from the filtered spatial frequencies by using the FFT-IFFT relationship. However, when we consider atomic resolution transmis-sion electron microscope (ARTEM) results, such as the images obtained from high-voltage electron microscopes (HVEMs), special care is required to avoid mistakes in applying the Fourier mask filtering technique.
In the present study, we discuss the practical as-pects of image processing by using a Fourier mask fil-tering for ARTEM images. First, we show examples of ARTEM images enhanced by using a digital image pro-cessing; then, we consider some precautions in applying the Fourier mask filtering technique. Furthermore, we detail the software techniques for effective visualization by digitally manipulating the image intensity.
IFFT calculation, for the ARTEM images was performed using a software program, Digital MicrographTM (DM, Gatan Inc.). For potential users, a demo version of this software, which is fully functional except for the sav-ing and the exportsav-ing functions, can be downloaded at the company web-site [6]. This software has built-in scripting capabilities to permit customizing the DM soft-ware functions for individual experimental needs ranging from simple image manipulation to microscope parame-ter control [7]. Most of the useful script resources in practical data manipulation have been established in the web-based database, Digital Micrograph Script Database (DMSD, Graz university of technology), which is avail-able for download [8]. In this study, the various short-comings of the software were solved by installing the user-made scripts.
III. RESULTS AND DISCUSSION 1. Image Processing of an ARTEM Image by using a FFT and a Fourier Mask Filtering Technique
In order to do the Fourier mask filtering, the diffrac-togram resulting from the FFT calculation for the region of interest of the ARTEM image should firstly be formed; then, the proper filter patterns should be applied. From the IFFT calculation, the reconstruction of the filtered diffractogram transforms to a processed image with en-hanced image quality.
Fig. 2 shows an example of the FFT, Fourier mask filtering, and reconstruction by IFFT for the [110] Si ARTEM image. The experimental ARTEM image of the [110] Si without image processing is not clear, as shown in Fig. 2(a), because of the background signal caused by the amorphous surface layer and the instru-mental noises. When the FFT and Fourier mask filtering technique is applied, the processed ARTEM image (Fig. 2(b)) presents an explicit atomic structure image. For a high-quality processed image, the Fourier mask filter-ing procedure should be performed in steps (1) through (5), as depicted in the lower part of Fig. 2. (1) First, the FFT operation is completed to obtain a diffractogram on
Fig. 2. Example of image quality enhancement by using the fast Fourier transform (FFT) and Fourier mask filtering technique for the [110] Si atomic resolution image. The inset superposed in (b) shows an atomic model of [110] Si.
the region of interest from the [110] Si ARTEM image. (2) A pattern mask filter corresponding to the structural periodicity of the [110] Si in the diffactogram is selected to do the filtering, which cuts off the spatial frequency components, irrespective of the structural signals of the [110] Si. (3) A band-pass mask filter should be used to avoid the accompanying background image originat-ing from the spatial frequency component beyond the spatial frequency maximum defined by the experimen-tal diffraction pattern of the [110] Si structure. (4) Af-ter band-pass mask filAf-tering, the diffraction spots corre-sponding to the spatial frequencies related only to the experimental ARTEM result of the [110] Si can be se-lected. In this step, the absorption and strain contrast effect can be digitally excluded by blocking the transmit-ted beam, which is similar to a so-called high-resolution dark-field image that eliminates the strong strain con-trast and equalizes the background concon-trast level in the experimental image. In particular, this technique is ef-fective for processing of ARTEM images generated from a single crystalline material or a material with a highly strained interface area. (5) Finally, the filtered diffrac-togram generated through the process is transformed to a real space image by using an IFFT calculation. As a result, the processed ARTEM image shows a higher image quality embracing a maximized structure contrast and minimized background signal.
To estimate the image quality, we compare the histogram results of the pre-processed and the post-processed images, as presented in Fig. 3. Since a higher electron count and a narrower peak width in the profile indicate that the image has higher contrast and bright-ness, the processed atomic resolution image of the [110] Si is expected to exhibit an improved image quality. Note, however, that the histogram profile of the
pro-Fig. 3. Comparison of the image quality between the un-processed and the un-processed images by using a histogram analysis.
cessed image does not show the real electron gain on the detector while that of the unprocessed image describes the real electron acquisition.
When the phase information is extended beyond the point resolution of the electron microscope, the phase contrast will not be intuitively coincident with the atomic potential of the specimen anymore because the phase amplitude oscillates with increasing spatial fre-quency (scattering vector) beyond the instrumental reso-lution. Therefore, such an image does not directly reflect the atomic structure of the specimen. Fig. 4(a) shows an example of a complicated image mixed with additional phase information over the information limit up to 19.2 nm−1, corresponding to the 0.52 ˚A planar spacing of the Si (10 2 2) plane. Although the phase information reaches the ultimate spatial frequency region exceeding the point resolution of the instrument (1.17 ˚A), that does not mean the resolving power of the instrument is im-proved. Contrarily, the consequence of the contrast re-versal arising from the oscillation of the phase amplitude makes it difficult to interpret the ARTEM image itself. If the extended phase information transfer in the basis of the super-resolution scheme is to be used constructively, a “through focus exit-wave reconstruction technique” for enhancing the direct resolving power of the electron mi-croscope has been employed by some researchers [9–11]. In that case, special care is required to obtain the in-strumental parameters and the experimental data. In-stead of applying the difficult technique above, there is a simple method to recover the atomic structure image of [110] Si. The main idea is to apply a virtual aper-ture by applying the Fourier mask filtering technique to the experimental diffractogram, which defines a phase information limit not exceeding the point resolution of the microscope. The Si-Si atomic spacing of the [110] Si dumbbell structure is 1.36 ˚A, corresponding to the planar spacing of the Si (4 0 0) plane. For that reason, higher-order diffraction than the Si (4 0 0) spot is of no
Fig. 4. Example of the reconstruction of the phase contrast image of the [110] Si atomic structure by using the Fourier mask filtering technique ((a) and (b)). Additional phase am-plitudes with no relation to the [110] Si atomic structure were excluded because their information exceeded the direct inter-pretable resolution of the instrument (1.17 ˚A), which led to removing the contrast reversal effect and reconstructing the real structure from the experimental figure in (a) as shown in (b). The related contrast transfer function of the microscope is shown in (c).
use in imaging the atomic structure of [110] Si, so the size of the virtual aperture should be selected to keep the masked area encircling only the Si (4 0 0) diffraction spot. The filtered diffractogram, as shown in the inset of Fig. 4(b), is attributed to the application of the vir-tual aperture filter after pattern mask filtering. From the IFFT calculation of the filtered diffractogram, we see in Fig. 4(b) that a realistic atomic image representing the [110] Si dumbbell structure can be clearly reconstructed and resolved. Considering the CTF (contrast transfer function) calculation with the experimental instrument parameters (Cs = 2.65 mm, convergence angle = 0.25 mrad, and spread of defocus = 50 ˚A) under the Scherzer condition (∆ f = −52 nm) [12], as shown in Fig. 4(c), we can clearly see that if we cut off the spatial frequency just beyond the instrument resolution (1.17 ˚A) by using a Fourier mask aperture, all the included phase amplitudes are in the same contrast region and contrast reversal in the image does not happen because the smallest atomic spacing of the [110] Si dumbbell structure is inside the direct interpretable resolution of the instrument.
From the results so far, we know that by using the FFT-Fourier mask filtering technique, contrast-enhanced ARTEM images and phase-modified ARTEM images can be obtained. However, this image processing should be
pattern mask. The high-order diffraction spots can be easily de-selected, especially in case of large crystal cells, because a small error in centering the circle mask on each diffraction spot will be tangentially magnified as the order of the diffraction increases. If such errors in the pattern masking are to be reduced, the best way is to enlarge the diffractogram as much as possible and to place the correct center of the pattern vector into the diffraction spot and then use the smallest circle mask fitted for the pattern. (3) After the IFFT calculation of a masked diffractogram for a second-power reduced region-of-interest of the ARTEM image, a coarse image can be readily formed because the image resolution (pixel number) is reduced in the same way. We, therefore, suggest that the reconstructed image should be scaled up afresh over the second power. (4) If the transmit-ted beam is de-selectransmit-ted in the band-pass mask filtering, strain contrast is removed. Accordingly, we recommend that this mask should be restrictedly applied to either a single-crystalline image or an image involved in severe strain contrast that obstructs the structure detail of the ARTEM image.
2. Digital Manipulation of the Image Intensity for Visualization
The image intensity in the ARTEM image is displayed in grey scales. For example, an 8-bit image gives 256 grey levels while 12-bit and 16-bit images are in accordance with 4096 and 65536 grey levels, respectively. Most of the recent CCD cameras in TEMs have either 12- or 16-bit intensity depth. Therefore, we recommend that the image reconstructed from the FFT-Fourier mask filter-ing be stored in higher than 12 bits for manipulatfilter-ing the image intensity quantitatively for fine assignment of the elements in the structure. Handling the image in-tensity of the processed ARTEM image for quantitative analysis has been diversely executed by others [13, 14]. This software technique provides impressive visualiza-tions to emphasize and diversify the result of ARTEM work. Tracing the grey intensity level of an ARTEM im-age and its diffractogram can be a way to quantitatively
Fig. 5. Quantitative atomic position assignment of Zn and O in the [010] ZnO nanowire structure by image intensity estimation. (a) Atomic resolution image and (b) schematic atomic model of [010] ZnO (symbols: black = Zn, grey = O). (c) Intensity profile and atomic distance measure of the Zn-O dumbbell corresponding to the solid white line marked in (a). Notice that the grey scale level in (c) is reversely represented for the familiar look.
investigate the structural change of a material caused by different experimental conditions, provided that all the data have been acquired under the same recording condition. Ohishi et al. [15] utilized this technique to graphically show the quantitative variation of thermally diffuse scattering in Au at different temperatures.
Since the digital image intensity of the ARTEM result directly reflects the atomic potentials of the elements in the specimen, obvious differences in the grey scale levels between the light and the heavy elements can be used as a resolving power to discriminate the elements. Image simulation for the ARTEM will also be a complemen-tary method to clarify the structure. Shindo et al. [13] achieved quantitative assignment of oxygen sites in the ARTEM image of a Tl2Ba2Cu1Oy superconducting ma-terial with this technique. Fig. 5(a) shows an example of realizing the atomic contrast between Zn and O in a [010] ZnO nanowire [16–19]. As the higher-atomic number of Zn is supposed to cause a darker contrast, meaning that its higher atomic potential gives rise to a larger scattering probability, Zn atoms are murkier than O atoms in the ARTEM image (Fig. 5(a)), which matches the atomic model depicted in Fig. 5(b) well. The intensity profile traced from the solid white line marked in Fig. 5(a) rep-resents the difference in the Zn-O chemical contrast, as shown in Fig. 5(c). Notice that the grey scale level in that graph is reversely represented for the familiar look. Although the atomic distance of Zn-O (1.14 ˚A) in the [010] direction is beyond the point resolution of our in-strument (1.17 ˚A), we can see that the intensity profile allows us to distinguish the atomic positions of Zn and O in the structure.
Fig. 6. Some illustrations of the digital processing tech-niques to exploit the image intensity in the [110] Si atomic resolution image for the versatile presentations.
Some displaying techniques for a monotone ARTEM result by manipulating the image intensity, such as a contour map, a rainbow map, a grey-scale contour map, and a temperature map, are presented in Fig. 6. The intensity depth of the experimental image is divided into a certain grey value decided by the user so that con-touring and colorization in the basis of an intensity di-vider can be accomplished. These functions can be ac-tivated in the DM software by installing the customer script (color-look-up-table) downloadable at DMSD [8]. Various scripts for image processing can also be found at that site, which is likely to respond to specific requests by a number of users for additional features or modified capabilities.
IV. CONCLUSIONS
In this paper, an image processing for an ARTEM image by using a FFT-Fourier mask filtering technique has been introduced. We have demonstrated that the processed ARTEM image resulting from this technique shows not only improved image quality which eliminates the unwanted noise and background signal in the experi-mental image but also the possibility of extracting quan-titative information, such as the chemical contrast, on a atomic scale. In addition, the possibility that com-plicated ARTEM image due to the complex phase in-terference is reconstructed by using a modified Fourier mask application is suggested, and practical precautions have been discussed for applications of this technique. Furthermore, some digital manipulation techniques that might be particularly useful for a monotone ARTEM im-age have been described for more versatile imim-age process-ing and presentation.
Y. Yokota,International Symposium on In Situ Experi-ments with HVEM(Osaka University, 1985), p. 295. [6] DM: Digital Micrograph version 3.0 manufactured by
Gatan. The demonstration program can be downloaded at www.gatan.com and provide full functions except for saving the data.
[7] D. R. G. Mitchell and B. Schaffer, Ultramicroscopy103, 319 (2005).
[8] DMSD: Digital Micrograph Script Database, http: //www.felmi-zfe.tugraz.at/dm scripts/dmscript1.html.
[16] K. Murakami, M. Saito, E. Takuma and H. Ichinose, J. Electron Microsc.52, 27 (2000).
[17] J. Y. Kim, H. W. Shim, E.-K. Suh, T. Y. Kim, S. H. Lee, Y. H. Mo and K. S. Nahm, J. Korean Phys. Soc.44, 137 (2004).
[18] W. I. Park, J. Yoo and G.-C. Yi, J. Korean Phys. Soc. 46, L1067 (2005).
[19] E. S. Jung, J. Y. Lee, H. S. Kim and N. W. Jang, J. Korean Phys. Soc.47, S480 (2005).
