6 Analysis Using Fourier Transforms
W. Fred McClure
CONTENTS
6.1 Introduction . . . 93 6.2 Fourier Mathematics . . . 94 6.3 The Fourier Transformation . . . 95 6.3.1 The Periodicity Requirement. . . 95 6.3.2 Fourier Transforms . . . 97 6.4 Qualitative Spectral Analysis. . . 100 6.4.1 Convolution Smoothing in Wavelength Space . . . 100 6.4.1.1 Convolution . . . 100 6.4.1.2 Moving Point Average Smoothing . . . 100 6.4.1.3 Polynomial Smoothing . . . 101 6.4.1.4 Fourier Smoothing . . . 102 6.4.2 Deconvolution via Fourier Space . . . 104 6.4.3 Computing Derivatives via Fourier Space . . . 106 6.4.3.1 Fourier Derivatives . . . 108 6.4.4 Corrects for Particle Size Anomaly . . . 109 6.5 Quantitative Spectral Analysis . . . 109 6.5.1 Estimating Composition with Fourier Coefficients . . . 109 6.5.2 Cutting Computer Storage Requirements by 98% . . . 111 6.5.3 Reducing Calibration Times by 93% . . . 112 6.5.4 Encouraging Calibration Maintenance . . . 112 6.5.5 Reducing Multicollinearity . . . 113 6.6 Artificial Intelligence . . . 113 6.6.1 Spectral Searching, Matching, and Component Identification . . . 113 6.6.2 Checking Instrument Noise in Real Time . . . 115 6.6.3 Testing for Instrument Anomalies in Real Time . . . 115 6.7 Interferogram Space . . . 117 6.7.1 Estimating Chemistry from Interferogram Data . . . 117 6.7.2 No-Moving-Parts FT-NIR . . . 118 References . . . 119 Bibliography . . . 119
6.1 INTRODUCTION
Fourier transform infrared (FTIR) techniques are well-established techniques [1–3] and have been for a long time. However, the use of Fourier transforms in the analysis of near-infrared (NIR) spectra 93
had not been tried until 1981, especially Fourier analysis of NIR spectra obtained from dispersion instruments. Our [1]∗ excitement over the use of Fourier transforms in the analysis of NIR data was stimulated by work that was ongoing in the field of information processing. There had been some very exciting results in using Fourier transforms in the transmission of voice information.
Electrical engineers had demonstrated that voice information could be carried in just a few low-frequency Fourier coefficients. The work produced magnificent results. First, the voice information or signal was digitized. The Fourier transform was taken of the signals and the Fourier coefficients were transmitted down the line. However, only the first few Fourier coefficients were transmitted and, on the other end, the Fourier transform was taken again, giving intelligible voice information.
Some fidelity, such as the distinction between male and female voice, was lost but the content of the information was there.
The driving force behind our interest in this area at that time was three problems encountered working in wavelength space. First there were too many data in a spectrum. At that time we were recording 1700 data points per spectrum from 900 to 2600 nm. Archiving spectra required a tre-mendous amount of magnetic storage and the problems of retrieval were becoming insurmountable.
Second, the data within a spectrum were highly intercorrelated, a phenomenon referred to as multi-collinearity. This meant there were superfluous data in the spectrum, but there was no sound basis for discarding any. Third, NIR calibrations based on wavelength data were rather unstable. That is to say, if new spectra were added to a calibration set and another multilinear calibration obtained, the wavelengths included in the new calibrations may not be the original ones. There were no funda-mental principals in place at that time for selecting the most appropriate wavelengths for calibration equations.
It occurred to us that if sufficient voice information was encoded in the first few Fourier coef-ficients to give intelligible information when “retransformed,” we should be able to use this same technique to compress NIR spectra to a smaller set of numbers. This in turn would result in much faster and more stable calibrations. We performed our first experiments in 1978 but, because the results were so unbelievable, we did not publish the data until much later [4,5]. We were extremely pleased to discover that as few as six pairs of Fourier coefficients, just 12 numbers, from log(1/R) spectra could be used to estimate chemistry in pulverized samples of tobacco without loss of accuracy or precision [5]. Since that time we have demonstrated that a number of advantages can be accrued by working in the Fourier domain (FD) rather than the wavelength domain (WD). In this chapter, we will discuss the advantages of working in Fourier space from the qualitative (special transformations) and quantitative (chemical determinations and efficiency) standpoint.
6.2 FOURIER MATHEMATICS
Any well-behaved periodic function can be represented by a Fourier series of sine and cosine waves of varying amplitudes and harmonically related frequencies. This may be illustrated using the square wave function given in Figure 6.1a. Couching the illustration in spectroscopy jargon, let us assume that the square wave spectrum represents an absorption band with an absorbance of 1.0 centered at 1300 nm. We know that this spectrum can be defined mathematically in terms of sine and cosine terms as follows:
f(λ) = a0+ (a1sinωλ + b1cosωλ) + (a2sin 2ωλ + b2cos 2ωλ)
+ · · · + (aksin kωλ + bkcos kωλ) (6.1)
∗ The author gratefully acknowledges the collaboration of Dr. Abdul Hamid in the early stages of the development of Fourier analysis of NIR data.
2.000 1.600
1.200 (f)
(c) (d)
0.800 0.400 0.000
log(1/R)
1.100 1.140 1.80 1.220 1.260 1.300 Wavelength (M)
1.340 1.380 1.420 1.460 1.500 0.400
0.800
−1.200
−1.600
−2.000
(a) (b) (e)
FIGURE 6.1 Square-wave spectrum (a), inverse Fourier transforms using a0and b0 (b), a1 and b1(c), a2and b2(d), a3and b3(e), and a0, b0, through a49and b49(f).
The first term a0 is the average value of the spectrum (called the mean term, Figure 6.1b). The weighting factors ai, biare sometimes referred to as coefficient pairs. The magnitude of ai and bi
determines the amplitudes of the contributing terms. We will show later that these are really the Fourier coefficients where i= 1, 2, 3, . . . , k is the Fourier index.
For periodic functions the b0coefficient is always zero. The terms involving a1and b1are the sine and cosine components with one oscillation in the internal 1200 to 1300 nm (Figure 6.1c);
the terms involving a2 and b2 are the sine and cosine components with two oscillations in this interval (Figure 6.1d); the terms involving a3and b3are the sine and cosine components with three oscillations in this interval (Figure 6.1e). Hence the terms involving aiand biare the sine and cosine components havingi oscillations in this interval. Including enough terms in Equation (6.1) (say, 50 pairs) will produce an excellent facsimile of the square wave (Figure 6.1f). Table 6.1 gives the coefficients for the square wave transformation; note that even-numbered coefficients are always zero, a characteristic peculiar to the square waveform.
6.3 THE FOURIER TRANSFORMATION 6.3.1 THEPERIODICITYREQUIREMENT
If a square waveform can be adequately represented by a series of sine and cosine terms, then any NIR spectrum can be represented as a Fourier series of sine and cosine waves of varying amplitude and harmonically related frequencies provided the spectrum is periodic and continuous. A typical NIR spectrum of tobacco seeds is given in Figure 6.2a. From the standpoint of periodicity, there exists a discontinuity on the right end of the spectrum due to the dominant linear trend upward as wavelength increases. That is, if we lay the same spectrum end-to-end repeatedly, this discontinuity makes the spectrum aperiodic. When we use multiple linear regression (MLR) in wavelength space [16] for the purpose of computing the chemical content of a series of samples, this linear trend as well as the discontinuity can be disregarded. However, it cannot be ignored if a spectrum is to be
TABLE 6.1
First 40 Fourier Coefficients of the Square-Wave Spectrum