Normalisation and scatter correction

1.8.1.3.1. Standard normal deviate (SNV)

In standard normal deviate (SNV, also called standard normal variate) normalisation^'

the ordinate (intensity, absorbance etc.) of a spectrum at each wavelength, ysNVi » is replaced by :

_ (yi - y) ysNVi - ---

where ÿ is the mean spectral value over entire range, s is the standard deviation of those values and y,. is the ordinate value at wavelength i. This transformation is useful for removing effects due to changes in particle size.

1.8.1.3.2. Multiplicative scatter correction (MSC)

Unlike SNV, this normalisation^^’ uses a group of spectra to calculate the

transformation. From a mean spectrum of the group a linear regression of the ordinate values from each original spectrum is performed. From the intercept, a, and slope of

the regression, b, an MSC transformed spectrum is calculated:

_ iy, - a) y MSC,i 1

where y. is the ordinate value at wavelength i. Effectively, each spectrum is therefore adjusted to fit the mean spectrum as closely as possible. It should be noted that the mean spectrum must be retained to transform spectra outside of this group, e.g. in validation and test sets.

1.8.1.3.3. Normalisation to 1, or maxima (N1, or Max)

In normalisation to 1 (or maxima)^"^ the ordinate (intensity, absorbance etc.) of a spectrum at each wavelength, y Maxi » replaced by :

yi yMaxi - y

^Max

where y,. is the ordinate value at wavelength i, and Ymox is the maximum value of

intensity or absorbance across the entire wavelength range.

1.8.1.3.4. Normalisation to 0, or minima (NO or Min)

In normalisation to 0 (or minima)^"^ the ordinate (intensity, absorbance etc.) of a spectrum at each wavelength, y Mini » i^ replaced by :

y M in i y i ~ ^M in

where y,. is the ordinate value at wavelength /, and Y Min is the minimum value of intensity or absorbance across the entire wavelength range.

1.8.1.3.5. Normalisation to maxima and minima (N01)

absorbance etc.) of a spectrum at each wavelength, » is replaced by :

y N o u =

^Max ^Min

where y,. is the ordinate value at wavelength i, and is the minimum value and

Yj^ax the maximum value of intensity or absorbance across the entire wavelength range.

1.8.1.3.6. Vector normalisation (VN)

In vector normalisation (VN)^"^ the ordinate (intensity, absorbance etc.) of a spectrum at each wavelength, yyj^ ., is replaced by :

yi

I 2 2 2 2 T

+ )^2 -^3 -^4

y»

where y,. is the ordinate value at wavelength i, and y„ is the value of intensity or absorbance for each wavelength across the spectral range from 1 to «.

1.8.1.3.7. Normaiisation by Ciosure (NC)

In normalisation by closure (NC)^"* the ordinate (intensity, absorbance etc.) of a spectrum at each wavelength, » is replaced by :

yi X »

yNCi = - — ;= i

Where y,. is the ordinate value at wavelength i, and y j is the value of intensity or absorbance for each wavelength j across the entire wavelength range (from 1 to n).

1.8.1.3.8. Normalisation by unit area (NAX)

In norm alisation by area (NAX)^'^ the ordinate (intensity, absorbance etc.) o f a spectrum

at each w avelength, y / ^ ^ X i, is replaced by :

_ \yi ^ "I

y N A X i - —

Eh

y=i

where y, is the ordinate value at wavelength i, and y j is the value o f intensity or

absorbance for each w avelength j across the entire w avelength range (from 1 to n). Essentially, this is very similar to normalisation by closure (N C ) except here the absolute values are used.

1.8.1.4. Derivatives (D1, D2 etc.)

Derivatisation (abbreviated to D #, where # is the derivative order) o f a zero order spectrum is a useful technique for visually enhancing the fine curved structure within a NIR spectrum, see Figure 1.17.

Figure 1.17 Effect of first and second order derivatisation on a zero order

Gaussian peak

Zero order First order I

Second order

A peak in a zero order spectrum becom es a point o f inflection in 1®‘ derivative and a m inim a (negative peak) in 2"^ derivative. Since noise is often com prised o f high

frequency com ponents then this m ay be am plified by differentiation^^ and therefore, the signal to noise ratio decreases as each operation adds noise. Sm oothing (see section

1.8.1.2, page 56) can be used to reduce the effects o f noise but m ay hide m ore weaker spectral features.^^ One useful outcom e is to rem ove baseline sloping.^^

1.8.1.4.1. Moving average difference derivative (MADD)

The principle o f this approach is to calculate the difference betw een the mean value o f blocks o f data points on either side o f the point at w hich the derivative is required. Increasing the block size increases smoothing. A further increase in sm oothing can be obtained by introducing a gap (also referred to as segm ent). This is explained more easily by w ay o f illustration, Figure 1.18.

Figure 1.18 Representation of spectral data points for calculation of a

derivative using a moving average approach, e.g. where block size = 7 and gap size = 3 data points

data point to be replaced by difference data points

2

c

block gap gap block

W a v e le n g th

To calculate the first derivative using the data in Figure 1.18 the m ean value o f intensity in the second block is subtracted from the mean value in the first to obtain the new value.

This is then repeated across the w h ole spectrum m oving one data point at a tim e. For higher order o f derivatives this procedure is sim ply repeated. For this type o f derivative the notation [« /, «2, nj,] is used in this thesis, where nj = block size, « 2 = gap (or

1.8.1.4.2. Polynomial derivatisation by Savitsky and Golay (SG)

This method for calculating derivatives (and also smoothing) is provided as a function for many commercial spectrometers. Effectively,* a polynomial of a specified order is fitted (by least squares) to a set number of data points either side of the point where the derivative is required. Typical parameters would include a polynomial order of between 2 and 4, with 3 to 10 data points either side of the data point to be replaced by

difference.

For this type of derivative the notation [«;, «2, « ij is also used in this thesis, where «7 =

block size, «2 = gap (or segment) size, and wj = derivative order.