Data reduction with principal components analysis (PCA)

It has been pointed out by D A. Bums and E.W. Ciurczak (Bums and Ciurczak, 1992) that the initial mathematical foundation for what is now called principal components analysis (PCA) can be traced as far back as 1901 given by K. Pearson, then further developed into the modem use PCA by H. Hotelling in 1933 (Hotelling, 1933). Some applications to NIR analysis are found in work of LA. Cowe (Cowe, 1985, 1988), H. Mark (Mark, 1986), H. Martens (Martens, 1988) and there are many others.

Before the report of principal components analysis, it is necessary to introduce important concepts. The following illustrates a matrix representation and the concept of projection of vectors, then the use of PCA in spectroscopic data (X matrix) is described.

N ea r In frared S pectroscopy Technique f o r B ioprocess M onitoring a n d C ontrol C hapter 3

3.4.2.1

Matrix representations and projection

In a low dimensional matrix where the number of rows and columns are small, graphical representation of the matrix can be simple. For example the following 2 x 3 matrix

X

X

X

X =

0 2 5

Row space is the space formed with the rows of X as the axes (figure 3.7a) and hence it is two dimensional. In figure 3.7a, three points representing a corresponding column in the matrix in this two dimensional space. Similarly, the column space (figure 3.7b) has two points in the three dimensional space.

R o w 2 6 - C o lu m n 3 ( 4 . ^ ^ 4 2 C o lu m n 2 ^ ( 3 , 2 ) ___ _— - C o lu m n 1 - 2 2 4 6 R o w 1 - 2 -A- -6 a) C o lu m n 3 6- ( 0 ,2 , 5 ) R o w 2 4 - ( 2 .3 , 4 ) 2 R o w 1 6 C o lu m n 1 -2 C o lu m n 2 - 4 - -6

Fig 3.7 a) The matrix X shown in row space, b) The matrix X shown in column space.

N ear Infrared S pectroscopy Technique f o r B ioprocess M onitoring an d C on trol C h apter 3

seen as being the perpendicular shadow of one object onto another. The illustration in figure 3.8, the result of projecting a vector co-ordinate by 3 dimension on a plane of 2 dimension.

q Projection of y

Fig 3.8 Projection of y down on the space spanned by the vectors and X2

3.4.2.2

Factors and principal component analysis (PCA)

The concepts of factors forms the basis of partial least squares (PLS) and principal component regression (PCR) model construction. These factors, also known as the principal components, are defined as any linear combination of the original variables in X or Y matrix (Sharaf, 1986). It can be shown that given J factors for / x 7 matrix X, one can also represent the variables in X as a linear combination of these same J

factors. It is important to determine factors that represents useful (analyte response) and also small features (such as noise components) in the X matrix and model can be based on significance of factors. For example, the information in the / x 7 matrix X can be expressed as / x 7 matrix X"^^, where the columns of X"®'^ are linear combinations of the original columns in X. The primary advantage is that if a particular column in X is not useful, then small weight is given to the factor representing that column. Similarly, if the column in X is useful then the factor will be given a larger weight. The following examples uses a simple 5 x 2 matrix X to illustrate the role of principal components.

N ear Infrared Spectroscopy Technique fo r Bioprocess M onitoring and Control Chapter 3

x =

In principal component analysis, the columns of X are often mean-centred and scaled. The process of mean-centring requires subtracting the average of the column from each element of that column. The scaling provides equal weights to each variable, achieved by dividing each element by the variance of the column. The PCA is then performed on the covariance matrix X X formed from the mean-centred and scaled X matrix . The first eigenvector corresponding to the largest eigenvalue is, by definition, the direction in the space defined by the columns of X that describes the maximum amount of variation in the samples. This is represented in figure 3.9, the data and the direction of the first eigenvector where the space defined by X is a plane (2 dimensional). C o lu m n 2 E i g e n v e c t o r 2- 2 C o lu m n 1 -2 - 3 --

N ear Infrared Spectroscopy Technique f o r Bioprocess Monitoring and Control Chapter 3

In this illustration, all of the variation in the data has been described by one eigenvector. The samples all fall on a line in the column space of X X, thus all of the variation lies in one direction. With PCA, if all of the variation in the samples cannot be explained by one eigenvector then a second or subsequent eigenvector will be found. The second eigenvector is perpendicular or orthogonal to the first and this describes the maximum amount of residual variation that was not described by the first eigenvector. In figure 3.10 is another demonstration of projecting a 20 x 2 data matrix X onto two eigenvectors in X X The direction of the first eigenvector describes the maximum amount of variation in the data set. The samples in column space fall within a two dimensional clusters, and the first eigenvector has described to the major axis of the cluster.

C o lu m n 2 E i g e n v e c t o r 1 6 - 4 6 C o lu m n 1 E i g e n v e c t o r 2 -6-

Fig 3.10 A 20 X 2 matrix X is plotted in column space with illustration of the first and second

eigenvector of X X the eigenvectors are orthogonal to each other.

The first factor or principal component is a linear combination of the original variables and can be written as follows;

N ear Infrared S pectroscopy Technique f o r B ioprocess M onitoring a n d C ontrol C h apter 3

where X can be the spectroscopic data, p is the first eigenvector of X X, and t is the