5.5 Evaluation and Discussion
5.5.5 Overall Discussion
A summary of the results generated by the experiments reported in this chapter is shown in Tables 5.9 and 5.10. In the tables each row represents the results produced by the best performing parameter settings, with respect to the best classification accuracy generated. The aim was to compare the performances of the three different proposed time series based image classification approaches advocated in this chapter. All features were extracted from the RGB colour model of the images. The left-most column labelled “Ap.” represents the nature of the CB used to generate the results. The first three rows present the results generated using a single CB approach, while the last row presents the results produced using two CBs.
Table 5.9: Summary of average best classification results obtained using the proposed approaches and theBDdataset
Ap. Sens (%) Spec (%) Accuracy (%) AUC (%)
CBH 74.0(1.5) 57.4(1.5) 67.8(0.6) 73.9(0.8)
CBH 76.0(1.2) 55.7(2.0) 68.4(1.2) 76.5(0.7)
CBHb 72.3(1.5) 61.8(2.8) 68.4(0.9) 70.9(1.3)
CBH+CBH 74.5(1.6) 60.4(1.4) 69.3(0.5) 73.2(1.4)
Table 5.10: Summary of average best classification results obtained using the proposed approaches and theMD dataset
Ap. Sens- Sens- Spec (%) Accuracy (%) AUC (%)
AMD(%) other(%) CBH 57.0(2.0) 46.2(1.1) 43.9(2.4) 50.2(0.6) 71.2(0.1) CBH 57.8(1.6) 41.4(1.1) 47.3(2.8) 49.8(0.9) 68.7(0.4) CB b H 57.0(1.9) 48.0(1.4) 50.5(3.1) 52.4(1.3) 70.7(0.2) CBH+CBH 54.1(1.4) 45.3(1.2) 48.8(1.5) 49.8(0.8) 73.0(0.3)
As can be seen from Table 5.9, use of CBH produced the best results with respect to sensitivity and AUC, the best specificity was produced using the CB
b
H, while the two CBs approach produced the highest accuracy. From Table 5.10 it can be seen that the use of CB
b
H produced the highest results with respect to three of the evaluation metrics used (sensitivity to identify other disease, specificity and accuracy), while use ofCBH and the two CBs approaches produced the highest sensitivity to identify AMD images and AUC respectively. Based on the results, histograms that capture both colour and spatial information, spatial-colour histograms (stored in case base CB
b H), performed the best when the MDdataset was used, while histogram without the optic disc information performed better with respect to the BD dataset. With regard to the appropriate number of histogram bins, with respect to the results shown in Tables 5.9 and 5.10, W = 64 tends to produce the best results. Identification of the most appropriate T value to adopt to obtain the best performance was not conclusive, but most of the best results were obtained using T ≥ 3. Using two CBs, as shown in the table, produced comparable results to a single CB approach but with a higher computational cost.
5.6
Summary
Approaches to classify images using a time series based representation derived from colour histograms, coupled with TSA and CBR, have been described in this chapter. A number of different types of colour histograms were advocated, including spatial-colour histograms that capture both colour and spatial pixel information. Two approaches to
classify images using CBR were described, the first used a single CB while the second utilised two CBs. Evaluation of the proposed approaches resulted in the following main findings:
1. The single CB approach produced most of the best results compared to the two CBs approach.
2. Best results were most likely to be produced usingW ≤ 64.
3. An ideal value for T was not identifiable though good results tended to be pro- duced usingT ≥3.
4. The generated classification results indicated that the utilisation of colour his- togram as image features, coupled with TSA and CBR, was not sufficient for multiclass classification (at least in the case of retinal image datasets used in this thesis).
The following chapter presents a different image representation, founded on a tabu- lar based image representation, that outperforms the time series approach described in this chapter. However, the time series representation does provide a benchmark with which other approaches described later in this thesis can be compared.
Chapter 6
Tabular Based Image
Representation for Image
Classification
This chapter presents the second proposed image classification method, founded on a tabular representation, that utilises the basic 2-D array image format. The evaluation section in the foregoing chapter demonstrated that the combination of colour and spa- tial information tends to produced better classification performance than when using colour information alone. Therefore, the proposed tabular representation presented in this chapter utilised both colour and spatial information to identify image features com- prised of statistical parameters which can be extracted either directly or indirectly from the representation. These direct and indirect parameters can be extracted according to two strategies, either globally from the entire image, or as a result of partitioning the image to some level of decomposition (d) and extract the parameters on a region by region basis. We refer to the first strategy (S1) as the non-partitioning strategy and the second (S2) as the partitioning strategy. In the case of the S2 strategy the number of parameters will depend on the level of decomposition. In both cases a feature selection process is applied where the topT features are selected, partly so that the most discrim- inating parameters are used for the classification and partly so that the overall number of parameters to be considered is reduced. The rest of this chapter is arranged as fol- lows. Section 6.1 presents a more detailed overview of the proposed method. Sections 6.2 and 6.3 provide the details of how the parameter extraction and feature selection processes are conducted. The adopted classification mechanism is described in Section 6.4, while the classification strategy to achieve multiclass classification is considered in Section 6.5. Section 6.6 discusses the experiments used to evaluate the proposed method and reports on the results obtained. The evaluation includes consideration of the most appropriate value for T to be used for both S1 and S2, and the best level of decomposition (d) to be used with respect to S2. Finally, a summary of this chapter is given in Section 6.7.
6.1
Introduction
As discussed in the literature review (see Sub-section 3.5), the 2-D array image repre- sentation is the most common method used to represent images in a way that permits both the general application of learning algorithms and the extraction of statistical measures for purposes such as image classification. In most cases, the features used are derived statistically from the colour, texture or shape information. For the work described in this chapter, only colour and texture information were considered as we are interested in composition of the entire image and not individual shapes within it. Two mechanisms for feature extraction were adopted. The first extracts statistical fea- ture information directly from the 2-D image colour representation. Examples include colour, histogram and co-occurrence matrix features. The second applies a transform to the basic colour representation to produce a frequency based representation, from which features can then be extracted. The Discrete Wavelet Transform (DWT) was used for this later purpose (an alternative might have been the Discrete Cosine Transform). Thus the features used with respect to the proposed solution presented in this chapter were extracted using both the basic and a transformed 2-D array image representations. As already noted, two alternative tabular based image representation strategies were adopted: (S1) extraction of features from the whole image, and (S2) partitioning of each image into equal sized sub-regions and then extracting features from each sub-region. For S1 the number of extracted features was 15, which comprised various image proper- ties that encompass colour (6 features), histogram (2 features), co-occurrence matrix (3 features) and wavelet (4 features) based features. For the second strategy, the identical 15 features were generated but from each sub-region of the image. The total num- ber of possible features in this later case was thus 15 ×R, where R is the number of sub-regions, which in turn is dictated by the level of decomposition (d). Details of the features used in the image representation are described in the following section.
6.2
Features Extraction
The fifteen features used in the proposed tabular based image representation may be categorised as follows:
1. Features generated directly from the pixel colour information con- tained in the image (six features). The six colour features extracted were the average values for each of the RGB colour channels (red, green and blue) and the HSI components (hue, saturation and intensity). These values were computed directly from the 2-D array colour representation of each image.
2. Features generated from a “colour histogram” representing the colour information contained in the image (two features). The two histogram
based features were: (i) histogram spread and (ii) histogram skewness. Only the green channel colour histogram was used as this has been demonstrated to be more informative than the other channels in the context of retina image analysis [32, 158]. Once extracted, each histogram was normalised with respect to the total number of pixels of the ROI in the image. The histogram spread (also known as variance), hspread, and skewness, hskew, were computed as follow:
hspread= 1 |h| |h| X i=1 (ˆhi−h¯)2 (6.1) hskew= 1 |h| |h| X i=1 (ˆhi−¯h)3 (6.2)
where|h| is the number of histogram bins, ˆh is the normalised histogram and ¯h
is the normalised histogram mean.
3. Features generated from the co-occurrence matrices representing the image (three features). A co-occurrence matrix is a matrix that represents im- age texture information in the form of the number of occurrences of immediately adjacent intensity values that appear in a given direction P[74, 85]. Figure 6.1 shows an example of a 6×6 imageI and its corresponding co-occurrence matrix,
L. To constructL, a position operator,P, has to be defined. Four possible differ- ent directions can be used to defineP: 0◦,45◦,90◦ or 135◦(see Figure 6.2 whereX
is the pixel of interest). With reference to the co-occurrence matrix,L, in Figure 6.1, the number of different intensity values is in the range of [0 7], thus a matrix of 8×8 size is produced. Pis defined as 0◦, which means that the neighbour of a pixel is the adjacent pixel to its right. As shown inL, the position (2, 1) contains a value of 2 as there are two occurrences of pixels with an intensity value of 1 positioned immediately to the right of a pixel with an intensity value of 2 (as indicated by oval shapes in the figure) inI. The same applies to the element (6, 4) ofL that holds a value of 1 as there is only one pixel with an intensity value of 6 with a pixel with an intensity value of 4 immediately to its right inI, and so on. With respect to the approach described in this chapter, four co-occurrence matrices (one for eachPdirection) were generated for each image. Three textural features were then extracted from each matrix: (i) correlation, (ii) energy and (iii) entropy:
Correlation: Measure of how correlated a pixel is to its neighbours over the entire image. The range of values is [-1, 1], corresponding to negative or positive correlation. Correlation, corr, for a |L| × |L|matrix is defined as:
2 3 3 2 1 5 4 2 7 6 0 0 5 7 7 2 2 1 0 7 6 5 6 4 3 3 0 0 0 0 0 0 0 0 0 0 9 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 2 1 1 0 0 0 1 1 0 1 2 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 1 1 0 0 0 0 1 0 0 0 2 1
Image I Co-occurrence matrix L
0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7
Figure 6.1: An example of image and its corresponding co-occurrence matrix (P = 0◦)
X
0o135o
90o
45o
corr = |L| X x=x+1 |L| X y=y+1 (x−µrow)(y−µcol)Pxy σrowσcol , σrow 6= 0;σcol 6= 0 (6.3)
whereµrowandµcolare the means of the rows and columns ofLrespectively, whileσrowandσcol are the standard deviations ofLcomputed along its rows and columns. Pxy is the element (x, y) normalised to the sum of elements of
L.
Energy: Measure of the uniformity of elements of L. The computed value is in the range of [0, 1], where energy is 1 for a constant image. Energy, E, is defined as: E = |L|−1 X x=1 |L|−1 X y=1 Pxy2 (6.4)
Entropy: Measure of the randomness of the elements of L; the computed value is in the range of [0, 2log2|L|]. Entropy is 0 if allPxy are zero and maximum if allPxy are equal. Entropy,entr, is defined as:
entr=− |L| X x=x+1 |L| X y=y+1 Pxylog2Pxy (6.5)
The average values ofcorr,E and entrwere then computed across the generated co-occurrence matrices (of different P values) to form the three co-occurrence based features.
4. Features generated using a wavelet transform (four features). A sin- gle level 2-D Discrete Wavelet Transform (DWT) was employed to generate the four wavelet based features used. The features were extracted by computing the average of four types of DWT coefficient:
(a) Wϕ(j0, m, n).
(b) WψH(j, m, n). (c) WψV(j, m, n). (d) WD
ψ (j, m, n).
The first wavelet based feature (feature 1) is a scale based DWT; while features 2, 3 and 4 correspond to the wavelet response to intensity variations in three different directions: horizontal, vertical and diagonal respectively. To generate
Wϕ(j0, m, n) and Wψi(j, m, n) (i ∈ {H, V, D}), assume an image f(x, y) of size
row×col. The DWTs were then computed as follows [74]:
Wϕ(j0, m, n) = 1 √ row×col row−1 X x=0 col−1 X y=0 f(x, y)ϕj0,m,n(x, y) (6.6) Wψi(j, m, n) = √ 1 row×col row−1 X x=0 col−1 X y=0 f(x, y)ψij,m,n(x, y) (6.7)
where j = 0,1, . . . , J −1 is the scaling value, m = n = 0,1, . . . ,2j −1 are the translation parameters, ϕj0,m,n and ψ
i
j,m,n are the scaling and translation basis functions respectively. Both functions are defined as [74]:
ϕj0,m,n = 2
j/2ϕ(2jx−m,2jy−n) (6.8)
ψij,m,n= 2j/2ψ(2jx−m,2jy−n) (6.9)
The 2-D scaling, ϕ(x, y), and translation, ψi(x, y), functions were derived from their corresponding 1-D functions as follows [74]:
ϕ(x, y) =ϕ(x)ϕ(y) (6.10)
ψH(x, y) =ψ(x)ϕ(y) (6.11)
ψV(x, y) =ϕ(x)ψ(y) (6.12)
ψD(x, y) =ψ(x)ψ(y) (6.13)
The values ofϕ(t) andψ(t) were determined by the types of wavelet filters used. In this chapter, the most common Haarwavelet filter was employed and defined as: ϕ(t) = 1, 0≤t <1 0, otherwise (6.14) ψ(t) = 1 0≤t <0.5 −1 0.5≤t <1 0 otherwise (6.15)
The overall feature generation process is illustrated by the block diagram presented in Figure 6.3. Each of the pre-processed colour images (see Chapter 4 for details of the image pre-processing method employed in this thesis) was first represented in a 2-D array form. The size of the array is equivalent to the size of the image it represents; each element of the array contains a pixel intensity value. The colour based features were extracted directly from this array. The other categories of feature (histograms, co- occurrence matrix and wavelet) were extracted from the green channel representation of the images. Thus, a 2-D array of the green channel image representation was generated from each image. The colour (green channel) histogram, co-occurrence matrix and wavelet based features were then extracted from this array. The resulting features were kept in a tabular form where each column represents a feature, and each row an image. In the second strategy, where images were partitioned into R sub-regions using a quad-tree image decomposition technique, the features were generated from each sub-region. Note that in the context of the work presented in this chapter, the decom- position of an image will be performed until it reaches a predefined depth, d. Since quad-trees are more suited to square images, the image size was first expanded so that both the height and width of the images were identical. In the context of the work described in this thesis, the dimensions of each retinal image was fixed to 768×768 pix- els. This was achieved by expanding the images with zero valued pixels. The extracted feature vectors were then arranged according to the order of the sub-regions that they represent in an ascending manner, such that the features of the first sub-region formed the first 15 features, the second sub-region formed the next 15 features, while theRth
sub-region formed the last 15 features. Figure 6.4 shows the sub-region ordering of an image using a quad-tree of depth, d = 2. The value of R is thus determined by the value of dsuch thatR= 4d.
6.3
Features Selection
The next step is to reduce the number of extracted features; the aim being to prune the feature space so as to increase the classification efficiency (through removal of redundant or insignificant features) while at the same time maximising the classification accuracy. A feature selection based method to filter the generated statistical parameters was thus applied.
The proposed feature selection method comprises a feature ranking strategy based on the discriminatory power of each feature and selecting the topT. By doing this, only the most appropriate features were selected for the classification task and consequently a better classification result could be produced. The feature ranking mechanism employed used SVM weights to rank features [30]. The main advantage of this approach was its implementational simplicity and effectiveness in identifying relevant features, as demonstrated in [30].
2-D array colour image representation Represent images in 2-D array Colour image dataset Extract colour information features 2-D array green channel image representation Extract green channel
information
Extract colour (green channel) histogram features Extract co- occurrence matrix features Extract wavelet features Colour features Colour histogram features Co- occurrence features Wavelet features Tabular image representation
Figure 6.3: Block diagram of features extraction steps
Algorithm 1: Feature Ranking using SVM Data: Training sets F and c
Result: Sorted features list sortedF
1 C ←GetP arameter(F, c) ; // Find best parameter for SVM
2 listF ←T rainM odel(F, C) ; // Get weight of each feature
3 sortedF ←Sort(listF); 4 return sortedF
0 1
2 3
4
5
6
7
8 9
10 11
12 13
14 15
Figure 6.4: Ordering of sub-regions produced using a quad-tree image decomposition of depth 2
Note that a C-Support Vector Classification (C-SVC) [22, 94] type of SVM was used in the work described in this thesis. C-SVC solves the optimisation problem as presented in equation 3.15 (in Chapter 3). For a training set that is linearly separable, a linear kernel function is used to map the original data into a higher dimensional space as follows [94]:
K(Xi,Xj) =XTi ·Xj (6.16) Algorithm 1 [30] describes the feature ranking process. The input, F, is the set of identified features presented in the form of a feature vector, and the output is a ranked list of the features, sortedF, sorted according to their weights. The algorithm commences with the selection of the best penalty parameter, C, for the SVM (the GetParameter function in line 1). To determine C, a SVM based parameter selection tool [29], a tool that trains the SVM with various parameter settings and performs classification on the training set using n-fold cross validation, was applied to the input setF. A range ofcvalues, such thatC= 2c, andF were supplied to theGetParameter function. The bestcvalue was then identified in the following manner:
1. SplitF into training and test sets.
2. Train the SVM using the suppliedc value on a linear kernel.
3. Perform classifications on the test set and get the results.
4. Repeat from step 2 until allc values are used.
Steps 1 to 3 of the above process were repeatedntimes, thusn-fold cross validation.