Tabular Image Representation

Most work on image classification found in the literature is founded on tabular input where features are extracted directly from either the basic 2-D array image representa- tion [9] or by applying image transformations such as wavelets and the Discrete Cosine Transform (DCT) to the array. In the resulting tables each row typically represents an image and each column some attribute (feature) that exists across the image set including the class label. One example is shown in Figure 3.7 where an image dataset containing M images with b attributes is presented. An alternative interpretation is that each row in the table describes a feature vector, each element of which corresponds to a numerical value associated with one of the identified set of attributes/features. There are various mechanisms whereby each feature can be described, the most com- mon mechanisms use statistical parameters. Different types of statistical parameters that have been used to express features are considered and described in the following sub-section.

0 0 0 0 0 0 0 0 0 0 50 50 0 0 0 0 0 50 80 80 80 50 0 0 0 80 120 120 120 80 50 0 0 80 120 150 120 80 50 0 0 80 120 120 120 80 50 0 0 0 80 80 80 50 0 0 0 0 0 0 0 0 0 0 0 0 0 0 100 100 180 200 0 0 0 0 30 100 180 180 0 0 0 0 0 30 100 100 0 0 0 0 0 0 30 100 0 30 30 0 0 0 0 0 30 100 100 30 0 0 0 0 100 180 100 30 0 0 0 0 180 100 30 0 0 0 0 0 (a) (b)

Figure 3.8: Example of two different 2-D array represented images that have identical global colour mean values

3.5.2.1 Statistical Parameters as Image Features

Considering the two example 2-D array representations, each measuring 8×8 pixels, given in Figure 3.8 there are various numerical values that can be derived to signify colour, texture or even shape information. The most basic information is the global mean colour which can be used to describe the general state of the image colour (for example dark, bright, reddish, etc.). In the case of image retrieval applications this information may be used as an early indicator so as to filter the number of images to be retrieved from the image database. However, for many classification applications using global features is unlikely to give useful results. For example, assume that the 2-D array image representation shown in Figure 3.8(a) and 3.8(b) are labelled as “ball” and “river” respectively. Using the global mean colour description, both images will produce an identical value of 38.6! One possible solution is to consider local features, whereby an image is partitioned into several non-redundant regions, and calculate each region’s mean colour.

From the literature we can identify a number of different mechanisms whereby statistical parameters have been used to define a feature space and the consequent feature vectors associated with individual images. These mechanisms include: (i) the extraction of global features describing an entire image [122], (ii) the extraction of local features from image regions as stated above [9, 11], (iii) the generation of both global and local features [7, 35] and (iv) the identification of interesting objects before generating features from the identified objects [150, 132]. Each is discussed further in the remainder of this sub-section.

An example of image classification using statistical global features can be found in [122], where a system to classify radiography images into more than 80 categories was presented. The proposed approach encoded each image according to its content using Information Retrieval in Medical Images (IRMA) codes comprised of technical,

directional, anatomical and biological indicators. The IRMA code was used to define the categories. Two types of features were used, texture features and scaled image representations extracted from the whole image. The texture features used were the Tamura features [189], fractal dimension, DCT coefficients and edge information. The images were also rescaled intor×rsizes, wherer∈(8,16,24,32), to extract the rescaled features.

In [9] an approach to classifying mammogram images as either normal or abnormal using local features was proposed. Each image was partitioned into sixteen regions. Then, for each region, texture features were extracted (namely mean, variance, skewness and kurtosis) and combined with two additional features (the type of tissue and the position of the breast) from the original database. These features were then combined into a table and neural networks and association rule mining applied to generate the desired classifier. Experiments demonstrated that good classification results could be obtained using this representation. In [11] the authors applied a geometrical image decomposition to partition images into different geometrical regions. Two types of features were used, colour and texture. The texture features used were computed from image wavelet coefficients, and used to classify textured images. The colour features were employed to classify non-textured images.

The use of combinations of both global and local features was presented in [7, 35]. In [7] the global features (such as variance, skewness and kurtosis) were extracted from the whole image while the local features (energy, entropy, contrast, homogeneity and corre- lation) were generated from an image co-occurrence matrix. In [35], colour histograms and a measure that describes the edge information of local textures (which can also be applied on the whole image), named Local Edge Pattern (LEP) histograms, were used. To extract local features, each image was partitioned into regions using a splitting and merging mechanism. Both colour and LEP histograms were then generated from each region. One of the proposed solutions promoted in this thesis utilises global and local features for image classification.

Approaches that applied image segmentation prior to feature extraction were de- scribed in [132, 144, 150]. In [132], the object of interest (potato chip) was first identified before the colour and texture features were extracted. Features based on co-occurrence matrices [85] were used to define the image textures. In [144], the shape information of the identified objects was used as features. Seven discriminative features were se- lected and extracted from the identified objects in each image, these included area ratio, perimeter-area ratio, eccentricity and the invariant moment. In [150], the ob- jects of interest (identified by image segmentation) were resized and normalised. The feature extraction consisted of three steps: (i) wavelet transformation to emphasise the shape of the object, (ii) conversion from the RGB colour model to the HSI colour model and (iii) extraction of texture features (such as contrast, diagonal moment and

energy) using the “I” component of the HSI colour model and a sliding windows mech- anism. Using this approach 49 different texture features were used to describe images. The evaluation reported in [150] indicated that good results were produced when the technique was applied to various types of images, particularly by the diagonal moment feature. However, difficulties were found using images (of different classes) that have similar shape and texture, and also when complex shapes and textures occur.

A comparison of various features, using mostly statistical measures, for image re- trieval and classification can be found in [44]. Note that all of the work described above used statistical parameters as feature vectors. These feature vectors were then represented in a tabular form as shown in Figure 3.7. The proposed solution described in Chapter 6 of this thesis employs statistical parameters as feature vectors for image classification.