Classifiers

In any pattern classification problem, the choice of classifier is one of the critical aspects. The other important factor is feature selection. The success of a pat- tern recognition algorithm depends upon feature and classifier selection. In the previous chapter, we have shown that the features extracted using ADWPT and GLCM can produce good clusters. Rand indices as high as 0.82 were obtained for some of the features. This indicates that the features obtained are capable of differentiating between the various textures under study. Since our aim is classi- fication, we must select an appropriate classifier. In this section, we discuss the choice of classifiers and present the results obtained with the one that produces the best results.

Over the last few decades, great progress has been made in the domain of classifiers. Many new classifiers have been invented. There are various types of classifiers. Mazhelis in [158] presents an overview of various types of classifiers and an analysis of the strengths and weaknesses of the classifiers in relation to a problem.

There are various types of binary-class and multi-class classifiers in use. A binary class classifier principally says whether a sample belongs to a class or not

5.1 Classification

i.e. it differentiates between two classes. Many binary-class classifiers can be combined to produce a multi-class classifier. A multi-class classifier as the name suggests classifies a test sample amongst a number of classes at the same time. In the process of classification, normally a process of training is involved with a number of training samples and subsequently test samples are used to test the classification accuracy of a classifier.

Types of Classifiers

The various classifiers whether one-class or multi-class can be categorised into one of the following based upon the internal model used by the classifier:

Density Methods

k-Nearest Neighbour (k-NN) is an example of a density based classifier. Other examples of such classifiers are Gaussian or Gaussian mixture based classifiers. In density methods an estimate of the probability density function of the feature values in the training samples is estimated. The training data is assumed to be representative of the texture class true data distributions. If a distribution of certain features is unknown then it is assumed to be Gaussian or approximated using techniques such as mixtures of Gaussians.

We have performed some analysis with k-Nearest Neighbour (k-NN) and the results were presented in [52]. A leave-one-out approach was used for test-trial runs while the classification accuracies produced were not very high. However, the k-NN results showed that GLCM features combined with ADWPT can be used to effectively differentiate between meningioma subtypes. ADWPT based GLCM features were found to be better than raw image based GLCM features for meningioma subtype classification.

Reconstruction Methods

In the reconstruction method, an underlying model of the data structure or fea- ture space is estimated. The model is learnt. The parameters of the model are

5.1 Classification

computed in the training phase and the sampling error is estimated at each stage. The training is stopped when an acceptable error-rate is attained. Examples of such a classifier are neural networks such as Self Organizing Maps (SOMs) and Learning Vector Quantization (LVQ) [159].

Lessmann et al. [67] in their work presented the results of using SOMs for clustering of the meningioma subtypes under study in this thesis. We used LVQs for the purpose of classifying meningioma subtypes and presented the results in [140]. Relatively good classification accuracies were obtained although GLCM features were not used. Though transitional and fibroblastic subtypes were not adequately classified with classification accuracies as low as 50%.

Boundary Methods

The boundary method stipulates the estimation of the distance between test fea- ture vectors and the boundary built around the training feature vectors. The test samples are classified based upon the minimum distance to a boundary represent- ing a class. The boundary construction is obtained during the training phase. Boundary methods are particularly aimed for one-class classification. Support Vector Machines (SVMs) are an example of boundary method based classifiers. Our analysis, in the subsequent section, is carried out with an SVM as it is one of the most powerful and versatile classifiers in use [158]. We will present a detailed description of SVMs and how they can be used to obtain a multi-class classifier, since, it is a one-class classifier.

Classification with various kinds of classifiers was carried out to provide an idea as to how various methods of classification function for our problem. This analysis was by no means aimed at presenting a comprehensive and exhaustive analysis of classifiers that may be used for classifying the various meningioma subtypes. The aim of this thesis is to attain a set of feature vectors that are good represen- tations of textural patterns under study as the quality of features is extremely important. A weak classifier with good and optimal feature set can obtain good classification accuracies but a sophisticated and powerful classifier would fail if the quality of the feature set is substandard.

5.1 Classification

5.1.2

Support Vector Machines (SVMs) for Multi-Class