The Proposed Method

The proposed method integrates the weighted k-nearest neighbours algorithm and

propagating the classification accuracy to a certain threshold. The k-NN is to find the

closest neighbours (𝑛1, … , 𝑛𝑘) for a certain instance (𝑥𝑖), that contains missing

feature values, using the Euclidean, Manhattan, Minkowski, and other distance

96 Minkowski. Then the proposed approach is to find the most similar instance to (𝑥_𝑖)

from (𝑛₁, … , 𝑛_𝑘) using formula number 4 by finding the distances values (𝑐𝑛_𝑖).

𝑐𝑛_𝑖 = 𝑛𝑖𝑗 𝑑 𝑥_𝑗, 𝑛_𝑗 𝑘 𝑗 =1 1 𝑑 𝑥_𝑗, 𝑛_𝑗 𝑘 𝑗 =1 (25)

Where 𝑐𝑛𝑖 donate to the closest neighbours to the instance 𝑥𝑖,𝑑 𝑥𝑗, 𝑛𝑗 is the distance

between the instance 𝑥_𝑗 and the neighbour𝑛_𝑗 , and 𝑛_𝑖𝑗 is the feature 𝑖of the

neighbour𝑛𝑗. After finding the closest neighbour (the smallest value of 𝑐𝑛𝑖) call it

𝑐𝑛′, the missing feature values in 𝑥𝑖 will be filled by the equivalent features values in

𝑛_𝑖 which have 𝑐𝑛′ distance to 𝑥_𝑖. The process of filling missing features values will

produce a new training dataset (NT) that contains no missing features values. To

verify the accurateness of the constructed missing features values, the new training

dataset is applied to k-NN and record the accuracy. If classification accuracy is less

than a threshold then the algorithm will step back to fill the missing features values

until the desired classification accuracy is reached. Figure 26 shows the flowchart for

97 Figure 26: The Flowchart for the proposed method (Constructing Missing Features Values)

98

5.4

The Experimental Results

In data mining and statistical researches, the split sample approach is a commonly

used study design in studies that contain large dataset. This design divides the dataset

into a training set and a testing set to approximate classification accuracy. The

classifier is designed and developed based on the theory and then trained using the

training dataset. After tainting the classifier, it is applied to each case in the testing

sample. In practice, dividing data is important to avoid large bias in estimation the

classifier accuracy [99]. Therefore, the dataset (WBC) has been divided into two

parts, training dataset and testing dataset. The dataset separation was random to

avoid unfairness. The training dataset contains 500 cases 16 of them contain missing

features values. The rest of the original dataset (WBC) reformed the testing cases

(199 cases). After preparing the datasets, a classifier has been developed using the

proposed method. The development tool was Microsoft Visual Studio 2010. The

selected language was C# programming language.

In the implementation, this work has used many metrics to compute the distance

including the Euclidean and Minkowski functions as a component toward the

iterative k-NN classifier. Constructing the missing features values using the proposed

method through iterative k-NN classifier with the Euclidean distance function

showed a classification accuracy enhancement of 0.005 when k=3 from the first

iteration and a maximum classification accuracy of 0.9648 . Figure 27 shows a

comparison of classification accuracy when the missing features values were not

treated and when treated. The figure also shows different classification accuracy

99 more neighbours reduce the classification accuracy. The reason may due to noise as a

result of more neighbours.

Figure 27: A comparison of classification accuracy for the proposed method through Euclidean/k-NN

The experiment of constructing the missing feature values using the proposed

method through k-NN classifier with the Minkowski distance function showed a

classification accuracy enhancement of 0.005 when k=3 and r=1.5 from the first

iteration and a maximum classification accuracy of 0.9698 . Figure 28 shows a

comparison of classification accuracy when the missing features values were not

100 The experiment also showed that Manhattan, Chebychev, and Canberra distance

metrics are not suitable for constructing the missing attributes values, in this

experiment, because the classification accuracy after treating the missing values

remain lower than the classification accuracy for the original dataset.

Figure 28: A comparison of classification accuracy for the proposed method through Minkoski/k-NN

5.5

Summary and Discussion

This chapter did propose a new approach for constructing missing features values

101 an iterative approach until finding the most suitable features values that satisfy

classification accuracy. The proposed approach showed improvement of 0.005 of

classification accuracy on the constructed dataset than the original dataset on both

Euclidean and Minkowski distance functions. Manhattan, Chebychev, and Canberra

distance metrics produced lower classification accuracy on the new dataset than the

original dataset. This work also noticed that classification accuracy depend greatly

on the number of neighbours (k). The experiment showed that less neighbours may

lead to more accuracy. The reason for that, in my opinion, is the amount of nose

produced from conflict neighbours. Finally, the maximum classification accuracy

was on k=1 which was 0.9698.

The next chapter will describe a new approach for diagnosing breast cancer. The

chapter present a comparison study between some well-known feature selection

102

CHAPTER SIX

Diagnosing Breast Cancer Based on Feature

Selection and Naïve Bayes

6.1

Overview

Feature selection techniques have become an obvious need for researchers in

computer science and many fields of science. Whether the target research is in

medicine, agriculture, business, or industry; the necessity for analysing large amount

of data is needed. Addition to that, finding the most excellent feature selection

technique that best satisfy a certain learning algorithm could bring the benefit for the

research and researchers. Therefore, a new method has been proposed for diagnosing

breast cancer on a combination of learning algorithm tools and features selections

techniques. The idea is to obtain a hybrid approach that combines between the best

performing learning algorithms and the best performing features selections

techniques. The experiment result shows that assemblage between correlation based

features selections method along with Naïve Bayes learning algorithm can produce a

promising results. However, no single feature selection methods that best satisfy all

datasets and learning algorithms. Therefore, machine learning researchers should

understand the nature of datasets and learning algorithms characteristics in order to

obtain better outcomes. Overall, consistency based subset evaluation performed

better than information gain, symmetrical uncertainty, relief, correlation based

103

6.2

Feature Selection Techniques

The advancement of information technology, the growing number of social networks

websites, electronic health information systems, and other factors have flooded

internet with data. The amount of data posted daily on internet is increasing daily. At

the same time, not all data are important or even needed. Therefore, data mining

researches started using the term features selections or data selections more often.

Feature selection or attribute subset combination is the process of identifying and

utilizing the most relevant attributes and removing as many redundant and irrelevant

attributes as possible [60, 61]. In addition, features selections mechanisms do not

alter the original representation of data in any way. It just selects an optimal useful

subset. Recently, the inspiration for applying features selection techniques in

machine learning has shifted from theoretical approach to one of steps in model

building. Many attribute selection methods use the task as a search problem, where

each result in the search space groups a distinct subset of the possible attributes

[100]. Since the space is exponential in the number of attributes which produce lots

of possible subsets, this requires the use of a heuristic search procedure for all data

sets. The search procedure is combined with an attribute utility estimator in order to

evaluate the relative merit of alternative subsets of attributes [60]. This large number

of possible subsets and the computation cost involved necessitate researchers to

conduct a benchmark feature selection methods that produce the best possible subset

in regards to more accurate results as well as low computation overhead.

Feature selection techniques could perform better if the researcher chooses the right

learning algorithm. Therefore, a new approach proposed which combines a promising

feature selection technique and one of well-known learning algorithm. In the current

104 evaluate the proposed approach. Yearly around the world, millions of ladies suffer

from breast cancer, making it the second common non-skin cancer after lung cancer,

and the fifth cause of death among cancer diseases in the world [7]. Thyroid disorder

in women is much more common than thyroid problems in men and may lead to

thyroid cancer [101]. Hepatitis can be caused by chemicals, drugs, drinking too much

alcohol, or by different kinds of viruses and may lead to liver problems [102]. This

paper begins with brief related work, then a description of benchmark feature

selection methods, a description of our methodology in the current paper, and the

results obtained by using the three datasets. Finally, a brief discussion and future work

are presented.