Maximum Relevance, Minimum Redundancy Technique

Peng et. al [94], and it find the feature which has maximal relevance and minimal redun- dancy. Maximal relevance is a process of finding a feature with the highest relevance to the target class, while minimal redundancy find a feature that maximize the joint depen- dency of features on the target class to reduce redundancy among features. mRMR uses a heuristic search algorithm to satisfy both maximal relevance and minimal redundancy. Mathematically, relevance and redundancy of feature subset is defined as follows:

V (i, h) = 1 |F | X i∈F I(i, h) (6.3) W (i) = 1 |F |2 X i,j∈F I(i, j) (6.4)

where Fm _{is a feature subset, h is the target class, i, j is features, and I(·) is the mu-} tual information function. V and W are the relevance and redundancy of feature set, respectively. The best feature subset, is found using greedy search algorithm to satisfy the following criteria:

max Φ(V, W ), Φ = V − W (6.5)

6.3.4

Experimental Design

In this experiment, three features selection methods were validated: mRMR, ReliefF and RF-VI. The experimental setup is exactly the same as the one used in classification analysis. Each method runs through the training dataset and ranks each predictor variable from the most influence one to the least. Then, random forest classification was employed on the subset of dermal radiomics sequences, and the number of feature used increased from 5 to 385 with a step of 10. To measure the accuracy of the testing feature selection algorithms, LOO cross validation and 50 trials of 80/20 cross validation were conducted.

Figure 6.4: The sensitivity results of Leave-one-out cross validation using mRMR, ReliefF and RF-VI as feature selection algorithms

Figure 6.5: The specificity results of Leave-one-out cross validation using mRMR, ReliefF and RF-VI as feature selection algorithms

Figure 6.6: The accuracy results of Leave-one-out cross validation using mRMR, ReliefF and RF-VI as feature selection algorithms

6.3.5

Results

Fig. 6.4, 6.5, and 6.6 shows the sensitivity, specificity, and accuracy of the random forest classification, which is coupled with feature selection algorithms. As expected, the use of feature selection algorithm improved the overall results. For example, The use of ReliefF improved its accuracy by 6.8%, compared to the accuracy when no feature selection method was used. Moreover, when feature selection algorithm was loosely applied to use more number of features, the results between three feature selection methods did not vary much. However, the performance of each feature selection was more visible when less than 80 features were chosen for the classification. While ReliefF outperformed over other two methods, mRMR showed the worst performance of all.

Table.6.4shows the quantitative results of the sensitivity, specificity and accuracy from each feature selection model. The results were chosen by the best accuracy with the number of features used. In 80/20 cross validation, ReliefF showed the excellent performance on ranking features. The number of features for the best results is between 35 and 45, which is about 10% of the entire dermal radiomics sequence. mRMR showed the worst performance

of all three testing feature selection algorithms. The number of features used to yield the best accuracy is 145, and mRMR did not work well on optimizng the DRS when compared to ReliefF algorithm.

Table 6.4: 50 trials of 80/20 cross validation results for melanoma detection: Compar- ing feature selection models (ReliefF, RF-VI and mRMR) on sensitivity, specificity, and accuracy(%). Results are shown with 95% confidence interval.

Feature selection method

Number of

features Sensitivity Specificity Accuracy ReliefF 35 88.8 [87.2 90.4] 77.9 [74.9 80.8] 84.1 [82.7 85.5]

RF-VI 75 85.8 [83.8 87.8] 72.9 [69.8 76.0] 80.2 [78.7 81.7] mRMR 170 83.3 [81.4 85.3] 71.7 [68.9 74.5] 78.4 [76.7 80.1]

Another classification metric was used to examine the performance of feature selection aglorithm. A receiver operating characteristic (ROC) curve was computed in Fig.6.7. The results from ROC curve also agree with our findings above as area under curve (AUC) for ROC is 0.91, 0.88, and 0.85 for ReliefF, RF-VI, and mRMR, respectively.

Lastly, the ranked features are examined to see how each feature are contributed to overall classification results. Among 35 highly ranked features by ReliefF feature selection algorithm, 9 features are from low level feature set. More specifically, four of nine describes asymmetry, three and two features describe color variation and differential structure of lesion, respectively. Two features, which describes irregularity of lesion, are from high level intuitive feature set. Moreover, mean and variance of eumelanin and hemoglobin are ranked as the most important features from physiological feature set. Lastly, a total of 20 features are highly ranked from physiological texture feature set for classification. Nine, six, and five features are derived from eumelanin, Pheomelanin, and hemoglobin, respectively.

Given that the majority of dermal radiomics sequence is made up by physiological texture feature set, a 35 highly ranked features have well-balanced mixture of each fea- ture set: 11 features are obtained from LLF and HLIF, and 24 are from PT and PTF. It reflects that the proposed dermal radiomics, which is composed of the feature set based

Figure 6.7: ROC for different feature selection methods

on both ABCD-rule and physiological biomarkers, improves the accuracy of melanoma diagnosis, compared to the feature sets only based on ABCD-rule. Moreover, no domi- nant physiological biomarker was observed as eumelanin, pheomelanin, and hemoglobin equally contributed to the highly ranked features, which concludes that the selection of physiological biomarkers was appropriate for the dermal radiomics.

In this section, a validation study was constructed to assess different feature selection algorithms on dermal radiomics sequence. Among ReliefF, RF-VI, and mRMR, ReliefF showed the superior performance on ranking dermal radiomics sequence to yield an excel- lent classification result. Feature analysis tuned the dermal radiomics sequence by using ReliefF to pick top 10% of its entire feature set and ultimately, the accuracy was improved by as high as 6.8%, compared to when no feature selection model was employed. While two other feature selection algorithm performed as good as ReliefF when a large number of features used , which is more than 80. However, the classification metrics were decreased when RF-VI and mRMR ranked fewer number of features for classification, which implies that they failed to rank features effectively.