DTs Performance Measures

Once a new machine learning algorithm has been developed, it is important to find out how effective is the model based on metrics and datasets. Performance measures (or evaluation measures) play important roles in machine learning. These measures not only serve as the criteria to assess learning algorithms but also play the role of heuristics to develop learning models. The most widely used performance criterion for DTs are statistical measures like accuracy and error rate; however, the most appropriate DT for a certain classification problem can be selected by considering other significant measures that have a significant impact on the performance of DT. These include DT comprehensibility and stability of predictions (Osei-

22 Bryson 2004). The most widely used performance criterion for assessing machine learning algorithms are presented in the following sub-sections.

2.2.2.1 Statistical Measurements

The literature of machine learning shows that various statistical performance tools were used to statistically analyze the efficiency of machine learning algorithms. The most widely used methods for this purpose are an Error rate, Accuracy Rate, Precision, Recall and F1-Score etc. A brief explanation of these evaluation metrics and how they are calculated are given in the following short paragraphs (Fawcett 2006; Powers 2011; Alvarez 2002; Davis & Goadrich 2006).

Accuracy Rate: It refers to the ratio that is attained by dividing the number of accurate estimations with the overall number of estimations.

Accuracy =_TP+TN+FP+FNTP+TN ‎2.8

Error Rate: It is the ratio of all estimations that are not correct. It basically measures how inaccurate a model is.

Error =_FP+FN+TP+TNFP+FN ‎2.9

Precision Rate: It refers to the number of positive predictions divided by the overall number of positive class values estimated. That is, it is the ratio of True Positives to the overall number of True Positives as well as the number of False Positives. This is also referred to as the Positive Predictive Value (PPV).

Precision (PPV) = _TP+FPTP ‎2.10

Recall Rate: It is the ratio of True Positives to the number of both True Positives and False Negatives. That is, it is the ratio of the positive estimations to the number of positive class values in the testing data. This ratio is also referred to as the Sensitivity or the True Positive Rate.

Recall =_TP+FNTP ‎2.11

F1-Score Rate: This ratio is also referred to as the F-Measure and demonstrates the balance between the Precision and the Recall.

23

F1 − Score = 2 ∗_{Precision+Recall}Precision∗Recall ‎2.12

2.2.2.2 DT Comprehensibility

Decision trees are usually considered to be the most precise and efficient classification methods, however, at times, DTs, particularly the bigger ones, are difficult to comprehend and explain. Therefore, they become incomprehensible to experts (J.R. Quinlan 1987), so it is vital to make sure that DTs are as easy to understand so that they can be interpreted even by non- professionals. The complexity of DT is normally measured using one of the metrics given below (Olivas 2007):

 The numbers of nodes that create and construct the tree.

 The number of leaves that are created by the tree

 The tree depth, i.e. the length of the largest path, from a root to the leaf, and a total number of attributes considered.

Various techniques have been presented in the last few years to simplify trees. These include pruning methods that are possibly the methods used most extensively to decrease the size of DTs. In machine learning, pruning is the process of removing non-predictive subtrees (branches) of a model so that its accuracy can be increased, size can be decreased, and the issue of overfitting can be avoided (Patel & Upadhyay 2012). Pruning is of two kinds, pre- pruning and post-pruning (Fürnkranz 1997). Pre-pruning is also known as forward pruning, which prevents the growth of trees early on, at the beginning of the process of constructing the DTs, so that the generation of unimportant branches can be avoided. Post-pruning is also known as backward pruning. In this method, the tree is first constructed and then, the unimportant branches of the decision trees are reduced.

Another technique that is used to simplify decision trees includes decreasing the size of the original learning set (referred to as data reduction techniques). For this, the unimportant features are eliminated before the tree induction process (Sebban et al. 2000).

24

2.2.2.3 Stability of the Results

Another important factor to consider when creating and assessing the efficiency of DTs is the stability of the DT prediction findings. The criterion of stability performance is important with respect to the quality of DT, as there should not be much difference in the predictive accuracy rate when a DT is used on various validation data sets (Osei-Bryson 2004). One of the most widely used methods by the developers is the k-fold cross-validation estimator (particularly when there is a limited amount of data) which helps to make sure that the outcomes attained by the various DTs are constant and also help in providing an optimal decrease in variance (Kale et al. 2011). More details on K-fold cross-validation is given before in section 2.1.3.