As mentioned in Chap. 2, decision trees can be simplified in two ways namely, pre-pruning and post-pruning. In the former way, the pruning action aims to stop a particular branch in a tree growing further. In the latter way, the pruning action aims to simplify each of the branches in a tree after the whole tree has been generated. In particular, the tree needs to be converted into a set of if-then rules before the pruning action is taken. The rest of this section introduces a particular type of tree pruning based on J-measure in the manner of both pre-pruning and post-pruning.
For pre-pruning of decision trees, it is unknown which class is eventually labelled on the leaf node of a particular branch. Therefore, the calculation for the value of J-measure needs to follow the Eqs. (2.3) and (2.6) shown in Sect.2.6. The corresponding upper bound of J-measure, which is referred to as Jmax, is calculated following the Eqs. (2.3) and (2.7) shown in Sect.2.6.
In the above context, J-measure based tree pre-pruning works in the procedures as illustrated in Fig.4.1.
©Springer International Publishing Switzerland 2016 H. Liu et al.,Rule Based Systems for Big Data,
Studies in Big Data 13, DOI 10.1007/978-3-319-23696-4_4
The stopping criteria mentioned in Fig.4.1can be in accordance with value of J-measure and Jmax. In particular, the stopping criteria may be met when the highest value of J-measure observed so far is already higher than the Jmax value achieved at the current node. In addition, the stopping criteria may also be met when the value of J-measure is equal to the Jmax value at the current node. However, the above J-measure based pre-pruning method may not be able to make an optimal tree so that each of the branches in the tree achieves the highest value of J-measure. In other words, the highest value of J-measure may be achieved in a node which is a precursor of the node at which the corresponding branch is stopped growing. Due to the possible presence of this issue, the above pre-pruning method needs to be evolved to a hybrid pruning method by means of combination of pre-pruning and post-pruning strategies. In particular, each branch is forced to stop growing when it is necessary. After the completion of the whole tree generation, each of the branches needs to be converted into a single if-then rule for post-pruning to have this rule achieve the highest value of J-measure. Section4.3will illustrate this in detail using a specific example.
On the other hand, decision trees can also be simplified purely by post-pruning each of its branches. The J-measure based post-pruning is illustrated in Fig.4.2.
In particular, there are two specific methods of J-measure based post pruning. Thefirst method is inspired by SQUEEZE [1] which is illustrated in Fig.4.3.
The second method is inspired by ITRULE [2] and is illustrated in Fig.4.4. The above pruning strategy involves a recursive process for searching for the global maximum of J-measure in order to get the rule with the optimized ante- cedent. Section4.3will illustrate the two methods introduced above in detail using specific examples.
Input: a tree node n, J-measure of a node J(n), Jmax of a node Jmax(n), Maximum of J- measure max(J), class C
Output: a decision tree T
Initialize: max(J)= 0;
foreach branch do
calculate J(n)and Jmax(n)
if J(n)> max(J)then
max(J)= J(n)
end if
if stopping criteria is metThen
stop this branch growing, label the current node nwith the most common class C
end if end for
Fig. 4.1 J-measure based tree pre-pruning
Input: a set of rules rule[i], where i is the index of rule, values of J-measure J[i][j][k], where j is the order of a particular child rule and k is the index of a child rule in the order of j, highest value of J-measure (for rule[i]) max[i], order for the optimal rule m,index for the optimal rule n[i]
Output: a set of simplified rules
foreach rule rule[i]do
max[i]=0, m=0, n[i]=0
for each order j
foreach child rule k
calculate J[i][j][k]
if J[i][j][k]> max[i]then
max[i]= J[i][j][k] m=j
n[i]=k
end if
else ifJ[i][j][k]= max[i]
take the rule with a higher order
end else if end for end for end for
Fig. 4.2 J-measure based tree post-pruning
For each single rule
1. Calculate the J-measure for the rule which is referred to as the parent rule.
2. For each of the child rules generated by removing a single rule term (at- tribute-value pair) from the parent rule, calculate the J-measure (if the parent rule is order K, each of the child rules is order K-1)
3. Choose the rule among the parent rule and the set of its child rules with highest J-measure. Special cases:
a) If two rules have the same J-measure, choose the one with the lower order.
b) If two rules of the same order have the same J-measure, ran- domly choose one of the two.
4. If the chosen rule is not the parent rule, the chosen rule becomes a new parent rule; repeat steps 2-4. If the chosen rule is the parent rule, termi- nate the process.