Interpretability evaluation in fuzzy rule-based systems

Interpretability evaluation is an important step that aims to compare between different fuzzy rule-based systems in order to choose the most interpretable one. Since interpretability constraints define the characteristics of interpretable fuzzy rule-based systems, they have been used to assess the interpretability by verifying to what degree these constraints are valid for a given system (Mencar, Castiello, Cannone, & Fanelli, 2011). Some approaches for interpretability evaluation were used to assess the semantic-based constraints (low-level constraints), which are related to the fuzzy sets and fuzzy partition. In many other cases, semantic-based interpretability is limited to the evaluation of the distinguishability constraint using different methods such as similarity measures (Roubos & Setnes, 2001; Setnes et al., 1998; Setnes & Roubos, 2000), Pointwise property approach (De Oliveira, 1999), Possibility measure (Mencar et

University

65

al., 2007), etc. One disadvantage of semantic constraints-based evaluation is the lack of a general and widely accepted way or measure to evaluate the interpretability semantic- constraints.

Complexity-based approach is the commonly used method for interpretability evaluation; its advantage is the use of widely accepted measures that are usually used to assess the complexity of the systems. This approach can be useful especially when the semantic constraints are fully satisfied by the system. For example, (Ishibuchi, Yamamoto, et al., 2005) used the terms of number of rules, total rule length and average rule length to measure the interpretability. They ignore the semantic evaluation as they assume that the semantic constraints are highly fulfilled because they produce rules with pre-defined linguistic terms with clear semantic meaning. In another study, (Marquez, Marquez, & Peregrin, 2010) employed the three metrics to measure the interpretability: (1) the total number of rules, (2) the number of rules with weight associated, and (3) the average number of firing rules. Even though complexity-related measures are easy for calculation and widely accepted, they may not be always suitable for comparison between different fuzzy rule-based systems because they evaluate the interpretability with more than one criterion (for example number of rules, number of antecedents, etc.) which makes, in some cases, finding the most interpretable fuzzy rule-based system more difficult.

The main drawback of either selecting semantic-based constraints or complexity-based constraints separately is the need to assess the overall interpretability of the fuzzy rule- based system for comparison purpose and the need to combine all semantic and complexity-based constraints into one interpretability index like the case of classification accuracy (Zhou & Gan, 2008).

University

66

Table 3.1 list of proposed methods and the constraints used to preserve the interpretability of fuzzy rule-based system

References

Interpretability constraints

Complexity-based constraints Semantic-based constraints

MOEA FS Rule reduction Fuzzy sets reduction CG Fuzzy partition Fuzzy rules Coin Pred RS RM FSet.S FSet.M Dis Cov Cons

Ishibuchi et al. (1995) Ishibuchi et al. (1997) De Oliveira (1999) Ishibuchi et al. (2001)

Cordon, Del Jesus, Herrera, Magdalena, and Villar (2003) Nauck (2003) Peña-Reyes and Sipper (2003) Tikk et al. (2003) S. Guillaume and Charnomordic (2004) Ishibuchi and Yamamoto (2004) Jorge Casillas et al. (2005)

Narukawa et al. (2005)

Mikut et al. (2005) Ishibuchi and Nojima (2007)

University

67

References

Interpretability constraints

Complexity-based constraints Semantic-based constraints

MOEA FS Rule reduction Fuzzy sets reduction CG Fuzzy partition Fuzzy rules Coin Pred F. Liu, Quek, and Ng

(2007)

Mencar et al. (2007) José M Alonso et al. (2008) Pulkkinen and Koivisto (2008) Pulkkinen and Koivisto (2008) J. M. Alonso, Magdalena, and González-Rodríguez (2009) Pulkkinen and Koivisto (2008) J. M. Alonso et al. (2010) J. M. Alonso and Magdalena (2011a) Gacto et al. (2010) Mencar et al. (2011) (Ishibuchi & Nojima, 2013)

(M. Antonelli et al., 2014)

Note: MOEA: Multi-Objective Evolutionary Algorithms, FS: Feature selection, RS: Rule selection, RM: Rule merging, FSet.S: Fuzzy set selection, FSet.M: Fuzzy set merging, CG: controlling the granularity, Dis: Distinguishability, Cov: Coverage, Cons: consistency, Coin: Cointension, Pred:

Predefined fuzzy sets

University

68

Interpretability index

Nauck (2003) is the first to introduce the index by combining the complexity of a classifier, the number of labels (linguistic terms) and the coverage degree of the fuzzy partition. The Interpretability index is the product of the three following terms:

Comp: it represents the complexity of the fuzzy rule-based system and it is calculated

as the number of classes divided by the total number of antecedent conditions.

Conv: denotes the average normalized coverage and it measures the degree of coverage

provided by the fuzzy partition.

Part: it denotes the average normalized partition index for all the input variables used in

the system. This index is used to penalize partitions with a high granularity.

This index, which is known as Nauck’s Index, is further improved by José M Alonso et al. (2008), in which they proposed a fuzzy Index instead of numerical index to measure the interpretability using six main inputs and one output, which is the interpretability. The inputs are: (1) the number of rules, (2) total number of antecedent conditions, (3) number of rules which use one input, (4) number of rules which use two inputs, (5) number of rules which use three or more inputs, and (6) total number of labels defined by input. The inputs are grouped into four linked knowledge bases to form a hierarchical fuzzy rule-based system. The output of the system, which is the interpretability index, composes of five linguistic labels: very low, low, medium, high, and very high. In addition, the authors assumed that fuzzy rule-based systems evaluated include only SFPs so that all the semantic-based constraints are satisfied to the highest level.

In J. M. Alonso et al. (2009), an experimental analysis in the form of a web poll was carried out to evaluate the most used indices for interpretability evaluation. The five indices are: the number of rules (NOR), total rule length (TRL) or the total number of antecedent conditions, average rule length (ARL) or the number of antecedent

University

69

conditions per rule, Nauck’s index (Nauck, 2003) and fuzzy index (José M Alonso et al., 2008). The results showed that there are some significant differences between naïve and expert users and subjectivity in the assessment of the indexes which suggests the need to define a new flexible index that can be easily adapted to the problem and the user preferences. In another study, J. M. Alonso and Magdalena (2010) proposed an index adaptable to the context of each problem by incorporating the user’s preferences in the interpretability of the evaluation of a fuzzy rule-based system.