Improvement of Interpretability

As one can see from Figure 5.3, a model simplification step is added to PAIA2. The aim is to remove the redundancy both in the rules and in the fuzzy sets so that one can achieve the FRBS structure optimisation along with the accuracy at the same time. There are five steps involved in the model simplification module, which are discussed in the following Sections. The effects of the thresholds introduced in Sections 5.5.4.1~5.5.4.5 will also be analysed in Section 5.6.2.

5.5.4.1 Removing Unimportant Rules

Inspired by the idea behind neural network pruning, the unimportant rules are those rules that contribute the least to any prediction error increase when not including this rule, as described by Eq. 5.21. This occurs because other rules may already have covered the input region under these rules.

min | ̃|      1, … , (5.21)

where, is the root mean square error when all the rules in the rule base are used for predicting; _̃ is the predictive error associated with the rule base when the th rule is temporarily excluded. Insignificant rules are deleted when the following condition is met:

· (5.22)

where, is the number of rules in the current FRBS; is the maximum allowable number of rules, which equals the number of clusters used in the first modelling stage; rnd is a random number between [0, 1]. is a design parameter which limits the fewest rules in FRBS (in other words, the maximum rules that can be regarded as the insignificant rules) and has been set to 0.5 in this work without any loss of generality. At each iteration step, each cloned individual has one insignificant rule removed unless the rule base reaches the fewest rules designated by Eq. 5.22.

- 135 -

5.5.4.2 Removing Singleton Rules

Singleton rules are those rules which include fuzzy sets that are similar to the singleton set. Such rules should be removed because they may not be fired in most cases and may not be desirable for the generation of an interpretable rule-base (Wang et al., 2005). These may be deleted subject to the following condition:

∑ _ · (5.23)

where, n is the input dimension; _ is a design parameter which randomly changes between [0, ] every t iterations and is 0.01 in the following experiments without any loss of generality. At each iteration step, one singleton rule is removed for each cloned individual given condition 5.23 is met.

5.5.4.3 Merging Similar Rules

During the simplification and the optimisation operations, rules may have similar fuzzy sets in the antecedent part. These rules should be merged together by taking the mean values of those fuzzy sets to keep the FRBS consistent and parsimonious. To measure the similarity of rules, the so-called similarity of rule premise (SRP) (Jin et al., 1999) is used in this thesis. The following condition should be met for merging a pair of similar rules of each cloned individual at each iteration step:

, , , _{, 1, … , ;}1, … , _ · 1 (5.24)

where, , are the similarity between two fuzzy sets and will be explained in Section 5.5.4.5; _ is the threshold which randomly changes between [ , 1] every t (specified by the user) iterations and is 0.95 in this work without any loss of generality.

The above three operations (see Sections 5.5.4.1~5.5.4.3) are applied to the rule level as visualised by Figure 5.12.

- 136 -

Figure 5.12 The example used in Section 4.3.1.1 with two inputs: (1) R1 and R5 are similar

rules; (2) R6 is the singleton rule; (3) R7 is the insignificant rule.

5.5.4.4 Removing Universal Fuzzy Sets

Fuzzy sets which meet the following condition are regarded as universal fuzzy sets and are therefore deleted:

, _ · 1 (5.25)

where, is the universal fuzzy set; _ is the threshold which randomly changes between [ , 1] every generations and is 0.85 in this work. For computation purpose, if the width of a fuzzy set is more than two times wider than the universe of discourse of the corresponding dimension, it is regarded as the universal fuzzy set. Figure 5.13 illustrates such a case, where the centre of the fuzzy set is 0.5 and the spread is 2 on the universe of discourse: [0, 1].

- 137 -

Figure 5.13 A fuzzy set with its spread more than two times wider than the universe of

discourse is regarded as the universal fuzzy set.

5.5.4.5 Merging Similar Fuzzy Sets

Jin (2000) proposed a simplified similarity measure based on the distance measure if Gaussian membership functions are involved. Although this measure does not satisfy all the conditions mentioned by Setnes et al. (1998), it works well when it tries to locate similar fuzzy sets in our case. Two fuzzy sets are considered to be similar if the following condition is met:

,     , _ · 1

, ,

(5.26)

where, _ is the threshold which randomly changes between [ , 1] every generations and is set to 0.95 in this work. The mean values of two similar fuzzy sets are calculated in order to substitute the original two fuzzy sets. It is worth mentioning that , is also checked if IMOFM_M is used. Figure 5.14 shows an example of merging two fuzzy sets with the same width and different centres at 0.45 and 0.5 respectively.

- 138 -

Figure 5.14 An example of merging similar fuzzy sets.

It is worth mentioning that all the simplification processes, except for the ‘insignificant rules’, have only αchance to be evoked at each iteration, where αis taken to be 20% in this work without any loss of generality. The similarity measures mentioned in Sections 5.5.4.4 and 5.5.4.5 will be checked for each fuzzy set. Only the ones with the maximum similarity values will be deleted or merged during each iteration step provided the conditions mentioned in Eqs. 5.25 and 5.26 are also met. For this reason and because of the elitism which records any non-dominated solution found at each iteration step during the experiments, it was found that the aforementioned thresholds are not critical parameters. Section 5.6.2 expands on such observation.