Constraint Back-Error-Propagation Algorithm

One problem associated with the above BEP updating formulas is that they include no constraints with respect to the update mechanism of these parameters. Hence, during the course of the optimisation, the centres are likely to be placed outside the boundaries. Although this does not affect the ultimate accuracy of FRBS, it may cause confusion for the users when assigning linguistic labels, and more importantly it may violate the search space

- 123 -

which will be defined in the next modelling stage. Hence, in this work, a constraint handling scheme is added, which checks the boundary violation for centres during each iteration step and drives any violated centres back to the boundaries. The process is illustrated in Figure 5.5.

Figure 5.5 Violated solutions are dragged back to the boundaries.

5.4.4 An Example of Application

As the continuation of the example shown in Sections 4.4.3 and 5.3.2, the elicited FRBSs in those sections are further optimised (viz. parameter optimisation) using the developed BEP updating formulas. It is worth mentioning that the step sizes ~ and the gains of momentum terms ~ are all set to 0.03 in this work without any loss of generality. The number of iterations is set to 1500 for Singleton FRBS and 600 for Mamdani FRBS, which are the empirical numbers that ensure the convergence of the BEP algorithm. Since this example is only exploited for illustration purposes and the data itself is very limited, the whole data set is used for training. Hence, the over-training problem is not the particular concern in this section. Such problem will be formally dealt with in Chapter 6 by dividing the data set into training and testing sets for all applications. For some applications, such as Ultimate Tensile Strength, a small extra data set is also available, which serves as the

- 124 -

validation set in our work. Figures 5.6 and 5.7 show the refined Singleton and Mamdani FRBSs along with their membership functions.

- 125 -

Figure 5.7 (a) the refined Mamdani FRBS; (b) its associated membership functions.

As one can see from Figures 5.6 and 5.7, the knowledge discovered by Singleton FRBS and Mamdani FRBS is consistent in terms of the distributions and the combinations of the membership functions (linguistic terms). However, the Mamdani FRBS has the advantage of being able to express clear semantic meanings in its consequents due to the inclusion of the width. As mentioned in Sections 4.4.3 and 5.1.4, the automatic rule induction process and unconstrained optimisation often lead to a deteriorated interpretability, and this is firmly supported by Figures 4.21, 5.6 and 5.7. It is because of this reason that the third modelling stage is a necessity and is normally included to improve model transparency. Figure 5.8 shows the predictive performances of the refined Singleton and Mamdani FRBSs by plotting their predicted outputs against the real outputs.

- 126 -

Figure 5.8 (a) the predictive performances of the initial and the refined Singleton FRBS; (b)

the predictive performances of the initial and the refined Mamdani FRBS.

Table 5.2 summarises the predictive performances of the second modelling stage when using Singleton FRBS and Mamdani FRBS. The results are the average values of 20 independent runs. It can be seen from this table that, after the BEP refinement, both FRBSs’ predictive performances are singificantly improved.

TABLE5.2

THE PREDICTIVE PERFORMANCES OF THE SECOND MODELING STAGE OF IMOFM_S AND IMOFM_M ON

ANONLINEAR STATIC SYSTEM WITH FIVE RULES

Modeling Methods  The Predictive Performance of FRBSs from the 2

nd _Stage 

RMSE (average)  Std.  Time (sec.) 

IMOFM_S  0.0688  0  120 

IMOFM_M  0.0702  0  37 

- 127 -

5.5 Third Stage: Immune Algorithms-based Multi-Objective

Fuzzy Modelling

An optimal FRBS can be obtained by optimising the rule-base structure and membership function parameters either simultaneously or separately. The previous two modeling stages can be viewed as the instances of a separate structure and parameter learning. The drawbacks of the separate learning option are as follows:

™ Only a ‘sub-optimal’ result may be obtained since both the structure and the parameters of the rule-base need to cooperate to provide a satisfactory FRBS.

™ The separate learning structure relies too strongly on subjective judgment. Hence, only challenge 2, namely the need to set the start points, as mentioned in Section 5.1.4 would have been solved by the first two stages, which should mainly be attributed to the global search capacity of the G3Kmeans algorithm. As far as the other two limitations are concerned, one still has to set the initial abstraction level and only an approximate FRBS with obscure semantics can be elicited as a result.

To improve the interpretability of such an approximate FRBS, the authors in (Setnes et al., 1998; Setnes et al., 2000; Roubos et al., 2001; Chen et al., 2001) performed model simplifications and fine-tunings. The learning procedure described in these research investigations can still be labeled as being a separate learning process so that model simplifications rely heavily on the pre-specified thresholds according to the designer’s choice. Wang et al. (2005) proposed a hierarchical scheme to evolve both parts. However, a rule matrix was required, which rendered the scheme vulnerable to high dimensional problems due to the exponential increase in the matrix dimension. Research work reported in (Jim ́nez et al., 2001; Jim ́nez et al., 2002; Gonz ́lez et al., 2007) adopted a variable length coding strategy in order to cope with high dimensional problems. However, as mentioned in Section 5.1.5, only heuristic variation operators are used in these works, which did not do justice to the idea of using variable length coding. In fact, it may somehow impede the search power of EAs as far as the real-valued optimisation part is concerned. Apart from these problems, research investigations in (Setnes et al., 1998; Setnes et al., 2000; Roubos et al., 2001; Jim ́nez et al., 2001; Jim ́nez et al., 2002; Wang et al., 2005; Gonz ́lez et al., 2007) dealt with TSK FRBS with linear functions as their consequents, which detracts from the linguistic attempts of the authors’ proposed methods.

- 128 -

The proposed approach in this current research work utilises a multi-objective optimisation framework and a variable length coding scheme, which does not suffer from ‘the curse of dimensionality’. A set of FRBSs representing the trade-offs between interpretability and accuracy are obtained through a single run, and only the maximum allowable number of rules is required a priori, which reduces any user intervention during the whole design process to a minimum level. As can be seen from Figure 5.3, a ‘variable length coding scheme’ and a ‘model simplification’ are integrated into the original PAIA2 to account for parameter and structure optimisation. A new distance index is proposed to facilitate the use of the original variation operator in PAIA2. Details of these operators and the way of formulating objective functions and the initial population pool are explained next.