A Multiobjective Genetic Fuzzy System for Obtaining Compact and Accurate Fuzzy Classifiers with Transparent Fuzzy Partitions

(1)

A Multiobjective Genetic Fuzzy System for

Obtaining Compact and Accurate Fuzzy Classifiers

with Transparent Fuzzy Partitions

Pietari Pulkkinen Tampere University of Technology Department of Automation Science and Engineering

Finland

(2)

Fuzzy Classifiers (FCs)

Classification is based on ”if-then” fuzzy rules. An example rule:

If temperature is high and humidity is high, then climate is tropical

Intuitive way of reasoning

Before applying an FC in practice, it is possible to verify that:

1 _{the FC is accurate enough}

2 _{that the fuzzy rules are reasonable}

Interpretability

Complex FCs with large number of rules are hard to interpret No reasonable linguistic labels for highly overlapping fuzzy sets Compact rule bases and

transparent fuzzy partitions are preferred!

Transparent fuzzy partitions

0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 0 0.25 0.5 0.75 1 x 7 Membership 2 3 4 5 6 7 8 9 10 11 0 0.25 0.5 0.75 1 x10 Membership 400 600 800 1000 1200 1400 1600 0 0.25 0.5 0.75 1 x13 Membership

(4)

An FC as a Reasoning Mechanism in a Bioaerosol Detector

Bioaerosol detector was developed by: Dekati, Environics, and TUT / Department of Physics

Reasoning mechanism was developed by: TUT / Department of Automation Science and Engineering

Pulkkinen, P., Hyt¨onen, J. and Koivisto, H.: Developing a bioaerosol detector using hybrid genetic fuzzy systems.

(5)

Interpretability Accuracy Trade-off

The purpose is to minimize the number of misclassifications and to minimize the complexity of FCs

These are conflicting objectives! Improving one objective,

deteriorates the other. ⇒ Search for Pareto-optimal FCs

Top: training set, bottom: testing set

0 5 10 15 0.25 0.3 0.35 0.4 0.45 0.5

Number of fuzzy rules

Error rate on train set

05 10 15 20 25 30 35 40 45 50 55 60 0.25 0.3 0.35 0.4 0.45 0.5

Total rule length

Error rate on train set

0 5 10 15 0.35 0.4 0.45 0.5 0.55

Number of fuzzy rules

Error rate on test set

05 10 15 20 25 30 35 40 45 50 55 60 0.35 0.4 0.45 0.5 0.55

Total rule length

Error rate on test set

Training set: Complex FCs are the most accurate Testing set: Some simpler FCs seem to be very accurate in this example

P. Pulkkinen and H. Koivisto, Fuzzy classifier identification using decision tree and multiobjective evolutionary

algorithms,Int. J. Approx. Reason., vol.

48, no. 2, pp. 526-543, June 2008.

(6)

Searching for FCs Involves a Large Search Space

A simple FC with 3 rules can be presented as:

Membership functions −50 0 5 0.5 1 x Jäsenyysaste Small Large −50 0 5 0.5 1 y Membership Small Large

Denote small with 1 and large with 2

Rule₁_{: If x is 1 and y is 1 then class is 3} Rule₂: If x is 1 and y is 2 then class is 2

Rule₃: If x is 2 then class is 1 The antecedents of rules: A= ( 1,1 |{z} Rule₁ , 1,2 |{z} Rule₂ , 2,0 |{z} Rule₃ ).

4 gbell membership functions: P= (P1,1,P1,2,P1,3,P1,4 | {z } Parametera ,P2,1,P2,2,P2,3,P2,4 | {z } Parameterb , P3,1,P3,2,P3,3,P3,4 | {z } Parameterc ).

Rule consequent (i.e. class number): S= ( 3 |{z} Rule₁ , 2 |{z} Rule₂ , 1 |{z} Rule₃ )

(7)

Multiobjective Genetic Fuzzy System

Initial population of FCs is further optimized by NSGA-II developed by Kalyanmoy Deb et al.

Purpose: to minimize the number of misclassifications and to minimize the number of rule conditions

MF parameters are adjusted and rules are learnt

Granularity, i.e., the number of fuzzy sets in each partition is also learnt

Dynamic constraints keep the fuzzy partitions always transparent. No need to minimize any transparency index

More efficient search

Result: A Pareto optimal set of compact and accurate FCs All of them have transparent fuzzy partitions

(8)

Transparency Conditions

MFs tuning usually improves the accuracy, but may deteriorate the transparency of fuzzy partitions

1 _α_-condition: At any intersection point of two MFs, the

membership value is at most α.

2 _γ_-condition: At the center of each MF, no other MF receives

membership value larger than γ.

3 β-condition: At each point of universe of discourse at least

one MF has membership value at least β.

β = 0.05, γ= 0.25 and α= 0.8 are used.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 α β γ

(9)

Dynamic Tuning of Membership Functions

Dynamically constrained 3-parameter MFs tuning strategy1 is used:

Start from a transparent fuzzy partition and modify one of the gbell MF parameter a,b, or c.

µ(x;a,b,c) = 1

1+|x−c

a |

2b

Only one parameter is modified at a time

If number of MFs is altered, a simple partition algorithm is used to create a new transparent partition

Every partition in each FC is always transparent!

1

More details available in: P. Pulkkinen and H. Koivisto. A dynamically constrained multiobjective genetic

fuzzy system for regression problems.IEEE Transactions on Fuzzy Systems(accepted)

(10)

Two Simple Partition Algorithms

Algorithms are used to:

provide a transparent starting point for MFs tuning

to find good partitions during further optimization

Partitions are always transparent

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 α β γ

An evenly distributed uniformly shaped partition 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 α β γ (a) An unevenly distributed non-uniformly shaped partition

(11)

Dynamic Tuning of Membership Functions: an Example

Modifying MF 2 (a) Original partition (b) decrease its width (c) alter its shape (d) move it towards right

The original and modified partitions

0 0.2 0.4 0.6 0.8 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 α β γ

(a) Original partition

Degree of membership 0 0.2 0.4 0.6 0.8 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

(b) Parameter a of MF 2 set to its minimum value

Degree of membership α β γ 0 0.2 0.4 0.6 0.8 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

(c) Parameter b of MF 2 set to its minimum value

Degree of membership α β γ 0 0.2 0.4 0.6 0.8 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

(d) Parameter c of MF 2 set to its maximum value

Degree of membership α β γ 1 1 1 1 2 2 3 3 4 4 4 4 3 3 2 5 5 5 5 2

(12)

Experiments

Two well known classification problem WineandGlasswere

studied. 10-fold cross-validation was repeated 10 times for both problems. (altogether 200 runs)

Wine is a problem with three different classes and 13 input variables

Glass is a problem with six different classes and 9 input variables

Results compared to our former approach2:

It also utilizes NSGA-II

It does not apply dynamic constraints and partitions are not always transparent

Expected to have better accuracy than the proposed method due to trade-off between accuracy and transparency of fuzzy partitions.

2

P. Pulkkinen and H. Koivisto, Fuzzy classifier identification using decision tree and multiobjective

(13)

Results

According to T-test, no statistical difference in test accuracy in both of the problems!

8 10 12 14 16 18 0 0.1 0.2 0.3 0.4 Rules Error rate 10 20 30 40 50 60 0 0.1 0.2 0.3 0.4 Rule conditions Error rate Method [10] (train) Method [10] (test) This paper (test) This paper (train)

Glass problem: Comparison of the averaged Pareto fronts: It was expected that the former approach should be more accurate

Surprisingly, especially test accuracy is almost the same for both approaches

(14)

Glass problem: comparison of the fuzzy partitions

An example FC by the former approach

Some partitions are not transparent 1.515 1.52 1.525 1.53 0 0.250.5 0.751 x 1 Membership 0 12 14 16 0.250.5 0.751 x 2 Membership 0 1 2 3 4 0 0.250.5 0.751 x3 Membership 0 1 2 3 0.250.5 0.751 x4 Membership 70 72 74 0 0.250.5 0.751 x 5 Membership 00 2 4 6 0.250.5 0.751 x 6 Membership 0 1 2 3 0 0.250.5 0.751 x 8 Membership 00 0.1 0.2 0.3 0.250.5 0.751 x 9 Membership

An example FC by the proposed approach

All partitions are transparent

1.515 1.52 1.525 1.53 0 0.25 0.5 0.75 1 x 1 Membership 12 14 16 0 0.25 0.5 0.75 1 x 2 Membership 0 1 2 3 4 0 0.25 0.5 0.75 1 x 3 Membership 1 2 3 0 0.25 0.5 0.75 1 x 4 Membership 70 72 74 0 0.25 0.5 0.75 1 x 5 Membership 0 0.2 0.4 0 0.25 0.5 0.75 1 x 9 Membership

(15)

Conclusions

A multiobjective genetic fuzzy system which searches for compact and accurate FCs with transparent fuzzy partitions was developed. Its strengths are:

Number of input variables is reduced already in the initialization phase

The number of fuzzy sets is learnt and MFs are tuned and resulting partitions are always transparent

Accuracy and compactness was comparable to our former approach even though that approach does not always lead to transparent fuzzy partitions

The proposed approach is not limited only to classification problems. Regression problem can be handled with some modifications3.

3

P. Pulkkinen and H. Koivisto. A dynamically constrained multiobjective genetic fuzzy system for regression

problems.IEEE Transactions on Fuzzy Systems(accepted)

A Multiobjective Genetic Fuzzy System for Obtaining Compact and Accurate Fuzzy Classifiers with Transparent Fuzzy Partitions