Fuzzy Signature Neural Network
Presented by:
U5251881 XuanYing ZHU
Supervisor
: Professor Tom GEDEON
Outline
l
Background
l Neural Network
l Fuzzy Logic, Fuzzy Rule Based System and
Fuzzy Signature
l Fuzzy Signature Neural Network l Previous work
l
Design & Implementation
l Construct Fuzzy Signature Neural Network l Implement testing suite
l
Experiment
l Experiment 1: with no missing data l Experiment 2: with 20% missing data l Experiment 3: with less fuzzy neurons
Background
l
Neural Network
:
Fig2. Example of a feed-forward back propagate neural network
Source: Chandra, P. “Fuzzy Signature Neural Networks for Rule Discovery”
Fig1. Example of a single neuron
Source: Kun, H. “Fuzzy Signature Neural Network”
A mathematical model that is inspired by biological neural network.
Background
l
Fuzzy Logic:
Fig 3: Difference between crisp set and fuzzy set
Source: Gedeon, T.D. 2013, Bio-‐inspired Compu8ng – COMP8420 Lecture Notes, Research School of Computer Science, Australian Na8onal University.
l
Fuzzy Rule Based System:
Represent knowledge based on degrees of membership
l Rule: If A THEN B
(A, B: collections of propositions containing linguistic variables)
e.g. Rule: IF x is A3 OR y is B1 THEN z is C1
l Problem:
Number of inputs
Background
Fig4: Two structures of fuzzy signature
Source: Gedeon, T.D. 2013, Bio-‐inspired Compu8ng – COMP8420 Lecture Notes, Research School of Computer Science, Australian Na8onal University.
l
Fuzzy Signature:
l Structure data into vectors of fuzzy values,
each of which can be a further vector
Background
Fig5: Example of aggregation
Source: Gedeon, T.D. 2013, Bio-‐inspired Compu8ng – COMP8420 Lecture Notes, Research School of Computer Science, Australian Na8onal University.
l
Fuzzy Signature:
l Aggregate:
l
GPLAB
l
a Gene8c Programming toolbox for MATLAB
l
Produce fuzzy signatures based on their inner-‐structures
Fig6: Example of Fuzzy Signature Neural Network
l
Fuzzy Signature Neural Network
Background
l
Previous work
l Similar neural network has been created by
Kun HE.
l Semi-randomly created fuzzy signatures. l Number of fuzzy signatures is determined
by users.
l
Our approach
l Data-driven way to create fuzzy signatures l Self-determined fuzzy signatures number l Improve HE’s fuzzy signature neural
network
l More automatic
l Reduce risks caused by manual selection
Design & Implementation
l
Construct fuzzy signature neural network
Fig7: Steps of constructing fuzzy signature neural network
l
Implement testing suite
Design & Implementation
l
Damage input
l Randomly remove some values
l
Cluster input
l Agglomerative hierarchical clustering l Advantages:
Fig9: Example of agglomerative hierarchical clustering
l Do not need users to
specify number of clusters
l More informative l Deterministic
Design & Implementation
l
Obtain fuzzy signatures
l Generate fuzzy signatures l Obtain membership values
l
Create & Train neural network
Fig6: Example of Fuzzy Signature Neural Network
Receive input Get membership value Generate actual output Compare with desired output Update weights Initialize weights
Design & Implementation
l
Implement testing suite
l
Test and collect results
l
K-fold cross validation -> split dataset into
training and testing datasets
l
Map function
l
Extract network information
0 0.5 1 1 2 3 4 5 membership value Class 0 0.5 1 1 2 3 4 5 membership value Class
Fig11(a): actual output Fig11(b): desired output Fig11: Example of actual output and desired output
Experiment
l
Experiment 1: with no missing data
Table 1: Results of our approach with no missing values and five fuzzy signatures
Experiment
l
Experiment 2: with 20% missing data
l Cancer dataset: missing one attribute
cancer diabetes high salary medium salary low salary This project 34.88170445 4.92186359 -‐1.840490798 -‐5.421686747 6.650860993 Kun's approach 0.308510638 5.324141977 -‐1.923076923 12.42937853 19.34968791 -‐10 -‐5 0 5 10 15 20 25 30 35 40 Decrea sed p ercen ta ge
Experiment
l
Experiment 3: with fewer fuzzy neurons
50 55 60 65 70 75 80 85 90 95 100 6 fuzzy
neurons neurons 5 fuzzy neurons 4 fuzzy neurons 3 fuzzy neurons 2 fuzzy Accuracy 50 55 60 65 70 75 80 85 90 95 100 6 fuzzy
neurons neurons 5 fuzzy neurons 4 fuzzy neurons 3 fuzzy neurons 2 fuzzy Accuracy
Fig 13: Testing accuracy for KUN’s and our approach as fuzzy neuron numbers decrease Fig 13(a): Testing accuracy for KUN’s
Conclusion & Future work
l
Conclusion
l This approach achieves stable and robust
good results in extreme situations
l With missing values
l With fewer fuzzy signatures l Data-oriented VS semi-random
l
Future work
l Find a more consistent and less
time-consuming fuzzy signature generation method.
