IJEDR1503031 International Journal of Engineering Development and Research (www.ijedr.org)
1
Fuzzy Neural Network for Clustering and
Classification
1Archana R. Shinde, 2 Prof .D.B Kshirsagar 1 Student, 2 Professor,
1 Department Computer Engineering,
1 S.R.E.S. College of Engineering, Kopargaon,Maharashtra India.
________________________________________________________________________________________________________ Abstract - This paper presents the implementation of Fuzzy Neural Network (FNN) for clustering and classification. Fuzzy neural network combines the advantage of both fuzzy logic and neural network. This paper mainly focuses on implementation of two algorithms. First algorithm is General Fuzzy min max Neural Network (GFMM) training algorithm which combines the processing of supervised and unsupervised data in a single training algorithm. Second algorithm is Data Core Based Fuzzy min max Neural Network (DCFMN) which considers the characteristics of the data and influence of noise simultaneously. These two algorithms provide high accuracy, flexibility, better performance, strong robustness in classification and clustering.
Index Terms - Classification, Clustering, Fuzzy systems, Fuzzy min-max neural networks ,Data core, Overlapped neuron, Pattern classification, Hyperboxes, Hyperbox expansion, Hyperbox contraction, Overlapping neurons, Classifying neurons ,Membership function, Robustness.
________________________________________________________________________________________________________
I.INTRODUCTION
Now a days fuzzy logic and neural network are greatly used to develop intelligent systems .The main reason to combine fuzzy logic and neural network is that fuzzy logic have greater ability to handle approximate or uncertain information with the capability of neural networks in learning from processes. This paper presents implementation of two algorithms. One is General Fuzzy min-max Neural Network (GFMM)[4] and second is Data Core Based Fuzzy min-min-max Neural Network (DCFMN)[7].
General Fuzzy min-max Neural Network (GFMM)[4] processes supervised and unsupervised data in a single training algorithm. In labeled or supervised learning ,which is also referred as a pattern classification problem, labels are provided with input patterns. In unlabeled or unsupervised learning, also called as a cluster analysis problem, the input pattern is unlabeled and we must deal with the splitting a set of input patterns into a number of more or less homogenous clusters with respect to a suitable similar property. Patterns which are similar are allocated to the same cluster, while the patterns which differ are put in different clusters.
In many industrial processes, the measurement data may include noise caused by various kinds of processes which affects system performance and may decrease the accuracy of classification. Therefore, Data Core Based Fuzzy min max Neural Network (DCFMN) [7] algorithm is implemented to improve the performance of classification.
II.RELATED WORK
IJEDR1503031 International Journal of Engineering Development and Research (www.ijedr.org)
2
succession of hyperbox cuts. The generalization capability of ARC/PARC technique mostly depends on the adopted cutting strategy. By using a recursive cutting procedure (RARC and R-PARC) it is possible to obtain better results.III.SYSTEM ARCHITECTURE
This paper presents the implementation of a fuzzy neural network for clustering and classification. In this fuzzy neural network two training algorithm are implemented for clustering and classification. First algorithm is General Fuzzy min max Neural Network (GFMM)[4] and second is Data Core Based Fuzzy min max Neural Network (DCFMN)[7]. In GFMM single training algorithm is required to processes labeled and unlabeled input patterns.Inthis algorithm learning process is completed in a few passes through the data and of placing and adjusting the hyperboxes in the pattern space which is referred as an expansion–contraction process. To retaining all the interesting features, a number of modifications to their definition have been made in order to accommodate fuzzy input patterns in the form of lower and upper bounds, combine the labeled and unlabeled input patterns, and improve the effectiveness of system , but in GFMM new data can be included without retraining of data. DCFMN considers the characteristics of the data and influence of noise simultaneously while performing classification. In DCFMN , membership function has been designed based on the geometric center and data core in a hyperbox. Instead of using the contraction process, overlapped neuron with new membership function based on the data core is proposed and added to neural network to represent the overlapping area of hyperboxes belonging to different classes. DCFMN has strong robustness and high accuracy in classification taking onto account the effect of data core and noise. The general architecture of the proposed system is shown in figure.1
Figure.1 General System Architecture
A.General Fuzzy min max neural network (GFMM) Algorithm
GFMM algorithm consist of mainly four steps which are Initialization, Expansion, Overlap Test, and Contraction. Basic Terms :
First we see some basic terms that are used in this algorithm 1. Input:
Input provided to the GFMM algorithm is in the form of ordered pair as
{𝑋ℎ,𝑑ℎ}
(1)
where 𝑋ℎ is the ℎ𝑡ℎ input pattern and 𝑑
ℎ ∈ {1,2,…,𝑝} is the index of one of the 𝑝+1 classes, where 𝑑ℎ =0 means that the input pattern is unlabeled.
2. Fuzzy Hyperbox Membership Function:
Fuzzy Hyperbox Membership Function is very important in fuzzy min max neural network algorithm. The decision whether the given input pattern belongs to a particular class or cluster, thus whether the corresponding hyperbox is to be expanded, depends mainly on the membership value describing the degree to which an input pattern fits within the hyperbox .In GFMM algorithm the membership function of 𝑗𝑡ℎhyperbox for ℎ𝑡ℎinput pattern is calculated as
𝑏𝑗(𝑋ℎ)=min
𝑖=1..,𝑛(min([1 − 𝑓(𝑥ℎ𝑖
𝑢 − 𝑤
𝑗𝑖, 𝛾𝑖)],
[1 − 𝑓(𝑣𝑗𝑖− 𝑥ℎ𝑖𝑙 , 𝛾𝑖)] )) (2) Where 𝑓(𝑟, 𝛾)={
1𝑖𝑓𝑟𝛾 > 1 𝑟𝛾𝑖𝑓0 ≤ 𝑟𝛾 ≤ 1
0𝑖𝑓𝑟𝛾 < 0 Where 𝑥ℎ𝑖𝑙 and 𝑥ℎ𝑖𝑢 is the lower and upper limit ofℎ𝑡ℎ input pattern, 𝑣
𝑗𝑖 and 𝑤𝑗𝑖 is the min and max points of 𝑗𝑡ℎ hyperbox and threshold function; 𝛾=[ 𝛾1,𝛾2,…,𝛾𝑛]—sensitivity parameters which regulates how fast the membership values decrease.
IJEDR1503031 International Journal of Engineering Development and Research (www.ijedr.org)
3
1. Initialization : Initially min point 𝑉𝑗 and max point 𝑊𝑗 of 𝑗𝑡ℎ hyperbox set to
𝑉𝑗= 0𝑎𝑛𝑑𝑊𝑗= 0 (3) When first input pattern is presented 𝑗𝑡ℎ hyperbox adjusted and min and max points set to
𝑉𝑗 = 𝑥ℎ𝑙𝑎𝑛𝑑𝑊𝑗= 𝑥ℎ𝑢 (4)
2. Hyperbox Expansion : When the input pattern is presented, find the hyperbox with the highest degree of membership and allowing expansion (if needed) is expanded to include input pattern . Expansion criteria is
∀i=1…n(max(wji, xhiu) − min(v
ji, xhil )) ≤ θ (5) And
ifdh= 0thenadjustBj else
ifclass(Bj) =
{
0 ⇒ adjustBj class(Bj) = dh dh ⇒ adjustBj
take else ⇒ anotherBj
(6)
We adjust 𝐵𝑗 in following way.
vjinew= min(v ji old, x
hil ) foreachi = 1, … , n
wjinew= max(w
ji old, x
hiu) foreachi = 1, … , n
If none of the existing hyperboxes can expand to include the input pattern 𝑥ℎ , then a new hyperbox 𝐵𝑘 is created, adjusted, and labeled by setting 𝑐𝑙𝑎𝑠𝑠(𝐵𝑘)=𝑑ℎ.
The θ is a user-defined parameter that define a bound on the maximum size of a hyperbox and its value significantly affects the effectiveness of the training algorithm.
3. Hyperbox Overlap Test: Due to inclusion of input patterns hyperboxes of different classes are overlapped with each other .So to detect overlapping between hyperboxes overlap test is performed as follows
If the hyperbox, expanded in the last expansion step ,is not labeled then test for overlapping with all the other hyperboxes. This ensures that all unlabeled hyperboxes do not overlap with any of the other existing ones.
If the hyperbox 𝐵𝑗 expanded in the last expansion step,belongs to one of the existing classes then test for the overlap only with the hyperboxes not being part of the same class as 𝐵𝑗. Notice that this allows to overlap the hyperboxes belonging to the same class.
The full hyperbox overlap test can be therefore summarized as follows.
class(Bj) = {
0 ⇒ testoverlappingwith 𝑎𝑙𝑙𝑜𝑡ℎ𝑒𝑟ℎ𝑦𝑝𝑒𝑟𝑏𝑜𝑥𝑒𝑠 𝑒𝑙𝑠𝑒 ⇒ 𝑡𝑒𝑠𝑡𝑜𝑣𝑒𝑟𝑙𝑎𝑝𝑝𝑖𝑛𝑔𝑜𝑛𝑙𝑦𝑖𝑓
𝑐𝑙𝑎𝑠𝑠(𝐵𝑗) ≠ 𝑐𝑙𝑎𝑠𝑠(𝐵𝑘)
(7)
The four cases are being considered (where initially 𝜕𝑜𝑙𝑑=1).
Case 1:𝑣𝑗𝑖< 𝑣𝑘𝑖< 𝑤𝑗𝑖< 𝑤𝑘𝑖
𝜕𝑛𝑒𝑤= min(𝑤
𝑗𝑖− 𝑣𝑘𝑖, 𝜕𝑜𝑙𝑑). Case 2:𝑣𝑘𝑖< 𝑣𝑗𝑖< 𝑤𝑘𝑖< 𝑤𝑗𝑖
𝜕𝑛𝑒𝑤 = min(𝑤
𝑘𝑖− 𝑣𝑗𝑖, 𝜕𝑜𝑙𝑑). Case 3:𝑣𝑗𝑖< 𝑣𝑘𝑖≤ 𝑤𝑘𝑖< 𝑤𝑗𝑖
𝜕𝑛𝑒𝑤= min(min(𝑤
𝑘𝑖− 𝑣𝑗𝑖, 𝑤𝑗𝑖− 𝑣𝑘𝑖) , 𝜕𝑜𝑙𝑑). Case 4:𝑣𝑘𝑖< 𝑣𝑗𝑖≤ 𝑤𝑗𝑖< 𝑤𝑘𝑖
𝜕𝑛𝑒𝑤 = min(min(𝑤
IJEDR1503031 International Journal of Engineering Development and Research (www.ijedr.org)
4
If overlap for the 𝑖𝑡ℎ dimension has been detected (one of the above four cases is valid) and 𝜕𝑜𝑙𝑑− 𝜕𝑛𝑒𝑤 > 0, then ∆=i, 𝜕𝑜𝑙𝑑= 𝜕𝑛𝑒𝑤and 𝑐𝑎𝑠𝑒 = 𝑙 (𝑙= {1,2,3,4}− the case for which the smallest overlap was found). If overlap for the𝑖𝑡ℎdimension has not been detected, set Δ=−1 which indicate that the contraction step is not necessary.4. Hyperbox Contraction: If Δ>0then only the Δ𝑡ℎ dimensions of the two hyperboxes are adjusted. For contraction again four cases are being considered.
Case 1:𝑣𝑗∆< 𝑣𝑘∆< 𝑤𝑗∆< 𝑤𝑘∆ 𝑣𝑘∆𝑛𝑒𝑤= 𝑤𝑗∆𝑛𝑒𝑤=𝑣𝑘∆
𝑜𝑙𝑑+ 𝑤
𝑗∆𝑜𝑙𝑑 2 or alternatively (𝑤𝑗∆𝑛𝑒𝑤=𝑣𝑘∆𝑜𝑙𝑑). Case 2:𝑣𝑘∆< 𝑣𝑗∆< 𝑤𝑘∆< 𝑤𝑗∆
𝑣𝑗∆𝑛𝑒𝑤= 𝑤 𝑘∆𝑛𝑒𝑤=
𝑣𝑗∆𝑜𝑙𝑑+ 𝑤 𝑘∆𝑜𝑙𝑑 2 or alternatively (𝑣𝑗∆𝑛𝑒𝑤=𝑤
𝑘∆𝑜𝑙𝑑). Case 3:𝑣𝑗∆< 𝑣𝑘∆≤ 𝑤𝑘∆< 𝑤𝑗∆
𝑖𝑓𝑤𝑘∆− 𝑣𝑗∆< 𝑤𝑗∆− 𝑣𝑘∆
𝑡ℎ𝑒𝑛𝑣𝑗∆𝑛𝑒𝑤 = 𝑤𝑘∆𝑜𝑙𝑑otherwise (𝑤𝑗∆𝑛𝑒𝑤=𝑣𝑘∆𝑛𝑒𝑤) Case 4:𝑣𝑘∆< 𝑣𝑗∆≤ 𝑤𝑗∆< 𝑤𝑘∆
𝑖𝑓𝑤𝑘∆− 𝑣𝑗∆< 𝑤𝑗∆− 𝑣𝑘∆ 𝑡ℎ𝑒𝑛𝑤𝑘∆𝑛𝑒𝑤= 𝑣
𝑗∆𝑜𝑙𝑑 otherwise (𝑣𝑘∆𝑛𝑒𝑤=𝑤𝑗∆𝑜𝑙𝑑). B.Data Core Based Fuzzy min max Neural Network (DCFMN) Algorithm:
In DCFMN a new membership function for classifying the neuron of DCFMN is defined in which the noise, the geometric center of the hyperbox, and the data core are considered.
1. CN Membership Function: For considering the influence of noise and the density of data in the hyperbox, an improved hyperbox membership function of CN is defined as follows:
𝑏𝑗(𝑥ℎ)=min
𝑖=1,..,𝑛(min(𝑓(𝑥ℎ𝑖− 𝑤𝑗𝑖+ 𝜀, 𝑐𝑗𝑖),
𝑓(𝑣𝑗𝑖+ 𝜀 − 𝑥ℎ𝑖, 𝑐𝑗𝑖)))) (8) where ε is a parameter representing noise, c is difference between the data core in the hyperbox and the geometric center of the corresponding hyperbox, and f is the ramp threshold function which is defined as.
𝑓(𝑟, 𝑐) = {
𝑒−𝑟2×(1+𝑐)×𝜆, 𝑟 > 0, 𝑐 > 0 𝑒−𝑟2×(1−𝑐)×1/𝜆, 𝑟 > 0, 𝑐 < 0
1,𝑟 < 0
2. OLN Membership Function: An OLN is created when there is an overlapping area between two hyperboxes that belong to two different classes 𝑉′, 𝑊′ nodes in OLN represent the minimum and maximum points of the overlapped area. The activation function used for OLN is given by
do,q(xh) = g(vo′, wo′, xh, yq)
={ 1 𝑛
0,∑ (1 − |𝑥ℎ𝑖− 𝑦𝑞𝑖|) 𝑛
𝑖=1 ,∀𝑖=1,..𝑛𝑤𝑜𝑖> (𝑥ℎ𝑖) > 𝑣𝑜𝑖 (9)
where o = 1, 2, . . . , l is the index of overlapped node, q =1, 2 is the index of hyperbox which has the same overlap, and𝑦𝑞is the data core of each overlapped hyperbox.
DCFMNLearning Algorithm
DCFMN algorithm consist of three steps as follows:
1. Hyperbox Expansion: Identify the expandable hyperboxes and expand them. For the hyperbox to be expanded, the following expansion criteria must be met:
IJEDR1503031 International Journal of Engineering Development and Research (www.ijedr.org)
5
𝑣𝑗𝑖𝑛𝑒𝑤= min(𝑣𝑗𝑖𝑜𝑙𝑑, 𝑥ℎ𝑖𝑙 ) 𝑓𝑜𝑟𝑒𝑎𝑐ℎ𝑖 = 1, … , 𝑛𝑤𝑗𝑖𝑛𝑒𝑤= max(𝑤𝑗𝑖𝑜𝑙𝑑, 𝑥ℎ𝑖𝑢) 𝑓𝑜𝑟𝑒𝑎𝑐ℎ𝑖 = 1, … , 𝑛
2. Hyperbox Overlap Test : Due to inclusion of input patterns two hyperboxes are overlapped with other. So detect overlapping between hyperbox of different classes overlap test is performed. Assuming that 𝛼𝑜𝑙𝑑= 1 initially, the four test cases and the corresponding minimum overlap value for the 𝑖𝑡ℎ dimension is as follows.
Case 1:𝑣𝑗𝑖< 𝑣𝑘𝑖< 𝑤𝑗𝑖< 𝑤𝑘𝑖 𝛼𝑛𝑒𝑤= min(𝑤
𝑗𝑖− 𝑣𝑘𝑖, 𝛼𝑜𝑙𝑑). Case 2:𝑣𝑘𝑖< 𝑣𝑗𝑖< 𝑤𝑘𝑖< 𝑤𝑗𝑖
𝛼𝑛𝑒𝑤= min(𝑤
𝑘𝑖− 𝑣𝑗𝑖, 𝛼𝑜𝑙𝑑). Case 3:𝑣𝑗𝑖< 𝑣𝑘𝑖≤ 𝑤𝑘𝑖< 𝑤𝑗𝑖
𝛼𝑛𝑒𝑤= min(min(𝑤
𝑘𝑖− 𝑣𝑗𝑖, 𝑤𝑗𝑖− 𝑣𝑘𝑖) , 𝛼𝑜𝑙𝑑). Case 4:𝑣𝑘𝑖< 𝑣𝑗𝑖≤ 𝑤𝑗𝑖< 𝑤𝑘𝑖
𝛼𝑛𝑒𝑤= min(min(𝑤
𝑘𝑖− 𝑣𝑗𝑖, 𝑤𝑗𝑖− 𝑣𝑘𝑖) , 𝛼𝑜𝑙𝑑).
If 𝛼𝑜𝑙𝑑−𝛼𝑛𝑒𝑤> 0, then there is an overlap for the 𝑖𝑡ℎ dimensions, and let ∆= i . Otherwise, the test stops and set 𝑖𝑡ℎ = 0, and nothing is done.
3. Adding OLN - If an overlap between hyperboxes of different classes exists, the process of adding OLN is performed as follows.
Case1:𝑣𝑗𝑖< 𝑣𝑘𝑖< 𝑤𝑗𝑖< 𝑤𝑘𝑖
𝑣𝑚+𝑜𝑖′ = 𝑣𝑘𝑖, 𝑤𝑚+𝑜𝑖′ = 𝑤𝑗𝑖 Case 2:𝑣𝑘𝑖 < 𝑣𝑗𝑖< 𝑤𝑘𝑖 < 𝑤𝑗𝑖
𝑣𝑚+𝑜𝑖′ = 𝑣
𝑗𝑖, 𝑤𝑚+𝑜𝑖′ = 𝑤𝑘𝑖. Case 3:𝑣𝑗𝑖< 𝑣𝑘𝑖 ≤ 𝑤𝑘𝑖 < 𝑤𝑗𝑖
𝑣𝑚+𝑜𝑖′ = 𝑣
𝑘𝑖, 𝑤𝑚+𝑜𝑖′ = 𝑤𝑘𝑖 Case 4:𝑣𝑘𝑖 < 𝑣𝑗𝑖≤ 𝑤𝑗𝑖< 𝑤𝑘𝑖
𝑣𝑚+𝑜𝑖′ = 𝑣
𝑗𝑖, 𝑤𝑚+𝑜𝑖′ = 𝑤𝑗𝑖
where o = (1, 2, . . . , l) is index of neuron to be added to the network, and 𝑉′ and𝑊′ are used to store the minimum and maximum points of the hyperboxes.
IV.IMPLEMENTATION DETAILS
This system mainly presents implementation of two algorithm, GFMM and DCFMN and our objective is to implements these two algorithm for standard datasets iris and wine .GFMM is four step algorithm which are Initialization, Expansion, Overlap Test, and Contraction which are successfully completed and gives desired output.
Figure 2.shows browsing of dataset as input for our system .Iris dataset is browsed which contains 150 records of three classes .All input patterns are labeled.
IJEDR1503031 International Journal of Engineering Development and Research (www.ijedr.org)
6
Figure 3. shows that number of hyperboxes created of different classes , here user defined parameter θ set 0.1 and number of hyperboxes created is 139. As we increase value of θ from 0.1 to 0.9 ,number of hyperboxes created is decreases.Figure 3: Number of Hyperboxes created.
Figure 4. Shows overlapping found between hyperboxes and contraction of hyperboxes performed to remove overlapping of hyperboxes of different classes.
Figure 4: Overlap test and contraction
Up to this step training of algorithm is completed , now testing is performed by presenting input pattern to the system and system classify that input pattern correctly and displaying class of that input pattern which shown in figure 5. Misclassification rate of GFMM algorithm is very less.
IJEDR1503031 International Journal of Engineering Development and Research (www.ijedr.org)
7
V.CONCLUSION
This paper presents the implementation of fuzzy neural network for clustering and classification which uses two training algorithm for clustering and classification. These two training algorithm is based on fuzzy logic and neural network. First algorithm is General Fuzzy min max Neural Network which combines the processing of supervised and un supervised input data within a single training algorithm and provide better performance. Second algorithm is Data Core Based Fuzzy min max Neural Network which considers the characteristics of the data and influence of noise simultaneously and provide strong robustness against noise and better performance in classification
VI.ACKNOWLEDGMENT
I would like to take this opportunity to express my profound gratitude and deep regard to my Project Guide Prof. D. B. Kshirsagar, for his exemplary guidance, valuable feedback and constant encouragement throughout the duration of the project. His valuable suggestions were of immense help throughout my project work. His perceptive criticism kept me working to make this project in a much better way. Working under him was an extremely knowledgeable experience for me.
REFERENCES
[1] P. K. Simpson, “Fuzzy min-max neural networks-part I: Classification,” IEEE Trans. Neural Netw., vol. 3, no. 5, pp. 776– 786, Sep. 1992.
[2] P. K. Simpson, “Fuzzy min-max neural network-part II: Clustering,” IEEE Trans. Fuzzy Syst., vol. 1, no. 1, pp. 32–45, Feb. 1993.
[3] M. Meneganti, F. S. Saviello, and R. Tagliaferri, “Fuzzy neural networks for classification and detection of anomalies,” IEEE Trans. Neural Netw., vol. 9, no. 5, pp. 846–861, Sep. 1998.
[4] B. Gabrys and A. Bargiela, "General fuzzy min-max neural network for clustering and classification," IEEE Trans. Neural Netw., vol. 11, no. 3, pp. 769-783, May 2000.
[5] D. Chralampidis , T. Kasparis, and G. Meorgiopoulos, "Classification of noisy signals using fuzzy ARTMAP neural networks " , IEEE Trans. Neural Netw., vol. 12, no. 5, pp. 1023-1036, Sep. 2001.
[6] F. J. Lin and P. H. Shen, "Robust fuzzy neural network sliding-mode control for two-axis motion control system" , IEEE Trans. Ind. Electron., vol. 53, no. 4, pp. 1209-1225, Aug. 2006.
[7] Huaguang Zhang, Jinhai Liu , Dazhong Ma,and Zhanshan Wang , "Data Core-Based Fuzzy Min-Max Neural Network for Pattern Classification", IEEE Trans. vol. 22, no. 12, Dec. 2011.
[8] S. Yilmaz and Y. Oysal, “Fuzzy wavelet neural network models for prediction and identification of dynamical systems ,” IEEE Trans. Neural Netw., vol. 21, no. 10, pp. 1599–1609, Oct. 2010.
[9] H. G. Zhang, C. Y. Wang, and D. Liu, “Delay-dependent guaranteed cost for uncertain stochastic fuzzy systems with multiple time delays,” IEEE Trans. Syst., Man Cybern., Part B, vol. 38, no. 1, pp. 126–140, Jan. 2008.