Final Evaluation Process

The final processing step aimed to evaluate the performance of the best FIS units. There were two parts in the final evaluation process. The first part (part A) is shown in Figure 5.24. In this part, the FIS units were evaluated using the same features (training) data which were used to generate the corresponding FIS units. In addition, the first part of the evaluation process aimed to get detailed information on the selected FIS units, such as which features were selected as Input1 and Input2, and which feature was selected as the Target Output.

Chapter 5 – ANFIS & Wavelet-Based Feature Extraction 133 In the second part of the process, the FIS units were evaluated using different feature (training) data segments. Evaluation using different features (training) data aimed to test the generalisation characteristics of the FIS units.

5.13.1 Final Evaluation Process (Part A)

Unique FIS units were selected from storage and loaded. The procedure then continued to load the corresponding training data used to generate the FIS units. At this stage, the same training and checking data sets that were used to train the evaluated FIS were used again to check whether the FIS could reproduce a Target output (i.e., energy level) that matched that in the training data. The output of energy levels predicted by the evaluated FIS was then compared using a correlation relationship toward the energy level values of the same training data. The unity correlation value would show that the energy level predicted or produced by the evaluated FIS matched the one in the training data. The FIS units and correlation / recognition factors were obtained and stored in the workspace.

Figure 5.24 FIS selection and evaluation scheme

The next step was the removal of any redundant FIS units leaving unique FIS units for evaluation. This was followed by loading the similar training / features data that had been used to generate the corresponding FIS units.

Chapter 5 – ANFIS & Wavelet-Based Feature Extraction 135 The process entered a cycle in order to process all of the selected unique FIS units. In each cycle, a routine was used to extract information from the FIS to be evaluated. The information extracted was Input1, Input2, Target Output, Level of decomposition data, Swap indices. These indices referred to the details of an FIS unit which were embedded in the FIS when it was generated during the ANFIS training process. The information (indices) extracted showed which of the seven features were selected as Input1, Input 2 and the Target Output. It also showed which decomposition level data (i.e., 1 – 9) and Swap index (i.e., 1-7) were used during the ANFIS training.

The information gathered from the FIS unit under evaluation was used to determine which level of feature/training data would be used to evaluate the FIS unit. In particular, the information of Input 1 and Input 2 was used to assign which columns in the feature (training) data would be used as Inputs in the evaluation process. For instance, if the Input 1 index was 5 and the Input 2 index was 2 then the two inputs would be taken from column 5 and 2 of the feature (training) data. Column 5 refers to Standard deviation and column 2 refers to RMS. In this case, the FIS unit would be evaluated using Standard deviation and RMS features.

The index of Target Output determined which one of the seven features was selected in relationship to Input 1 and Input 2 during the production process of the FIS in the ANFIS training process. For instance, if the index of Target Output was 1, it referred to column 1 of the feature data which was the Energy level. In this example, it was shown that the FIS under evaluation would use the Standard deviation and RMS as inputs (Input 1 and Input 2) and it would produce the Energy level as the Target Output.

Once the suitable feature data was loaded and proper inputs and output values and names were assigned using the indices information obtained, then the FIS unit was evaluated. The evaluated FIS unit then produced the intended Target Output values.

The process continued with a step to compare the similarity between the FIS Target Output and the one in the features (training) data, and the similarity was measured using a correlation factor. A unity correlation factor meant that the FIS Target Output

and the one in the features (training) data were similar. It showed that the evaluated FIS unit was 100% accurate in producing the Target Output based on the two input features supplied.

The process continued with saving the correlation factor results and plot graphs of the FIS Target Output and the Target Output of the feature (training) data. The process was then continued on its cycle if there were further FIS units to be evaluated. Once all of the FIS units had been evaluated the process ended.

5.13.2 Final Evaluation Process (Part B)

All routines in part B of the evaluation were similar to the ones in part A with the exception that the features (training) data used were different than the one used to generate the evaluated FIS through ANFIS training.

The objective of this final evaluation part was to test the performance of an FIS unit when it was supplied with a different feature data set. The different data is data that was not used to generate the evaluated FIS. This data was obtained using similar bearing fault conditions but it was acquired under different loading conditions and different shaft speeds. Part B of the evaluation process provided the ability to assess the generalisation of the evaluated FIS through the utilisation of different data segments in the evaluation process.