ANN Training from File_Graphical

The “ANN Training from File_Graphical.vi” program, as summarised in Figure 5.8, was created as a generic tool for BPFF ANN training from file. Process input and output data is supplied from a correctly formatted text (*.txt) file and the user is allowed to enter a number of parameters to affect the training strategy employed. The program then proceeds to train the ANN to minimize error, saves the architecture to file and completes an input importance analysis. The data written to file can be incorporated into other programs to run the ANN models. This program is useful for training BPFF ANN for any process because it has a generic structure and can handle multi-input, multi-output models with one or two hidden layers. It has the capacity to train the ANN multiple times using different architectures as an aid in identifying the model that produces minimum error.

The user firstly enters the 1st and 2nd layer neuron range desired and the program produces a 2D array of all of the possible architecture combinations. The user enters the file training location, the number of input and the number of outputs, and a subVI is used to extract the input parameter names, the input data array, the input min. and max., the output parameter names, the output data array, and the output min. and max. The program then separates this data into “Training” and “Testing” sets, where the training set is used only for ANN training and the testing set only for training evaluation, such as the determination of model error and input parameter importance analysis.

The training and testing data is normalised for use in the ANN models and the initial weights randomly generated into 2D arrays of the required size. ANN training is completed iteratively within a While Loop, where the training array is randomly indexed and used as input to the “Train ANN_Graphical_SubVI_.vi”, which is described below. This subVI then updates the arrays of the ANN weights and the ANN weight adjustments (for training momentum) to be used in the next iteration. The rate of convergence is controlled by the “Learning Constant” and the “Momentum Constant”, which can be modified at each iteration if desired. A facility is provided to progressively

reduce these values along an exponential curve during training to provide large weight updates initially and small adjustments towards the end of training.

A capability is included within the training While Loop to display the ANN training progress. After a user selectable number of loop iterations a function is initiated to run the ANN (“Use ANN_Single Pattern_Graphical_SubVI_.vi” detailed above) with the current weights through all of the testing data to display the results on a graph and calculate RMS error. If the error is the lowest yet seen the program will save these weights separately for later use.

Once ANN training has been completed (the number of iterations has reached the user’s limit) the program outputs the final weights to “1st Layer Weights (Last Iteration)”, “2nd Layer Weights (Last Iteration)” and “Output Layer Weights (Last Iteration)” and the minimum error weights to “1st Layer Weights (Min. Error for Display Output)”, “2nd Layer Weights (Min. Error for Display Output)” and “Output Layer Weights (Min. Error for Display Output)”. These weight arrays are saved to file in a specific format for later use. The weights are used again through the “Use ANN_Single_Pattern_Graphical_ SubVI_.vi” for all of the data to produce the ANN predictions, and the results written to file. The same process is repeated for the testing data only for error analysis.

The last stage of the program is to conduct the input importance analysis. Here the normalised testing input array is taken and one of the inputs replaced with its half magnitude value (0.5) for the entire set. This is analogous to that particular input sensor failing. The “Use ANN_Single_Pattern_Graphical_SubVI_.vi” is again used to run the data through the ANN model, and the resultant RMS error compared to the RMS error of the ANN with all working inputs. This process is repeated for all of the ANN inputs, and the results written to file and to the display.

5.4.3.1 Train ANN_Graphical_SubVI_.vi

This subVI is designed to compute the weight updates required for BPFF ANN training, and is summarised in Figure 5.9. This is achieved in three stages, the first stage is identical to the “Use ANN_Single Pattern_Graphical_SubVI_.vi” detailed above. It also has more program inputs, which include learning rate, momentum constant and 1st, 2nd and output layer weight adjustments for the previous iteration.

The second stage of the program is used to back propagate the output error through the ANN layers. The scaled error for each of the output layer neurons is calculated by comparing the ANN output to the desired output using a For Loop, before passing the resultant information to the 2nd layer neurons to calculate the 2nd layer scaled error (if applicable). This information is then passed to the 1st layer neurons for scaled error evaluation using the same method.

The third stage updates the ANN weights based on their scaled error values. This function includes the learning rate to augment training convergence speed and momentum (incorporating the 1st, 2nd and output layer weight adjustments from the previous iteration) to help avoid local minima. These final functions output the new, updated, ANN weights and the ANN weight adjustments to be used for further iterations. The ANN predicted output is provided if needed.

Figure 5.9: “Train ANN_Graphical_SubVI_.vi” basic functions

The graphical nature of this subVI made it very complicated to follow and program, as seen from the code. Indeed, the debugging phase of the development of this subVI was extremely long and there is still a possibility of small errors, which if they exist are

considered to have a small to negligible effect on training speed and accuracy. This fact highlights the complexity of graphical programming in some situations, and should be avoided in future work.