Parameter selection for the neural network in the algorithm

Using a 10 bin window

At first, a window with ten input neurons is used to detect the right peak. Therefore, the following set of parameters is used for the analysing:

Table A.1: Starting parameters for the neural network with 10 bin window size Parameter Value

Hidden neurons 5 Learning rate 0.01

Epochs 10

As it can be seen in table A.2, the number of hidden neurons is just the half of the number of input numbers to see if it is already enough, whereas the number of epochs is also lower than the previous ones. The reason is that now the neural network has to learn fewer parameters so that it can be assumed, that it also needs less number of epochs. To see better the differences, the results with 600 and 900 training waveforms are shown at the same time. With this one can see if the 600 waveforms are already enough for this small neural network. As done before,

XXIV Appendix A Appendix

one will first have a look at the changes in the accuracy by changing the number of hidden neurons and the learning rate. In the Figures A.5 it is interesting to see, that the graphs for the hidden neurons are not smooth any more. Instead, they show more variations, mostly at the lower numbers of hidden neurons, where one also can see the maxima. Thus it is an interesting effect that the small neural network shows now a different behaviour compared to the previous bigger one. Also for the test with 900 waveforms for training a high accuracy for a high number of hidden neurons can be seen. There in Figure A.5 (b) a maximum of 93.1%

is reached in the case in which 900 waveforms are used, which could be reached by using ten hidden neurons. Whereas in the other in Figure A.5 (a) case a maximum with five hidden neurons reaches an accuracy of 85.2%. Hence, one can already see an improvement. Next, the learning rate will be analysed which looks more similar to the previous ones. Here one can see in both cases one clear maximum peak which is for the first example in Figure A.5 (d) with 600 training waveforms with an accuracy of 95% with a learning rate of 0.0075. In the second case in Figure A.5 (e) a maximum accuracy of 95.6% is reached with a learning rate of 0.005.

Again it can be seen that the accuracy is also improving by using this approach. However, the comparison shows that there is only a slight improvement between the two training data sets.

Now one can have a look at the changes of the accuracy by increasing the number of epochs.

It is interesting to see in Figures A.6, that now also a very different pattern can be seen. It can be assumed that in Figure A.6 (a) the effect of overfitting can be seen well, by a drop in the accuracy after around 90 epochs. Because now a smaller number of weights are used, the effect appears faster. In the other case with 900 waveforms in Figure A.6 (b), the same effect can be seen. This can also be assumed to be an overfitting effect. In the first case with 600 training waveforms, the maximum occurs with 55 epochs and reaches an accuracy of 85.2%, whereas in the second case a maximum after ten epochs can be seen which results in an accuracy of 94.9%.

Next, the results for the case with 30 input neurons will be analysed.

(a) (b)

(c) (d)

Figure A.5: Accuracy of the neural network which uses ten input neurons with different numbers of hidden neu-rons with 600 (a) and 900 (c) waveforms respectively and also the learning rate with 600 (b) and 900 (d) waveforms respectively

(a) (b)

Figure A.6: Accuracy of the neural network which uses ten input neurons by increasing the number of epochs for 600 (a) and 900 (b) waveforms respectively

Using a 30 bin window

In this case, the same standard parameters than in the previous section are used which can be seen in table A.2. With this, the results are more comparable. This can be justified because the number of input neurons is just increasing with 20 neurons, compared to the 104 input neurons, it is a small change. So in a first step the changes of the accuracy by increasing the hidden neurons and the learning rate are shown. As it can be seen in the Figures A.7 the accuracy increased again and reached in all four cases values close to 95%. Thus it can already be said, that using 30 bins improves the results of the tests. Also interesting is the fact, that the graphs of the hidden neurons are now more smooth than the previous ones. However, they show a different pattern than the ones of the big neural networks. In the case of the 600 waveforms for testing in Figure A.7 (a) it can be seen that the accuracy is decreasing after a while, but in the end it is increasing again. In that case, a maximum of 30 hidden neurons with 94.9% is visible.

In the other case with 70 hidden neurons in Figure A.7 (b), an accuracy of 96.3% is reached which shows improvement. Striking is the fact, that the accuracy is now steadily increasing if the learning rate is also increasing, which is the opposite to the previous observations. In the previous cases, a clear peak could be seen which indicated the best learning rate for the neural network. In the first case in which 600 training waveforms are used and that can be seen in A.7 (c), the accuracy of 97.5% is reached with a learning rate of 0.0575. In the second case in A.7 (d), an accuracy of 96.2% is reached which a learning rate of 0.0625. Now the changes in the accuracy by increasing the number of epochs are shown as the last test. Figure A.8 shows, that as in the previous cases the accuracy is increasing at the same time as the number of epochs is increasing. However, it can be seen that the two approaches are close together. By using the smaller training set in Figure A.8 (a), an accuracy of 94.7% is reached by using either 70 or 85 epochs. This is a significant increase in the accuracy compared to the smaller network with just ten input neurons. In the case of the bigger training set in Figure A.8 (b), nearly the same accuracy with 94.6% is reached by using 95 epochs.

XXVI Appendix A Appendix

(a) (b)

(c) (d)

Figure A.7: Accuracy of the neural network which uses 30 input neurons with different numbers of hidden neu-rons with 600 (a) and 900 (c) waveforms respectively and also the learning rate with 600 (b) and 900 (d) waveforms respectively

(a) (b)

Figure A.8: Accuracy of the neural network which uses 30 input neurons by increasing the number of epochs for 600 (a) and 900 (b) waveforms respectively