Gestures used for the Interactive Display

A basic neuron model as shown in Fig 2.20 can be described by a function that calculates the output as a function of a number of inputs to it. The basic idea behind the entire neuron model, including the activation functions illustrated below, has been adopted from (Anderson, 2003).

Figure 2.20: Mathematical Model of a Neuron.

- 83 - The output of the neuron is given by

𝑓 𝜑 = 𝑓( ^𝑁_𝑖=0⁰ 𝑤_𝑖𝑎_𝑖) 2.10

Where: w₀a₀ is the threshold value (polarization), f φ is the neuron activation function, φ is the summation output signal and y is the neuron output.

𝜑 = 𝑾^𝑻𝑨 2.11

Where: 𝑾 = 𝑤₀𝑤₁… 𝑤_𝑁0 , 𝑨 = 𝑎₀𝑎₁… 𝑎_𝑁0 ^𝑻 2.12 The activation function acts as a squashing function, such that the output of a neuron in a neural network is between certain values (usually 0 and 1, or -1 and 1). An activation function decides how powerful the output from the neuron should be, based on the sum of its inputs. Depending upon the application‘s requirements, the most appropriate activation function is chosen. The activation function f (φ) can be in different forms. A few of which are illustrated below:

▪ Threshold (or Step) Function

This takes on a value of 0 if the summed input is less than a certain threshold value (φ), and the value 1 if the summed input is greater than or equal to the threshold value.

𝑓 𝜑 = 1, if φ ≥ 0 0, if φ < 0

- 84 - Figure 2.21: Step activation function

▪ Piece wise linear function

This function again can take on the values of 0 or −1, but can also take on values between that depending on the amplification factor in a certain region of linear operation.

𝑓 𝜑 =

1, for φ > 1

−1, for φ < −1 𝜑, for − 1 < 𝜑 < 1

Figure 2.22: Piece wise linear activation function.

▪ Sigmoid bipolar function

This function can range between 0 and 1, but it is also sometimes useful to use the -1 to 1 range.

An example of the sigmoid function is the hyperbolic tangent function

𝑓 𝜑 = tanh 𝛽𝜑 =^{1 − 𝑒}_{1 + 𝑒}^−2𝛽𝜑_−2𝛽𝜑

- 85 - Figure 2.23: Bipolar activation function.

▪ Sigmoid unipolar function

This function can range between 0 and 1 and in this work the Sigmoid activation function has been used since the emphasis is that the neural network output should be 0 or 1 (no or yes).

𝑓 𝜑 = _{1+ 𝑒}¹_−𝛽𝜑

Figure 2.24: Sigmoid unipolar activation function.

Based on the way the neurons are interconnected in a model, neural networks can be broadly classified into two types namely feed-forward and feed-back networks. As the name suggests, feedback networks unlike feed-forward networks have a feedback connection fed back into the network along with the inputs. Due to their simplicity and the existence of a well-defined learning algorithm, only feed-forward networks have been used in this dissertation for the simulation and hence the application is presented in the upcoming sections.

- 86 -

2.17.1 The Feed-forward Networks

Feed-forward networks are the simplest neural networks where there is no feedback connection involved in the network and hence the information travel is unidirectional (El-Sharkawi, Niebur, 1996). A feed-forward network with 𝑎_N0 input and 𝑦_NR output signals is shown in Fig 3.14. The computation process in the ith layer can be described by the following (2.13)

𝒑^(𝑖)= 𝑓^(𝑖) 𝒘^(𝑖)𝒈^(𝑖−1) 2.13

Where 𝒑^(𝑖) = 𝑝₁^(𝑖)𝑝₂^(𝑖)𝑝₃^(𝑖) … 𝑝_𝑁^(𝑖)_𝑖 ^𝑻 is the fault signal vector at the output of the ith layer.

And 𝑾^(𝑖) =

𝑤₁₀^(𝑖) 𝑤₁₁^(𝑖) … 𝑤_1𝑁^(𝑖)_𝑖−1 𝑤₂₀^(𝑖) 𝑤₂₁^(𝑖) … 𝑤_2𝑁^(𝑖)_𝑖−1

⋮ 𝑤_𝑁^(𝑖)_𝑖0

⋮ 𝑤_𝑁^(𝑖)_𝑖1 ⋱

…

⋮ 𝑤_𝑁^(𝑖)_{𝑖𝑁𝑖−1}

is the weighing matrix between the (i-1)th and

the ith layer.

𝒈^(𝑖−1)=

𝐴 𝑓𝑜𝑟 𝑖 = 1 1

𝒑^(𝑖−1) 𝑓𝑜𝑟 𝑖 = 2,3, … , 𝑅 2.14

A is the vector containing the input signals,𝒇^(𝑖) is the activation function of the neurons in the ith layer and p is the number of processing layers. All the neurons in a particular layer are assumed to be similar in all aspects and the number of hidden layers can be more than one and is usually determined by the purpose of the neural network. The output of the processed neural network is represented by the output vector:

𝒚 = 𝒑^(𝑹)= [y₁y₂y₃ … y_NR]^𝑇 2.15

- 87 -

Figure 2.25: Structure of a two-layered feed-forward network.

2.17.2 The Training Process for the HVDC Neural Network Controller

Two important steps in the application of neural networks for any purpose are training and testing. The first of the two steps namely training of the chosen neural networks is dealt with in this section. Training is the process by which the neural network learns from the inputs and updates its weights accordingly. In order to train the neural network, a set of data called the training data-set is needed which is a set of input output pairs fed into the neural network.

Thereby, the neural network is taught what the output should be, when that particular input is fed into it. The ANN slowly learns the training set and slowly develops an ability to generalize upon this data and will eventually be able to produce an output when a new data is provided to it.

During the training process, the neural network‘s weights are updated with the prime goal of minimizing the performance function. This performance function can be user defined, but usually

- 88 -

feed-forward networks employ Mean Square Error as the performance function and the same is adopted throughout this work.

As aforementioned, The input to the suggested ANN controller is the current reference 𝐼_𝑟𝑒𝑓 and measured dc current 𝐼_𝑑𝑐 and its output is the firing angle. The error between 𝐼_𝑑𝑐 and 𝐼_𝑟𝑒𝑓 is used to adjust the weight of the ANN according to the delta rule. The speed of the controller and the system stability will depend on the learning rate 𝜂 and the momentum 𝜇 used in adjusting the weights of the ANN.