First order gradient methods

Backpropagation

The backpropagation (BP) algorithm is the most well known and widely used among other learning algorithms described in the literature. It is based on steepest-descent techniques extended to each of the layer in the network by the chain rule. Hence, the algorithm computes the partial derivative @E(k )

@W(k 1)

, of the error function with respect to the weights. Suppose we want to model a nonlinear plant using a NN

y

e

NN

_m

Plant

u

TDL TDL TDL

y

_m p

Figure 3.4: Multilayer feedforward neural network modelling a nonlinear

dynamic plant.

(see Figure 3.4). The NN model output is y

m. The error function is defined as E(k)= 1 2 [y p (k) y m (k)] 2 wherey

pis the plant output and e m (k)=y p (k) y m (k). The objective is to minimize the error functionE(k)by taking the error gradient with respect to the parameters or weight vector, sayW, that is to be adapted. The weights are then updated by using

W(k)=W(k 1)+(

@E(k) @W(k 1)

) (3.5)

whereis the learning rate and

@E(k) = e m (k) @y m (k)

46 = e m (k) @O o (k) @W(k 1) The network output isO

o (k)=y

m

(k), in this example.

This standard BP algorithm is simple to implement and computationally less complex than other modified forms of BP algorithms. Nevertheless, it has some disadvantages such as:

1. Slow convergence speed.

2. Sensitivity to initial conditions [124]. 3. Trapping in local minima.

4. Instability if learning rate is too large.

Despite the above disadvantages, it is popularly used. There are numerous extensions to improve the BP algorithm. Some of the more common techniques are as follows.

Backpropagation with momentum

The basic improvement to a BP algorithm is to introduce a momentum term in the weights updating equation [201]. Hence, the weights updated by backpropagation with momentum (BPM) are

W(k)=W(k 1)+(

@E(k) @W(k 1)

)+W(k 1) (3.6)

where the momentum factoris commonly selected between0<<1. Adding the momentum term improves the convergence speed and helps the network from being trapped in a local minimum.

A modification to equation (3.6), such as

W(k)=W(k 1)+(

@E(k) @W(k 1)

)+W(k 1)+W(k 2) (3.7)

was proposed by Nagataet al. [168], where is a constant value decided by the user. Nagataet al. [168] claimed that the term reduces the possibility of the network being trapped in the local minimum. However, this seems to be repeating the role of theterm. Hence, the advantage of adding theterm is not clear at the moment.

Delta-bar-delta

The delta-bar-delta (DBD) learning rules use an adaptive learning rate to speed up the convergence. The adaptive learning rate adopted is based on a local optimization

47

method. It is an improvement from the delta-delta learning rule. This technique uses gradient descent for the search direction, and then applies individual step sizes for each weight. This means that the actual direction taken in weight space is not necessarily along the line of the steepest gradient [110].

If the weight updates between consecutive iterations are in opposite directions, the step size is decreased, otherwise it is increased. This is prompted by the idea that if the weight changes are oscillating, the minimum is between the oscillations, and a smaller step size might find that minimum. The step size may be increased again once the error has stopped oscillating.

Let ij

(k)denote the learning rate for the weightw ij (k), then ij (k+1)= ij (k)+ ij (k) (3.8) and ij (k)is as follows: ij (k)= 8 < : ij (k 1) ij (k)>0 ij (k) ij (k 1) ij (k)<0 0 otherwise (3.9) where ij (k) = @E(k ) @wij and ij (k) =(1 ") ij (k 1)+" ij (k 1). The"is a positive constant. ; and"are parameters specified by the user. The quantity

ij

(k)is basically an exponentially decaying trace of gradient values. When the andare set to zero, the learning rates assume a constant value as in a standard BP algorithm.

Results cited by Jacobs [110] indicate that using momentum along with the DBD algorithm can enhance performance considerably. However, it can also make the search diverge wildly; especially ifis moderately large. The reason is that mo- mentum ‘magnifies’ learning rate increments and quickly leads to inordinately large learning steps. One possible solution is to keep thefactor very small, but this can easily lead to slow increase inand little speedup.

Simulations have also shown that the DBD algorithm is very sensitive to small variations in the value of its parameters, especially. To overcome these problems the decoupled momentum is suggested.

Second order gradient methods

Second order gradient methods make use of second derivatives of the error in the weight space. The key to second order gradient methods is the Hessian matrixH. The Hessian matrix is a matrix of second derivatives ofEwith respect to the weights

W. H(k)= @ 2 E(k) 2 (3.10)

48

The Hessian matrix contains information about how the gradient changes in different directions in weight space. It answers the question: ‘How does the gradient change if I move off from this point in that direction?’.

Newton

The Newton method weights update are [58]

W(k)=W(k 1)+[H(k)] 1

@E(k) @W(k 1)

(3.11) However, the Newton method is not commonly used because calculating the Hessian matrix is computationally expensive. Furthermore an Hessian matrix may not be positive definite at every point in the error surface. To overcome the problem, several methods have being proposed to approximate the Hessian matrix.

Gauss-Newton

The Gauss-Newton method produces annnmatrix which is an approximation to the Hessian matrix, having elements represented by

H(k)= T (k) (k) (3.12) where (k)= @e(k ) @W(k 1)

[145]. However, the Gauss-Newton method may still have ill-conditioning ifH(k)is close to or is singular.

Levenberg-Marquardt

The Levenberg-Marquardt (LM) method overcomes this difficulty by including an additional termwhich is added to the Gauss-Newton approximation of the Hessian matrix giving

H(k)= T

(k) (k)+I (3.13)

whereis a small positive value andI is the identity matrix.could also be made adaptive by having = E c (k)>E c (k 1) E c (k)<E c (k 1)

where is a value decided by the user. Notice that whenis large, the algorithm becomes a backpropagation algorithm with learning rate 1

, and whenis small, the algorithm becomes Gauss-Newton.

49

Quickprop

Fahlman [76] proposed the Quickprop method using heuristics based loosely on the Newton method. Fahlman highlighted that the Quickprop algorithm is based on two risky assumptions:

The error vs weight curve for each weight can be approximated by a parabola.

The change in the shape of the error curve, as seen by each weight, is not affected by all the other weights that are changing at the same time.

Hence, Quickprop implicitly assumes that the Hessian matrix is diagonal. The weights update of the Quickprop algorithm,W(k), is as follows:

W(k)= (k) (k 1) (k) W(k 1) where (k)= @E(k ) @W(k 1)

. Fahlman claimed that when applied iteratively Quickprop is very efficient.

Conjugate gradient decent

The idea behind conjugate gradient decent (CGD) is to choose search directions that complement one another. This is to avoid the possibility of ‘undoing’ the minimiza- tion of previous iterations, by choosing appropriate search directions.

Assume that the error surface is quadratic in weight space.

E(k)=C T W(k 1)+ 1 2 W T (k 1)H(k)W(k 1) (3.14)

whereC is a constant matrix andH is a Hessian matrix, constant throughout the weight space. The search direction at iterationkis denoted byv(k).

Set the first direction,v(1), to minus the gradient

@E(1) @W(0)

. Set subsequent di- rections using:

v(k)=

@E(k) @W(k 1)

+(k)v(k 1) (3.15)

where(k)is a scalar value like an adaptive momentum term. may be defined in a number of ways, each of which produces conjugate directions. The Polak-Ribiere rule foris [99] (k)= [ (k) (k 1)] T [ (k)] T (3.16)

50

The Hestenes-Stiefel rule foris [163]

(k)= [ (k 1) (k)] T [ (k)] v(k) T [ (k 1)] (3.17)

CGD will reach the minimum of a quadratic error surface in, at most, as many steps as there are dimensions of the weight space. For non-quadratic error surfaces, the minimum may not have been reached after this number of steps. Instead, the algorithm is restarted by settingto zero for a step, and the procedure continues as before.

Conjugate gradients give only a search direction, not a step size. Furthermore, the power of conjugate gradients is only apparent if the error is minimized along the current search direction.

In document Intelligent Systems Fusion, Tracking, and Control GeeWah Ng pdf (Page 70-75)