Learning or training method

Learning takes place when there is any change in the memory, W. Mathematically, it is defined as:

A

Learning = --- * 0

dt

All learning or training methods can be classified into two categories: supervised learning and

unsupervised learning. Supervised learning is a process that incorporates an external teacher and/or

global information. It requires pairs o f data consisting o f an input pattern and the correct result;

training data must contain the solution that the network is expected to provide, which is sometimes

difficult to obtain. A series o f case history, either actual or contrived, is therefore required. In

supervised learning, external control of the training process is also necessary, such as, deciding when

to stop the training. On the contrary , unsupervised learning, also referred to as self-organisation, is

a process that incorporates no external teacher and only relies upon local information and internal

control. Unsupervised learning self-organises presented data and discovers its emergent collective properties. It classifies input patterns internally and has no need for an expected result. Hence, the

data requirements for unsupervised learning are less demanding.

Learning methods form the backbone o f all ANN paradigms. In essence, learning models in ANN

are rules or procedures that tell a PE how to modify its synaptic weights in response to stimuli

(Caudill and Butler, 1989). The following sub-sections briefly describe some popular learning

algorithms.

9.4.6.1 Error-correction Learning

Error-correction learning is a supervised learning procedure that adjusts the connection weights

between PEs in proportion to the difference between the desired and computed values o f each PE in

the output layer. If the desired value o f the jth output layer PE is Cj and the computed value o f the

AW.. = a a . [ c ^ - ô . ]

where Wÿ is the memory connection strength from to bj; and

a is the learning rate, typically 0< a « 1.

The error-correction learning algorithm is adopted by one o f the most popular ANN paradigms,

Backpropagation. This ANN paradigm will be fully discussed in Section 9.5.3.

9.4.6.2 Reinforcement Learning

Reinforcement learning is similar to error-correction learning in that weights are reinforced for

properly performed actions and punished for poorly performed actions. The difference is that error-

correction learning requires an error value for each output layer PE to derive a vector o f error values,

and reinforcement learning requires only one value to describe the output layer's performance, that is, a scalar error value. The scalar success or failure value is provided by the environment. Bailey

and Thompson (1990) considers this learning algorithm as a compromise between supervised and

unsupervised training as it requires an input and only a grade or reward signal as an output. Hence,

training data requirements are less stringent than those for supervised learning.

9.4.6.3 Hebbian Learning

Hebbian learning, named after Donald Hebb (1949) who articulated the concept o f correlation

learning but not the mathematical formalisation, is the adjustment o f a connection weight according

to the correlation o f the values o f two PEs it connects. The original statement o f Hebb's law reads

as follows:

"When an axon o f cell A is near enough to excite a cell B and repeatedly or persistently takes p a rt in firing it, some growth process or metabolic change takes place in one or both cells such that A's efficiency, as one o f the cells firing B, is increased. "

The sim plest mathematical form of this learning rule, the simple Hebbian correlation, where the

The net efifect o f this process is that the strength o f the interconnection from A to B increases. This

implies that B will become more sensitive to A's stimulus after appropriate training has occurred.

9.4.6.4 Competitive and Cooperative Learning

Competitive and cooperative processes are described as ANNs with self-exciting recurrent

connections and neighbour-inhibiting (competitive) and/or neighbour-exciting (cooperative)

connections. They are essentially self-organising systems that can learn by themselves as the PEs

in the network compete for the privilege of learning. Competitive learning, introduced by Grossberg

(1972 and 1976), is a pattern o f classification procedures for conditioning inter-layer connections

in a two-layer ANN. In the simplest form where "winner takes all", competitive learning works in

concert with recall in the following manner;

1) an input pattern is presented to the input layer of PEs;

2) the input layer PEs send their activations through the to Fg inter-layer connections

to the Fg layer;

3) each Fg PE competes with the others by sending positive signals to itself (recurrent

self-excitation) and negative signals to all its neighbours (lateral neighbour-inhibition);

and

4) eventually the Fg PE with the greatest activation will be singularly active and all others

will be nullified. The Fg PE with the largest activation is called the Fg-winner.

Once the competition is complete, adjustment or conditioning o f the F^s^ to Fg-winner inter-layer

connections takes place. This means that only the weights o f the connections emanating from the

winning Fg PE are adjusted, leaving all other connections unaffected. In this respect, only the

winning PE is allowed to learn, which is a distinctive feature o f a competitive learning network. In