Artificial Neural Network Models

Of the ideas within artificial intelligence, it is that of artificial neural networks that has had the most influence over models of classical conditioning. However, the models that use artificial neural networks are out of the scope of this thesis.

This is because it would be circular to argue that the phenomena that make up classical conditioning can be used for machine learning in a manner that is independent of implementation and then test this idea by implementing those phenomena by using a well-established method of machine learning. While this argument could also apply to the T.D. model, the ideas presented in this thesis are sufficiently close to warrant a discussion of the differences between

that model and those used in this thesis – this is not the case for artificial neural networks. As they are out of scope, but make up a large body of the work on models of conditioning, these models shall only be covered by a short overview.

The earliest work on modelling classical conditioning as an artificial neural network came from Grossberg (1969; 1974). Grossberg’s early work attempted to derive a real-time general model of learning by applying well-defined con-straints or postulates about how the neurons and the network they compose must act, each based either on observation or deductive argument. The predic-tions of the model were then compared with psychological and neuroscientific phenomena, including conditioning. Later on, Grossberg proposed the concept of a gated dipole – a sub-structure of an artificial neural network whereby the onset of an event and its offset compete to condition to various stimuli signals and drive signals that are active at the same time. When this is used as part of a larger network, the network can learn to signal the expectation of the im-minent presentation of a given stimuli based on present stimuli. An overview of this work can be found in (Grossberg, 1982). With Carpenter, Grossberg developed a highly successful artificial neural network theory known as Adap-tive Resonance Theory (ART) (Carpenter & Grossberg, 1987). This theory and accompanying neural networks are artificial neural networks that address categorisation and prediction problems.

Starting at a similar time, but independently of Grossberg, Klopf (1972) described a real-time neuronal model based on cybernetic principles. Klopf proposed that a network that is composed of components that seek to max-imise some metric¹ will itself as a whole network seek to maximise a metric (that could be the same or different to that of the component) . From such a model, Klopf argued that the phenomena of classical and instrumental con-ditioning arise as epiphenomena, along with the phenomenon of habituation.

Later, Klopf was influenced by the S.B. model (Sutton & Barto, 1981) (which in turn was influenced by Klopf’s earlier work) to create another model of con-ditioning, which Klopf named the “drive-reinforcement model” (Klopf, 1988).

In the drive-reinforcement model, Klopf mixed a variant of the S.B. model with the ideas from the Hebbian neuronal model (Hebb, 1949). In doing so, Klopf changed the Hebbian neuronal model, primarily by having the neuron correlate the derivative of delayed pre- and post-synaptic activity levels, rather than directly correlating immediate pre- and post-synaptic activity levels.

A large contribution to the field of neural-network-based models of con-ditioning has been made by Schmajuk. Schmajuk’s first contribution to the field was in conjunction with Grossberg (Grossberg & Schmajuk, 1987). This

1Klopf termed these metric-maximising components “heterostatic”. This phrase appears to be used to differentiate from the theory of homeostasis – where a component seeks to maintain a given state. However this name is too broad for how it is used – a chaotic system that neither maintains nor maximises its state could also be called “heterostatic”.

model expanded Grossberg’s gated dipole model, adding an associative learn-ing mechanism. This model was then used to predict phenomena such as blocking and overshadowing in a real-time manner.

Of the models produced by Schmajuk, the most notable are the G.S. model, the S.D. model, and the S.L.G. model. The G.S. model by Grossberg & Schma-juk (1989) is again, an augmentation of Grossberg’s earlier work, though is not an expansion of the previous collaboration between the two. The G.S. model relies on its real-time nature by adding an array of neurons that each peak at slightly different times, giving a spectrum of peaks that can then be associated with stimuli and drive signals to give timing predictions between stimuli. This model predicts both the drop in association for stimulus pairings paired with very short inter-stimulus intervals and the drop in association for stimulus pairings paired with long inter-stimulus intervals.

The S.D. model by Schmajuk & DiCarlo (1992) uses a real-time biologically plausible version of the standard three-layer, artificial neural network that uses the back-propagation algorithm. The most notable differences are firstly, the input units connect both to the hidden layer and a single output layer unit directly. Secondly all output layer units only receive input from a single lower-layer unit. Thirdly, the back-propagation is implemented in real-time as an external set of recurrent units that compare the outputs from the output layer with the expected output and uses the error difference to update the weights of the output and hidden layers. Each layer of the model was then mapped to various regions of the brain and simulations of lesions to each layer were made. The model was found to match both lesion data and able to predict a number of the phenomena of classical conditioning, including patterning and generalisation.

The S.L.G. model by Schmajuk, Lam & Gray (1996) was designed in order to model in real-time the data relating to the phenomenon of latent inhibition.

The model works due to the feedback between several different networks. An attentional system controls how fast the model of the environment adapts.

The model of the environment attempts to predict future presentations of stimuli. The error between the expectations of the model and what actually happens feeds into a novelty system. The total novelty at that particular time then feeds into the attentional system. By using total novelty, the system can predict latent inhibition because stimuli that have been encountered earlier have less novelty than those that have not. A lower total novelty feeds into the attentional system and so changes to the model are slower. Schmajuk’s more recent work develops variants of this model (Schmajuk, 2005; Schmajuk et al., 2010; Kutlu & Schmajuk, 2012).

There are a couple of other notable neural network models that are by authors that do not have a wider corpus of work on models of conditioning.

The first of these is by Pearce & Hall (1980). Pearce & Hall contended that unlike the Mackintosh (1975) model, the rate of association is related to the reliability of a conditioned stimulus to predict its own consequences. Pearce &

Hall represented this as a network of information flows that updated in a trial-level manner. The other notable model, a trial-trial-level model, is by Kehoe (1988).

Kehoe’s model is probably the first to test the standard three-layer artificial neural network against classical conditioning phenomena.