As seen so far, the traditional STDP plasticity rule provides an interesting bio-inspired mechanism for unsupervised learning in neural circuits. The use of STDP as an underlying plasticity mechanism has been extensively and successfully tested in several experimental setups [20, 32, 37, 48]; demonstrating that with STDP, a neuron is able to discriminate through the time dimension between the real contributors to its activation and the noisy neurons whose firing patterns are not synchronized or correlated with its own activation. In other words, STDP allows a neuron to decide which presynaptic neurons are worth listening to and which ones should be given less priority or possibly be completely ignored [49]. Moreover, through the combination of associative network architectures as described above, a SNN system implementing STDP is able to create associations between multi- modal stimuli (i.e., from different types of sensory inputs). This allows the engineering of sophisticated neural circuits that can be applied to control the behaviour of autonomous systems (as will be demonstrated in the next chapters).
Nevertheless, in spite of these desirable features, STDP also has some limitations that need to be considered when implementing neural circuits on a greater scale. This thesis will focus on two known issues found in STDP based systems:
1. Runaway dynamics.
2. Timing as single source of spike information.
3.4.1
Runaway dynamics
Out of control or runaway dynamics [17] refer to the positive feedback loop that emerges from the mechanisms that rule the induction and amplification of synaptic changes in hebbian based plasticity. As already explained, in order for the induction of synaptic change to take place, it is required that the presynaptic spikes arrive shortly before the activation of the postsynaptic neuron. These presynaptic spikes elicite excitatory postsynaptic potentials that keep adding to the postsynaptic membrane potential as they arrive. Therefore, the presynaptic spikes coming from a synapse with stronger synaptic efficacy will have a higher probability of bringing the membrane potential of the postsynaptic neuron to reach
its firing threshold and make it fire. Consequently, the already stronger synapse that contributed to the activation will be reinforced and hence its probability of activating again the postsynaptic neuron is increased. Over time, the synaptic efficacy (or weight) will reach its maximum value or ceiling.
A similar situation occurs with the long term depression of synapses. The synapses with lower synaptic efficacy have a lower probability of activating a postsynaptic neuron given that the elicited postsynaptic spikes are mostly not large enough to make a significant contribution to the postsynaptic membrane potential and to elicite a postsynaptic action potential. Therefore, over time the already weaker synapses tend to be depressed to a minimum or floor value as their spikes do not have the chance to activate the postsynaptic neuron.
One of the critical issues with the runaway dynamics in hebbian plasticity is that not only does it affect the system at the level of individual synapses, but it creates chains or cascades affecting the entire neural circuit [50]. For instance, as it is observed in some of the experimental setups carried out in this thesis, in neural circuits implementing associative and classical conditioning architectures, the stronger over-potentiated synapses originating from conditioned sensory (neutral) neurons gain the ability to activate their associated motoneurons (behaviour) acting as unconditioned sensory neurons. Whilst this is an expected behaviour in a conditioned system, the issue arises when the conditioned neurons in addition to acting as stronger activators of motor behaviour acquire the ability to reinforce other neutral sensory neurons creating new conditioning behaviour between neutral stimuli.
As described so far, runaway dynamics may render the behaviour of a neural circuit unpredictable or unstable. Moreover, it may have a negative impact on the computational abilities of the system [17].
3.4.2
Timing as single source of spike information
As mentioned before, a neuron implementing STDP is able to identify the coincidences between its own firing time and the firing time from presynaptic neurons. However, apart from the timing of the spikes there is no other information about the event that triggered
the firing of the incoming spike or about the neuron that originated the incoming spike. For instance, the standard STDP implementation neglects any information regarding reward, success, punishment and novelty [9].
This limitation is not only specific to STDP but it is a general characteristic of hebbian- based plasticity systems. The reason for this is that the canonical hebbian postulate is solely focused on the relationship between pre- and post-synaptic activity whilst neuromodulation is completely out of the scope of its implementation.
From a neuro-engineering perspective, having a plasticity system that is solely based on timing information brings along some technical challenges that arise when designing and implementing an artificial neural system. One of these key challenges is the undesirable reinforcement of a synapse resulting from the hebbian coactivation between two temporar- ily coinciding spikes. This is an issue that emerges from the lack of a feedback signal in response to the synaptic change (e.g. reward, punishment, error). The missing feedback in response to the change in the system is an inherent characteristic of unsupervised learning systems that in certain cases requires complex workarounds in order to guide or regulate the learning of the system.