Neural sampling — sampling with spiking neurons

The model of sampling with still abstract but spiking neurons was first described by Buesing et al. [2011]. In their work, the authors propose a model where a network of abstract neurons samples from the probability distribution of a Boltz- mann machine. The novelty of their work was that they explicitly included the spiking behavior and the refractory mechanism of the neurons. Further, with the mathematical equivalence to Boltzmann machines, this model not only provides a mechanistic model for sampling in the brain but it also connects machine learning to computational neuroscience. Hence, existing methods, for example annealing, can serve as inspiration to tackle questions in the neuroscience. We can motivate the model with the following observations:

• In biological neurons [Gerstner and Kistler, 2002b], after the production of an action potential the generation of another spike is prohibited for a certain

τrefrefractory time. Hence, it is tempting to model the neuron as a two-state

system. Immediately after spike the neuron is in the refractory state and otherwise the membrane potential freely evolves while spike generation is possible.

• We can see neurons in the sense of a linear non-linear response model [Si- moncelli et al., 2004]: The neuron sums up/integrates the incoming input linearly (for current based synapses) and produces a stochastic output via a non-linear transfer function. The concept of linear summation is similar to the idea of abstract membrane potential in Boltzmann machines (equa- tion (3.4)) and the non-linear response to the sigmoid probability function to be in state 1 (section 3.2.1).

• The interaction between neurons takes place via all-or-nothing action poten- tials, and the generation of these action potentials is strictly coupled to the refractoriness. This mechanism closely resembles the Glauber-dynamics of sampling from a Bolzmann machine.

In the setup of neural sampling, we treat the time as a discrete variable increasing in time-steps of dt, such that the time is represented by a natural number t ∈ N. In

this sense the model is close to the Glauber dynamics of Gibbs-sampling because the model introduces time not as a naturally continuous variable but rather by relating the time-steps to characteristic time-constants in neural dynamics. We define the state zk(t)of neuron k at time t as 1, or on-state, if the neuron has spiked

in the last τref ∈ Z+time-steps and 0, or off-state, otherwise. τrefis the refractory

time-constant of the neuron. In equation: zk(t) =

{

1 if t<ts+τref ,

0 otherwise . (3.17)

Here is ts is the timestamp of a spike generated by the k-th neuron. The con-

3.2 Theoretical background — sampling with neurons Boltzmann Machine (section 3.2.1). For the mathematical description of the dy- namics we introduce the auxiliary variable ζk(t) ∈Z with 0≤ζk(t) ≤τref. ζk(t)

plays the role of a countdown variable of neuron k during its refractory time. The relation between ζkand zkis given by

zk(t) =

{

1 if ζk ≥1 ,

0 if ζk =0 .

(3.18)

With the help of the auxiliary variable ζk now we can define the dynamics of

the model as a first order Markov process (a description using directly zkwould

not be Markovian):

• If a neuron is refractory, i.e. ζk(t) ≥ 1, then the auxiliary variable counts

down:

ζk(t+1) =ζk(t) −1 (3.19)

• If a neuron is in the off-state or at the end of the refractory time in the on state, i.e. ζk ∈ {0, 1}, then the neuron can fire with the probability:

p(ζk(t+1) =τref|ζk ∈ {0, 1}) =σ(uk−ln(τref)) , (3.20)

where σ(·)is the logistic function

σ(x) = 1

1+exp(−x) , (3.21)

and ui is the abstract membrane potential given by Equation (3.4). If the

neuron fired, then ζk is set to τref. The state transition to spike from ζk =1

is necessary, so that continuous on-states are possible. The factor ln(τref)

corrects for over-counting the on-state, for cases when τref >1. If no spike

occurred then we set ζk =0 and zk =0.

• In the original publication, the neurons are updated sequentially just as in Gibbs sampling. If all the neurons are updated at the same time, mimicking a continuous time, then the model still provides reasonable results in practical tasks.

The above process is compactly depicted in figure 3.5. According to the dynam- ics, we can interpret the interaction between the neurons. When a neuron spikes, then it generates a rectangular PSP in its post-synaptic partners, because the effect of the pre-synaptic neuron on the membrane potential of the post-synaptic neurons is flat. The synaptic time-constant, that is the length of the PSP, exactly matches the refractory time of the pre-synaptic neuron. The interaction between any two neurons is strictly symmetric as the model is tailored for Boltzmann Machines. Buesing et al. [2011] proved that the described dynamics of the network constitutes an exact sampling from the probability distribution of the underlying Boltzmann Machine, and they showed that the model features biological phenomena like switching between modes when an ambiguous picture is presented to an observer (binocular ambiguity).

3. Bayesian inference on BSS-1

Figure 3.5: Dynamics of neural sampling: After the neuron spikes, the auxiliary variable ζkcounts down during the refractory time. Red circles represent on-states

and the black circle represents an off-sate. After the end of the refractory period, i.e.

ζk ∈ {0, 1}the neuron can spike with the probability of pspike =σ(uk−ln(τref))

(with σ(·) defined in Equation (3.21)). It can be shown that this Markov chain

samples exactly from the probability distribution of a BM. Image taken from Buesing et al. [2011].