4.1 Sequence Replay With Binary Neurons
4.1.1 Assumptions of the LK2006 Model
Here we discuss the main abstractions involved in the model: discrete time to represent oscillation cycles as units of computation, coding by principal neurons only, binary character of neurons and synapses, sparseness of neuronal activation, sparseness of connectivity, and the definition of a combinatorial code based on the activation of subsets of neurons. We finally discuss how to evaluate the performance of a replay network.
4.1.1.1 Structural Assumptions
These are the constitutive hypotheses of the model.
Oscillations and discrete-time dynamics A central problem in neuroscience is the
notion of simultaneity of neural activity. An activity pattern (combination of neurons that fire, and when) is synchronous in a practical sense if it it fits within the integration time of the relevant readout structures. Here we assume that single cycles in neural oscillations provide a rhythmical basis for such an integration. Thus, one theta cycle (∼200 ms) would subserve the encoding of a pattern in the sequence and one ripple cycle (∼5ms) its replay. On first approximation, it is enough to account for those
network state changes across cycles, and to consider activity within a cycle to be effec- tively simultaneous. These assumptions allow to dispense with continuous time and write a discrete map for the dynamics of the variables of interest.
Coding by principal neurons Experiments in the hippocampus have concentrated so
far mainly in the relation between activity of principal neurons and external stimuli. It seems that the firing of pyramidal neurons correlates better with the behavioral expe- rience than that of inhibitory interneurons, and projection patterns of interneurons in cortex are highly non-specific (Fino and Yuste, 2011). The original model thus does away with inhibitory neurons; in our extension (Section 4.1.5) we do introduce an inhibitory population. Following the observation above, however, this inhibitory pop- ulation has a dynamical, not coding function. In particular the synapses of inhibitory neurons are established at random, independently of the memories loaded on the net- work.
Neuronal state Neurons, or units, are assumed to have binary state. Firing is the
state “1” and silence the state “0”. Units integrate their input linearly and compare it to a threshold θ; for simplicity all units have the same threshold. A presynaptic spike is translated into a postsynaptic membrane potential increase of one. The linear integration performed by individual units extends back just one step (i.e. takes into account activity of afferents in the previous cycle only). We shall use a vector withN
components, denotedx(t), to represent the binary firing state of the network at cycle
numbert i.e. xj(t)∈ {0,1}, j= 1, , N. In the rest of the Section we seek to write out explicitly a discrete update rule for the network state during replay with this form:
x(t+ 1) =f(x(t);px) (4.1)
wherepare the parameters that configure the network and neitherf norpare time-
dependent. We then explore its behavior in cellular simulations of replay. When including inhibition we will track an additional population of K neurons and write
the update rule in terms ofu(t)withuj(t)∈ {0,1}, j= 1, , N+K:
u(t+ 1) =f(u(t);pu). (4.2)
Memory traces Amemory traceis a pattern of activation of the network, hence also called memory pattern. Here, we assume that all such patterns involve the firing of same number of randomly chosen neurons, M (pattern size) and the silence of the
remainingF≡N−M. Thus, patterns are represented by binary arrays ξ of lengthN
withP i=1
N
ξi=M. We assume that patterns in the hippocampus represent an episodic
memory built from the confluence of perceptions from several sensory modalities (Fig. 4.1). 39 40 36 16 1 7 11 13 242526 27 28 31 33 37 Percept A hippocampus
Figure 4.1. Combinatorial Neural Code for Sensory Percepts. A multi-sensory percept is
encoded as a binary pattern of activity ofNexcitatory units,Mof them active andF=N−M inactive.
Associations An association is an ordered pair consisting of a cue pattern ξA and a
target pattern,ξB. An association is considered to be stored if approximate activation
of the cue leads to the network activating approximately the target in the next time step (symbolized ξA→ξB).
39 40 36 16 1 7 11 13 2425262728 31 33 37 20 17 2 25 29 31 37 12 33 6 8 16 5 77 26 39
Figure 4.2. Association of Memory Traces. An association of a cue patternξA(active neu- rons in red) representative of a multi-sensory perceptAwith a target patternξB(active neurons in blue) representative of a perceptB. An association can be seen as a link in a chain, or sequence, of memories.
Associations embedded in sequences A sequence of length Qis a chain of Q−1
associations (“→”),
[ξ1→ξ2→ξ3→→ξ
Q−1→ξQ].
Its successful storage requires that approximate initialization with the initial cue ξ1
leads to the network autonomously replaying the rest of the sequence with high fidelity. Precisely, an appealing property of associative memories that becomes crucial in the context of sequences, is that they can retrieve a target pattern from a damaged cue pat- tern. There is obviously a tradeoff between the allowable deterioration of the patterns (the network’s resilience) and the overall capacity of the system. By feeding the target pattern back as a the next cue pattern, here we challenge the capability of the network torepeatedlyrecall target patterns from distorted cue patterns.
Encoding and retrieval The storage of memories (encoding) and its retrieval are
assumed to happen during different network operating regimes that are mutually exclu- sive. There is experimental support for the existence of an encodingand a retrieval state in the hippocampus (Chrobak et al., 2000), each tied to one dominating hip- pocampal rhythm. Briefly, the standard view on the switching between the two modes is that the retrieval state, associated with sharp waves, kicks in when a global inhibitory brake is released and thus fails to prevent CA3 bursts that propagate to CA1 (Chrobak and Buzsáki, 1996). Encoding, assigned to the theta oscillation, is by external sequential presentation of patterns ρ= 1, , Pcausing changes in the states
of the synapses. Retrieval, or replay is the output of the spontaneous dynamics upon presentation of an initial cue, one memory per oscillation cycle,t= 1, , P.
Morphological synapses Units are connected by a morphological synapse if there exists a physical substrate between them susceptible of acquiring functional transmis- sion properties. The morphological connectivity is randomized with probability cm,
i.e. neuronifeeds into neuronjwith probability
Pr(wij= 1) =cm. (4.3)
However, morphological synapses are initially allsilent. How do they become active?
Functional synapses and storage of memories The process of storing memories
in the network consists in the activation or strengthening of particular synapses to a potentiated state. The state of potentiation is represented with the binary variables,
which equals one if the synapse transmits presynaptic spikes into contributions to the postsynaptic membrane potential, and zero otherwise. The probability that an existing synapse feeding from neuron i to neuron j is in the potentiated state grows during memorization. Accordingly, it is denoted
where the subindex ρ counts how many memories have been stored so far. We
abbreviateq=qPat the end of the storage process: it is the (final)probability of poten-
tiation. The resulting effective connectivity is calledc, corresponding to the fraction
of synapses that is operational at the end of storage out of the total number N2 of
combinatorially possible synapses, i.ec=q cm. The difference between morpholog-
ical and functional connectivity is the silent connectivitycs≡cm(1−q). The ratioq≡ c/cmis the complementary fraction of morphological synapses that remain available
for encoding of even further memories. In order to compare with the literature it is convenient to express those resources for plasticity rather in terms of the ratio of silent to potentiated synapses, r ≡ cs/c = (1 − q)/q, which is more readily esti-
mated in experiment (anatomically in CA1; Megas et al., 2001or physiologically, in CA3; Montgomery et al., 2001). In the model presented here, a synapse that has been potentiated once cannot be silenced (a possible mechanism forforgetting) and cannot be further potentiated either.
4.1.1.2 Quantitative Assumptions
The model can gain biological relevance by the judicious choice of its parameters.
Sparse activity A prominent feature of neural activity, and in particular of hip-
pocampal pyramidal neurons, is that firing is infrequent orsparse. We define the frac- tion of neurons whose activity could represent a pattern as thecoding ratiof ≡M/N
of the system. The network is expected to replay the memories at high quality, i.e. with a “firing rate” (computed over one time bin, or cycle) close to this number. The assumption of sparseness is then
sparse activity: M≪N . (4.5)
In order for the mean-field model (Section 4.2) to furnish a good approximation, both M and N have to be substantially large, i.e. 1 ≪ M ≪ N. As we shall see below, the biologically reasonable values of N = 105 and f ≃ 1% do well
enough4.1. These numbers are supported by anatomical studies (West et al., 1991 give
N≃2.5×105forregio inferior in rat hippocampus) and electrophysiological record- ings (Csicsvari et al., 2000; f ranges from 1 to 10% as the window goes from one ripple cycle to the whole oscillation, a rate of 2 Hz is equivalent to 0.01 spikes per ripple cycle assuming that all of the active neurons in a pattern, and only them, fire in a cycle).
Sparse recurrent connectivity The hallmark of the CA3 region of the hippocampus
is the abundance of recurrent connections between pyramidal cells. Recurrence being an essential ingredient of an associative network, models have often been inspired by CA3. The reference value of effective connectivity of about 5% is large (neocortex: 1 in 70 to 100, CA3 1 in 60, CA1 1 in 100; Deuchars and Thomson, 1996), but still far from full connectivity as e.g. assumed by Willshaw et al. (1969). Consequently, it is important to transpose models of memory based on densely connected networks to the more biologically realistic case of sparse connectivity. This is done here by the intro- duction of the morphological connectivitycm<1and the assumption of sparseness.
sparse connectivity: cm≪1. (4.6)
4.1. A coding ratio of 1% represents a firing rate of about 2 Hz assuming that theMneurons spike in a replay interval with the duration of a ripple cycle (about5ms), seeOscillations and discrete-time dynamicsin Section 4.1.1.
Synaptic weights The weights of synapses represent the translation factor between
a presynaptic spike and a postsynaptic effect in the form of a change of the target neuron’s membrane potential. The synaptic weights of recurrent excitatory synapses is set to one, i.e. wEE= 1because any freedom in this factor would be compensated by
a scaling in the firing thresholdθ.
4.1.1.3 Full Specification of the Cellular Model
The binary firing statexi(t+ 1)of neuroniat iterationt+ 1can be related to the firing
state of the whole network at timet,xj(t), j= 1, , N via the binary connectivitieswij
and the binary synaptic statessijusing the following deterministic update rule:
xi(t+ 1) = ( 1 if P j=1 N wijsijxj(t)> θ 0 otherwise. (4.7)
In vector form, it reads
x(t+ 1) =Θ(EEx(t)−θ), (4.8)
whereEEis the effective recurrent connectivity matrix with elementseij≡wijsij, θ is
a vector of lengthN with the identical thresholdsθas entries everywhere andΘ(x)is
the elementwise Heaviside step function with value one forx >0, and zero elsewhere.
4.1.1.4 Assessment of the Replay Functionality
How to evaluate the performance of a replay network?
Hitandfalse alarmneurons For every pattern, theMneurons which should be active
are called for shortOn neuronsorhit neurons(light red in Fig. 4.1). The remaining
F=N−M—in white—are calledOff neuronsorfalse alarmneurons, since activating one of them att is ringing the bell at the wrong time. We shall occasionally use the subindices 1 and 2 to indicate respectively On and Off neurons.
Ont neurons ≡ {j(t): ξjt= 1},
Offt neurons ≡ {j(t): ξjt= 0}. (4.9)
Activation variables In order to characterize the quality of replay it suffices to con-
sider how many of the On, or hit neurons, and how many of the Off, or false alarm neurons are active at any one time step. We call these activation variables4.2 respec- tivelyhitsmt∈0, , M and false alarmsnt∈0, , N−M.
hits mt ≡ x(t)·ξt
false alarms nt ≡ x(t)·(1−ξt). (4.10)
4.2. It may make more sense to the reader at this point to rather considersuccess variables: hits and true negativesN−nt∈0, , N−M, since then our goal of good replay corresponds to the concurrent maximization
of both quantities. The equations that we will derive below are more symmetric, however, if we stick to either activation (or inactivation variables) as opposed to success or failure variables.
From the point of view of a readout, hits represent the signal and false alarms the background, or noise, that jeopardises the identification of a memory item.
Pattern with perfect quality, m = M, n = 0
37 40 16
1 7 11 13 2425 2627 28313336 39 2 3 4 5 6 8 9 10 12 1415 171819 20 2122 232930 323435 38
Any pattern with perfect quality
A replay state x with m = M -- 3, n = 4
M = 16 N -- M = 24
Figure 4.3. Activation Variables. At top, patternξAfrom previous examples, now with all firing neurons laid out contiguously. PatternξAhere is reproduced perfectly, i.e.m=M=16, n=N−M=24. The transition to activation variables is sketched in the middle: in order to assess quality, pattern identity is not necessary. At bottom, a realization of the pattern during replay with three false dismissalsM −m= 3and four false alarms,n= 4; theM
m
ways to obtainmhits inMOn neurons, as well as the n
N−M
ways to arrangenfalse alarms inNOff neurons are statistically equivalent when deciding which neurons should fire next.
Quality Both false dismissals and false alarms in a cue pattern challenge the correct
recall of the target pattern. Deciding on a quality measure requires knowledge of the readout structure, and in particular of the relative importance of false dismissals and false alarms. A naive choice is
Γt≡mt/M−nt/F. (4.11)
This definition emphasizes the false dismissals in the regime of interest of low coding ratios,f=M/N≪1. Note that−16Γt61. A predominance of negative values would
induce us to switch our perspective and observe silences as responsible for coding, instead of discharges.
Figure 4.4. Populations of On and Off Neurons. During successful replay the majority of
the On neurons should stay active (hits) and the majority of the Off neurons should be silent (correct dismissals). The inactive (white) neurons in the ON group are false dismissals (gt).
The inactive (white) Off neurons are true negatives.