STDP provides the substrate for igniting synfire chains by spatiotemporal input patterns

STDP provides the substrate for igniting synfire chains by spatio-temporal input patterns

Authors

Ryosuke HOSAKA†⋆_{, Osamu ARAKI}§_{, and Tohru IKEGUCHI}†

Affiliations / Address

†_{Graduate School of Science and Engineering, Saitama University}

255 Shimo-Ohkubo, Sakura-ku, Saitama 338-8570, Japan Phone,Fax: +81-48-858-9105

§_{Department of Applied Physics, Tokyo University of Science}

1-3 Kagurazaka, Shinjuku-ku, Tokyo 162-8601, Japan Phone: +81-3-3260-4271

⋆_{Present address: Aihara Complexity Modelling Project, ERATO, JST}

3-23-5-201 Uehara, Shibuya-ku, Tokyo 151–0064, Japan Phone: +81-3-5452-0371, FAX: +81-3-5452-0377

E-mail: [email protected] Key words

spike-timing-dependent plasticity, spatiotemporal pattern, synchrony, spiking neural network, synfire chain, phase coding

Abstract

1

Spike-timing-dependent synaptic plasticity (STDP) which depends on the temporal difference 2

between pre- and postsynaptic action potentials is observed in the cortices and hippocampus 3

Although several theoretical and experimental studies have revealed its fundamental aspects, 4

its functional role still remains unclear. To examine how an input spatiotemporal spike pat-5

tern is altered by STDP, we observed the output spike patterns of a spiking neural network 6

model with an asymmetrical STDP rule when the input spatiotemporal pattern is repeatedly 7

applied. The spiking neural network comprises excitatory and inhibitory neurons that exhibit 8

local interactions. Numerical experiments show that the spiking neural network generates a 9

single global synchrony whose relative timing depends on the input spatiotemporal pattern and 10

the neural network structure. This result implies that the spiking neural network learns the 11

transformation from spatiotemporal to temporal information. In the literature, the origin of 12

the synfire chain has not been sufficiently focused on. Our results indicate that spiking neural 13

networks with STDP can ignite synfire chains in the cortices. 14

1

Introduction

1

Recent experimental and theoretical studies indicate a possible information processing mech-2

anism based on the precise spike timings of the neurons in the brain, which include synchronous 3

firings or the temporal correlation of spikes (Gray et al., 1989; Riehle et al., 1997; Aertsen et al., 4

1989, 1994; Vaadia et al., 1995; de Oliveira et al., 1997; Fujii et al., 1996; Gerstner et al., 1996; 5

Masuda and Aihara, 2004). With regard to synaptic plasticity, it has been reported that the 6

modification of synaptic strength depends on the temporal difference between the pre- and 7

postsynaptic action potentials in the cortices and the hippocampus this is termed spike-timing-8

dependent synaptic plasticity (STDP) (Abbott and Nelson, 2000; Markram et al., 1997; Bell 9

et al., 1997; Bi and Poo, 1998; Zhang et al., 1998; Debanne et al., 1998; Nishiyama et al., 2000; 10

Froemke and Dan, 2002). Based on the STDP learning rule, it is considered that the precise 11

timings of the spikes of the pre- and postsynaptic neurons are encoded into the patterns of the 12

synaptic weights (Gerstner et al., 1996; Song et al., 2000; van Rossum et al., 2000; Rubin et al., 13

2001; G¨utig et al., 2003). 14

Several studies have reported the functional roles of the STDP in learning, memory (Mat-15

sumoto and Okada, 2002; Levy et al., 2001; Rao and Sejnowski, 2001; Song and Abbott, 2001), 16

and the transformation of spike patterns. Generally, the transformation of spike patterns is 17

classified into three levels: single-neuron, two-neuron, and neural network levels. 18

• Single-neuron level: In the case that a postsynaptic neuron receives random inputs from 19

a considerable number of presynaptic neurons through the STDP synapses, although the 20

interspike interval (ISI) of this neuron is highly irregular, the frequency of the output spikes 21

is more stable than the change in the input spike frequency (Song et al., 2000; Rubin et al., 22

2001). 23

• Two-neuron level: Pulse-coupled Hodgkin-Huxley neurons have a tendency to synchronize 24

through the STDP synapses (Zhigulin et al., 2003). 25

• Neural network level: The STDP learning and realistic axonal conduction delays make 26

neurons self-organize into groups and repeatedly generate spike patterns with a millisec-27

ond spike-timing precision (Izhikevich et al., 2004). A neural network with an inhibitory 28

neuron that suppresses other neurons transforms the applied periodical spatiotemporal 29

patterns into periodical firing patterns (Gerstner et al., 1993; Yoshioka, 2001). A recur-30

rent neural network with STDP synapses activated by random external stimuli generates 31

distributed synchronies (Levy et al., 2001). The distributed synchrony is a cyclic firing 32

pattern expressed by some neural clusters. The neurons in each cluster simultaneously 1

fire, which induce the next cluster to fire. A global synchrony, on the other hand, is a 2

unison of the firings of most of the neurons included in a neural network. In contrast to 3

distributed synchrony, global synchrony does not evoke other synchronies. Masuda and 4

Aihara (2004) reported an interesting finding for distributed synchrony: The higher the 5

shared connectivity in the network, the greater is the number of synchronous clusters that 6

tend to occur. 7

These studies showed that the STDP controls the output spike rates and generates synchronous 8

spikes or spike patterns depending on the precise spike timings of the pre- and postsynaptic 9

neurons. 10

The spike timings of the neurons in a neural network are determined by the interactions of 11

the neurons. In addition, external stimuli from other networks, for example, sensory organs, 12

thalamus, or other cortical networks, strongly affect the spike timings. In previous studies, 13

most of the external stimuli are assumed to be random and having no specific structure (Levy 14

et al., 2001; Masuda and Aihara, 2004; Izhikevich et al., 2004). However, it can be reasonably 15

expected that the external stimuli have some structures, for example, spatial, temporal, or 16

both. Thus, it is important to analyze the output spike patterns of the neural network with 17

STDP synapses activating it with spatiotemporal inputs. Although some studies considered the 18

case of an external stimulus with a spatiotemporal structure, they assumed only one inhibitory 19

neuron that globally suppresses all of the neurons under consideration (Gerstner et al., 1993; 20

Yoshioka, 2001). We conducted the simulations with spiking neural network model which was 21

comprised partially connected excitatory and inhibitory neurons, the inhibitory neurons had 22

local interactions. 23

As a result, the neural network generates a global synchrony. In other words, the input 24

spatiotemporal pattern is transformed into the global synchrony by the neural network and the 25

STDP synapses. If a synchrony occurs in the brain, it stably propagates through feedforward 26

layered neural networks (the synfire chain) (Diesmann et al., 1999; Reyes, 2003; Abeles, 1991). 27

However, the source of the synfire chain has been assumed to be the stochastic spikes that occur 28

almost simultaneously in divergent/convergent networks. The occurrence of the synfire chain 29

in such a model is probabilistic. Our result suggests that a neural network with STDP could be 30

a deterministic source of synfire chains. With an assumption that the spiking neural network 31

with STDP learns a pattern whose spatiotemporal structure represents some information, our 32

results suggest that the global synchrony ignites the synfire chain and announces that some 33

information has been applied. 1

2

Methods

1

2.1 The neural network model 2

We used a neural network model comprising leaky integrate-and-fire (LIF) neurons. The mem-3

brane potential of an LIF neuron is described by 4

CdVj

dt = g(Vrest − Vj) + Ij(t), (1)

where C = 1µF/cm2 is membrane capacitance; Vj, membrane potential of the j-th neuron;

5

g = 0.05mS/cm2_{, membrane resistance; Vrest, resting potential (−70mV); and I}j(t), the sum

6

of the recurrent and external synaptic current inputs. With these parameters, the time constant 7

of the membrane potential is calculated as τm = C/g = 20ms. When the membrane potential

8

exceeds a threshold of −54mV, the neuron generates an action potential and the membrane 9

potential is reset to the reset potential, −90mV. The neural network was composed of 800 10

excitatory and 200 inhibitory neurons, and the neurons were connected through synapses. 11

Without the inhibitory neurons, the excitatory neurons would infinitely excite each other, and 12

the network would burst. 13

The probability of forming at least one synapse between two pyramidal cells in the cortex 14

is generally estimated as being between 0.1 and 0.3 (Liley and Wright, 1994; Braitenberg and 15

Sch¨uz, 1991). Thus, from this anatomical viewpoint, we used a partially connected neural 16

network in this study. The percentage of connections was 20%, excluding the self-feedback 17

connections. That is, each neuron had 200 synapses. We modeled the excitatory postsynaptic 18

currents and inhibitory postsynaptic currents of the postsynaptic neurons by using the Dirac 19

delta function δ(t). 20

The input to the neuron can be described as follows: 21 Ij(t) = N X i=1 Qiwij X l δ(t − tl_i− dij) + IEXT(t), (2)

where N is the number of neurons; wij, the strength of the synaptic connection from the i-th

22

neuron to the j-th neuron; tl

i, the timing of the l-th spike of the i-th neuron; and dij, the delay

23

time of the spike transmission from the i-th neuron to the j-th neuron. Further, we assumed 24

that Qi = QEXC = 11.25nA/cm

2

for the excitatory synapses and Qi = QINH = 13.5nA/cm

2

25

for the inhibitory synapses. Each neuron received a suprathreshold stimulus as the external 26

input, IEXT(t). If the external input exists, IEXT(t) = 0.2µA/cm2, otherwise IEXT(t) = 0.

27

The strength of the synapse connections, wij, from the excitatory neurons was initially set to

28

0.45, while those from the inhibitory neurons were set to −1. Based on the result of Bi and 29

Poo (1998), we applied the STDP rule (described below) only to the excitatory-to-excitatory 1

connections, while the other connections were fixed. Although we also attempted simulations 2

that include the STDP learning of excitatory-to-inhibitory connections, the result was found to 3

be qualitatively the same as that in the case without this learning. We used the Euler algorithm 4

with a time step of 0.1 ms for the numerical simulations. In this study, synaptic propagation 5

delays were entirely axonal. The period of the synaptic delays was randomly selected to be 6

between 0.1 and 3 ms. 7

2.2 The STDP learning rule 8

Several studies have reported on the window functions of the STDP learning rule (Song et al., 9

2000; van Rossum et al., 2000; Rubin et al., 2001; G¨utig et al., 2003). Song et al. (2000) 10

modeled a synaptic modification rate that is independent of the present synaptic weight. This 11

modeling is called an additive STDP rule. With this learning rule and random inputs, the 12

distribution of the synaptic weights becomes bimodal. If the synaptic modification is assumed 13

to depend on the present synaptic weight, a unimodal distribution appears (van Rossum et al., 14

2000; Rubin et al., 2001; G¨utig et al., 2003). This modeling is called a multiplicative STDP 15

rule. To cover both rules, we used the window function proposed by G¨utig et al. (2003). With 16

this window function, the extent of synaptic weight modification (∆w) decreases exponentially 17

with the temporal difference (∆t) between the arrival of the presynaptic action potential and 18

the occurrence of the postsynaptic action potential: 19

∆t = tpre+ dpre,post− tpost (3)

where tpre is the spike time of the presynaptic neuron; dpre,post, the delay time of the spike

20

transmission from the presynaptic neuron to the postsynaptic neuron; and tpost, the spike time

21

of the postsynaptic neuron. The synaptic weight modification ∆w is then described by the 22 following equations: 23 ∆w(∆t) = ( −λf−(w) × exp(−|∆t|/τp) (∆t ≥ 0), λf+(w) × exp(−|∆t|/τd) (∆t < 0), (4)

where λ = 0.05 is the maximum modification rate. We used the same time constant for 24

potentiation and depression; τp = τd = 20ms (Bi and Poo, 1998). We assumed that the

25

synaptic efficacy is limited to the range [0, 1]. According to G¨utig et al. (2003), for the weight 26

updating functions f+(w) and f−(w) ≥ 0, we used nonlinear functions in which the weight

27

dependence has a power-law form with a non-negative exponent µ: 28

where α > 0 denotes the asymmetry between the scales of the potentiation and depression. We 1

set α to 1.05, unless otherwise stated. For µ = 0, the updating functions are independent of 2

the present synaptic weight, which corresponds to the additive STDP rule. The case µ = 1 3

corresponds to the multiplicative STDP rule. Then, the present synaptic weight linearly affects 4

the updating functions. For 0 < µ < 1, the updating functions depend on the present synaptic 5

weight in a nonlinear form. See G¨utig et al. (2003) for further details of this learning rule. 6

The STDP learning can be implemented in two ways; all possible pairs of pre- and postsynap-7

tic spikes (Song et al., 2000; Froemke and Dan, 2002) or only nearest neighbor spike pairs (e.g., 8

Izhikevich et al., 2004). In this study, we assumed the latter scheme to calculate the weight 9

update. 10

2.3 External inputs 11

We assumed that some important information is represented by the spatiotemporal spike pat-12

tern applied to the spiking neural network. For simplicity, independent Poisson-process spike 13

trains were assumed to be the external input spatiotemporal patterns for the neurons. The 14

amplitude of the external input was set to 0.2µA/cm2 (i.e., IEXT(t) = 0.2µA/cm

2

in Eq.(2)). 15

If a neuron at the resting potential receives the external input at this amplitude, the neuron 16

immediately emits a spike. 17

Because we wish to examine the effects of the input information on the output spike patterns 18

through the changes in the synaptic weights, the external input should be more significant than 19

the other random external inputs. Thus, in this study, the input spatiotemporal pattern was 20

applied repeatedly and frequently with period T ms, and the neural network could learn this 21

pattern. Examples of the input stimuli are shown in Fig.1. The assumption that the inputs 22

are repeatedly applied is not arbitrary because it is often observed that some spike patterns 23

repetitively exist in long spike trains in the brain (Nadasdy et al., 1999; Ikegaya et al., 2004). 24

3

Simulation results

1

We observed the output spike sequences of the neurons from the spiking neural network to 2

reveal how the output spike pattern of the spiking neural network is altered by the STDP when 3

the spiking neural network repeatedly receives a specific spatiotemporal pattern as an external 4

input. Unless otherwise stated, the parameter values were as follows: The period and the mean 5

ISI of the input spatiotemporal pattern were 100 ms and 100 ms, respectively. α and µ in the 6

STDP learning were set to 1.05 and 1, respectively. 7

3.1 STDP transformation: a synchrony in response to a spatiotemporal pattern 8

Figure 2 shows a typical example of the simulation results. Figure 2a shows the input stimulus. 9

The bar indicates the period (100 ms) of one input pattern. Without the STDP learning, 10

the neural network generated spikes following the input stimulus. Thus, the output was an 11

asynchronous pattern in response to the input (Fig.2b). In contrast, if the neural network 12

develops with the STDP learning, global synchronies gradually occur (Fig.2c). Figure 2d shows 13

three enlargements of the raster plot from t = 0 to t = 100 ms (left), t = 1900 to t = 2000 14

ms (center), and t = 19900 to t = 20000 ms (right). The synchrony sharpens as the learning 15

proceeds. Only one synchrony occurs in each period of the input spatiotemporal pattern in 16

this example. We evaluated the number of generated synchronies in the case that the spiking 17

neural network received the input pattern 100 times; the percentage of one synchrony, more 18

than one synchrony, and no synchronies were 99.2%, 0.4%, and 0.4% over 1,000 simulations. 19

To evaluate the degree of synchrony, we introduced a synchrony coefficient S(t0):

20 S(t0) = max t′_{=0,1,··· ,T −1}C(t0− t ′ ; t0), (6) C(t; t0) = c Nτs N X i=1 X l t+τs/2 X t′_=t−τs/2 δ(t′− tl_i)  1 NT N X i=1 X l t0 X t′_=t0−T δ(t′− tl_i), (7)

where c is a scaling parameter (c = 0.5 in this study); N = 1000, the number of neurons; 21

T , the period of the input spatiotemporal pattern; tl

i, the timing of the l-th spike of the

i-22

th neuron; and τs, a temporal window epoch for synchrony evaluation. Even if the observed

23

synchrony is not sufficiently sharp, the synchrony coefficient defined by Eqs.(6) and (7) can 24

detect the synchronies, because the synchrony coefficient counts the number of spikes by shifting 25

the windows temporally from −τs/2 to τs/2. This temporal shift τs should be less than the

26

refractory period of the neuron in order to avoid double counting of the neuron spikes. In this 27

study, the neuron is reset to −90 mV after a firing and takes approximately 10 ms to reach the 28

resting potential, −70 mV. Therefore, we set τs to 5 ms. We evaluated the synchrony coefficient

in every input period, that is, t = t0 = T, 2T, 3T, · · · (Figs.2c and e).

1

To evaluate the statistical significance of the synchrony coefficient, we adopted the simplified 2

Monte-Carlo significance test (Hope, 1968) for the synchrony coefficient, because we could not 3

assume the distribution of the coefficients to be normal. Here, we summarize the method of 4

performing the simplified Monte-Carlo significance test (see Fig.3). 5

1. For every input period, we obtained a raster plot. We call this raster plot an original raster 6

plot. 7

2. We produced a surrogate raster plot by randomly shuffling the temporal indices of the 8

original raster plot. Although the temporal structure of the surrogate raster plot is different 9

from that of the original raster plot, the firing rate of each neuron was maintained. We 10

produced a set of 39 different surrogate raster plots for each original raster plot. 11

3. We evaluated the synchrony coefficients of the original raster plot and its surrogate raster 12

plots by using Eqs.(6) and (7). 13

4. We obtained 1 + 39 = 40 synchrony coefficients by the procedures described above. If the 14

synchrony coefficient of the original raster plot is larger than those of the surrogate raster 15

plots, the synchrony coefficient of the original is considered to be statistically significant, 16

which implies that synchronies exist in the original raster plot. Otherwise, we cannot con-17

clude that synchronies are present in the original raster plot. In this study, the synchrony 18

coefficient of the original raster plot was not below the distribution of the synchrony coef-19

ficients of the surrogate raster plots. Therefore, in the following simulations, we show only 20

the largest synchrony coefficient of the surrogate raster plots. 21

Figure 2e shows the synchrony coefficients of the original raster plot (closed circles) and the 22

largest synchrony coefficients of the surrogate raster plots (open squares) of Fig.2c. In Fig.2c, no 23

synchronies are observed during the first six periods of the input. The corresponding synchrony 24

coefficients of the original raster plots are as low as those of the surrogate raster plots. When the 25

seventh input is applied, a synchrony begins to occur at around t = 645 ms and the synchrony 26

coefficient begins to increase. This demonstrates that the synchrony coefficient reflects the 27

synchrony. The synchrony coefficient becomes stable in longer simulations (Fig.2f). In this 28

study, the synchrony coefficients of the input pattern and the output of the neural network 29

without STDP are smaller than the largest synchrony coefficient of the surrogate raster plots. 30

These are not shown in the figures. 31

Figure 2g shows a transition of the mean synaptic weight that includes not only excitatory-to-1

excitatory connections but also other connections. As the learning proceeds, the mean synaptic 2

weight decreases and converges to 0.263. As shown in Fig.2i, most of the synaptic weights are 3

divided into two groups at the upper and lower limits. However, some weights remain between 4

these limits. The shape of the synaptic weight distribution depends on µ of the STDP. This is 5

discussed in Section 3.4. 6

We define the phase of the synchrony as the relative timing of the synchrony to the input 7

pattern period. The phase is defined by (nT − t′_{) mod T of max}

t′_{=0,1,··· ,T −1}C(nT − t

′

; t0), where

8

T is the period of the input pattern, C(t; t0) is defined by Eq.(7), and n = 1, 2, · · · . The

9

transition of the phase is shown in Fig.2h. The phase of this example is 25.1 ms at t = 20, 000 10

ms. The phase of the synchrony depends on the input spatiotemporal pattern and the neural 11

network structure. The neural network structure includes the pattern of the synaptic weights, 12

the delays, and the type of neurons, i.e., excitatory or inhibitory. If the same neural network 13

learns a different input pattern, the synchrony occurs at a different phase. These results imply 14

that the spiking neural network learns the spatiotemporal structure of the input pattern and 15

encodes it as the phase of the synchrony. 16

3.2 Mechanism of synchrony generation 17

Here, we discuss how synchronies are generated. In this study, the synaptic weights from the 18

excitatory neurons were initially set to 0.45. With this value for the weights, the excitatory 19

neurons rarely evoke postsynaptic neural firings; thus, the network had a low firing rate at first. 20

By the STDP learning, if a connection has a causal relationship with firing, it is potentiated 21

by the STDP learning. Due to the periodicity of the input, such potentiated connections are 22

repeatedly potentiated. As a result, the neurons come to fire only due to the contributions 23

of the other neurons through the potentiated connections, and the activity of the network 24

increases. The excitatory neurons that receive potentiated synaptic inputs excite not only the 25

excitatory neurons but also the inhibitory neurons. A sharp synchrony occurs if the strong firing 26

of the excitatory neurons excites a sufficient number of inhibitory neurons to temporarily shut 27

down the network activity. The mechanism of synchrony generation is similar to the gamma 28

oscillation generation known as “the pyramidal-interneurons network gamma” (Whittington 29

et al., 2000; Izhikevich, 2006). 30

Although the above mechanism explains synchrony generation, it does not explain how the 31

phase of the synchrony is decided and why multiple synchronies do not occur in every input 32

period. For the former problem, we expected that the input pattern and the network structure 33

decide the phase. To confirm this hypothesis, we first observed the dependency of the phase of 1

the synchrony on the initial synaptic weights by assigning random numbers between 0 and 1 to 2

the excitatory-to-excitatory connections. If the input pattern and the network structure decide 3

the phase, the initial values of the synaptic weights will not affect the phase. As expected, the 4

emerged phases of the synchrony were the same as those in the case where the initial values 5

were fixed at a constant value (0.45). 6

Second, we show that the pattern of the initial ∆w (Eq.(4)) strongly affects the phase of the 7

synchrony. On repetitive application of the input pattern with a period of 500 ms, the spiking 8

neural network generated the synchrony shown in Fig.4a; here, the initial synaptic weights of 9

the excitatory-to-excitatory connections were equal to 0.45. If the input pattern is applied once 10

(Fig.4b), the synaptic weights of the initial condition (Fig.4h (initial)) are slightly changed by 11

the STDP, as shown in Fig.4h (∆w). We then modified the synaptic weights; the slight changes 12

in the synaptic weights (Fig.4h (∆w)) are multiplied with 11, as shown in Fig.4h (multiplied 13

∆w), which is an operation to adjust the smallest synaptic weight to be equal to zero. If the 14

pattern of the initial ∆w (Fig.4h (∆w)) strongly affects the phase of the synchrony, the neural 15

network with the modified weights (Fig.4h (multiplied ∆w)) would generate a synchrony in 16

phase with that shown in Fig.4a. When the input pattern was reapplied, the modified spiking 17

neural network generated four synchronies per input period (Fig.4c). As the input pattern 18

was repeatedly applied, three of the initial four synchronies gradually disappeared and a single 19

synchrony remained (Figs.4c, d, e, and f). The phase of the remaining synchrony corresponded 20

to the phase of the synchrony without the artificial modification (Figs.4f and a), indicating 21

that the pattern of the initial ∆w strongly affected the phase of the synchrony. The pattern of 22

the initial ∆w was decided by the input pattern and the network structure. Thus, the firings 23

resulting from the recurrent neural interactions did not affect the phase of the synchrony. These 24

results indicate that the phase of the synchrony is uniquely decided by the input pattern and 25

the neural network structure. In fact, when we applied a different input pattern to the same 26

spiking neural network shown in Figs.4a ∼ f, with the same initial synaptic weights, delays, 27

and the types of neurons, the network generated a synchrony with a different phase (Fig.4g). 28

Finally, we considered why multiple synchronies do not occur in every input period. In 29

Fig.4c, there were four synchronies at the beginning. However, three of them disappeared and 30

only a single synchrony remained. This is a result of competition between the synchronies. 31

The synchronies shared the 800 ∼ 900 neurons. However, in most cases, the order of the 32

firings of the neurons was different for different synchronies. Therefore, connections that were 33

potentiated in one synchrony were often depressed in another. If the depression was greater 1

than the potentiation, the connection became smaller and finally became zero. As a result, 2

only a single synchrony remained. 3

3.3 Dependency on input pattern 4

In this section, we examine how the synchrony coefficients are altered by the properties of 5

the input pattern. Figure 5a shows the synchrony coefficients when the period of the input 6

spatiotemporal pattern is changed. When the period of the input pattern is less than 30 ms, the 7

synchrony coefficient is below that of the surrogate raster plots; this indicates that the spiking 8

neural network cannot generate synchronies with such an extremely short input pattern. If the 9

period of the input pattern is longer, the synchrony coefficients increase. When the period of 10

the input pattern is close to 1,000 ms, the synchrony coefficient peaks. For periods longer than 11

1,000 ms, the synchrony coefficients decrease. Figure 5b shows the synchrony coefficients when 12

the mean ISI of the input pattern is changed. The synchrony coefficient peaks when the mean 13

ISI is close to 100 ms. When the mean ISI is extremely low, every output spike tends to burst, 14

and the synchrony coefficients decrease. On the other hand, when the mean ISI is extremely 15

high, few spikes are observed, and the synchrony coefficients also assume low values. 16

To quantitatively estimate the sensitivity of the response of the network to a specific spa-17

tiotemporal pattern, we examined the robustness of the synchrony generation to noise. We 18

considered the following two cases for adding noise: noise is added (i) during the learning 19

(learning period) or (ii) after the learning (recalling period). The amount of noise added X% 20

to an input pattern is defined as follows: 21

1. X% of the spikes are randomly selected from an input pattern. 22

2. The spike timings of the selected spikes are randomly changed by avoiding overlap with 23

another spike timings. 24

3. For each input pattern, the above procedures are independently repeated. 25

In the following two experiments, we referred the phase of the synchrony to evaluate whether 26

the network correctly identifies the input pattern. If the phase of the synchrony differed from 27

that of the noiseless input pattern by 10 ms (half value of the time constant τm), the synchrony

28

coefficient was set to 0. Figure 6a shows the synchrony coefficients for the noiseless input pat-29

terns obtained after the learning with noise. When the noise was less than 40%, the synchrony 30

coefficients were larger than those of the surrogate raster plots. Figure 6b shows the synchrony 31

coefficients for the noisy input patterns obtained after the learning without noise. In contrast to 32

Fig.6a, even if the noise level is 10%, the synchrony coefficients are below that of the surrogate 1

raster plots, that is, an incomplete input pattern cannot be a cue for the synchrony. These 2

results indicate that even if the input pattern is disturbed by low-level noise, the spiking neural 3

network can learn the hidden spatiotemporal structure of the input pattern. In addition, once 4

the spiking neural network has learned the input pattern, it can correctly identify incoming 5

inputs by generating the synchrony in response to the learned input pattern. 6

In addition, we investigated the case that noise is added during both the learning and recalling 7

periods. The results are shown in Fig.6c. Interestingly, these results indicate that when noise 8

is added during recalling, the synchrony is recalled better, when some noise has also been 9

added during learning than that for noiseless learning. In addition, with appropriate levels of 10

noise (within 5%–20%) during learning and recalling, the synchrony generation exhibits relative 11

robustness to increases in noise. These results may be because the model implicitly learns to 12

deal with noise during the training phase. 13

Finally, we examined the effects of the repetition of input patterns: Can the spiking neural 14

network learn the input pattern if random patterns are inserted between the input patterns? We 15

found that the application of random patterns between two specific input spatiotemporal pat-16

terns does not significantly affect the results. Even when 200-ms-long random patterns (Poisson 17

process trains) are inserted between the input patterns of T = 100 ms, the synchrony coeffi-18

cients of the original raster plots exceed those of the surrogate raster plots for 82 out of 100 19

simulations; that is, the neural network can learn synchrony generation with rates greater than 20

80%. 21

3.4 Dependency on the form of the STDP function 22

Figure 7 shows the effects of the STDP rule on the synchrony coefficient. Figure 7a shows the 23

synchrony coefficients when α of the STDP learning is altered. This parameter decides the 24

asymmetry between the scales of the synaptic potentiation and depression. When α is 1.15, 25

the synchrony coefficient peaks. As α is increased, the synchrony coefficient decreases. When α 26

is 2.2, the error bar includes the synchrony coefficient of the surrogate raster plots. Therefore, 27

α should be smaller than 2. On the other hand, α must be larger than 0.83, for otherwise, the 28

network bursts. 29

Several reports have indicated that the two candidates for the STDP learning rule —the 30

additive and the multiplicative STDP rules— lead to different results (van Rossum et al., 31

2000; Rubin et al., 2001; G¨utig et al., 2003). We investigated the effects of the rules on the 32

synchrony coefficients (Fig.7b). In the Fig.7b, µ = 0 corresponds to the additive STDP rule 33

and µ = 1 corresponds to the multiplicative STDP rule. As is shown in Fig.7b, as µ increases, 1

the synchrony coefficients increase slightly, and the error bars shrink. This implies that the 2

multiplicative STDP rule is more suitable for synchrony generation. However, the fact that 3

the synchrony coefficient assumes a large value regardless of the value of µ indicates that the 4

dependence on the rule is not essential for synchrony generation. 5

It is well known that the synaptic weights with the multiplicative STDP rule driven by random 6

Poisson inputs result in a unimodal distribution of the synaptic weights (van Rossum et al., 7

2000; Rubin et al., 2001; G¨utig et al., 2003). However, in our study, even when the multiplicative 8

STDP rule is adopted, the synaptic weight distribution does not become unimodal but bimodal, 9

even after a significantly long learning time of t = 500, 000 ms (Fig.2i). This is attributed to 10

the repetition of the specific inputs. In the case of the random inputs, the synaptic weights are 11

equally altered by the potentiation and the depression of the STDP. In contrast, in the case of 12

the repetitive inputs, the synaptic weights which are strengthened by the one-time application 13

of the input pattern are also strengthened by the successively applied input patterns. As a 14

result, even in the case of the multiplicative STDP rule, the synaptic weights have a bimodal 15

distribution. However, the precise distribution forms are different for the case of the additive 16

and the multiplicative STDP rules. Figure 8 shows the temporal evolution of the histograms 17

of the synaptic weights (Fig.8a) and outputs (Fig.8b) of the spiking neural network with the 18

additive STDP rule, where all the other conditions are the same those in Fig.2. If we compare 19

Fig.2i and Fig.8a, we can see that the synaptic weights with the multiplicative STDP rule 20

do not completely split into lower and higher limits, but some weights remain between these 21

limits (Fig.2i); however, the weights with the additive STDP rule do (Fig.8a). In addition, the 22

synaptic weights modified by the additive STDP rule are split into boundaries faster than in the 23

case of the multiplicative STDP rule. However, the shape of the synchrony and its generation 24

speed in the case of the additive STDP rule are almost the same as those in the case of the 25

multiplicative STDP rule (Fig.2c and Fig.8b). We summarize the shapes of the histograms of 26

the synaptic weights in Table 1. 27

4

Discussion

1

From the results of the simulations, we observe that the spiking neural network transforms 2

the input spatiotemporal pattern into a synchrony whose timing depends on the input spa-3

tiotemporal pattern and the neural network structure. This implies that the neural network 4

can self-organize its transformation function from spatiotemporal to temporal information. 5

The transformation from a spatiotemporal pattern to a synchrony possibly exists in actual 6

neural networks. Synaptic plasticity is usually observed in the cortices and the hippocampus. 7

Thus, this transformation should play an important role in information representation in the 8

cortices and the hippocampus. 9

Even though we used a simple spiking neural network in this study, the results are suggestive. 10

In particular, the results raise an important issue: Does a similar mechanism truly exist in our 11

brains? To construct more realistic neural network models, alpha-function shaped postsynaptic 12

currents should be considered instead of the delta-shaped ones and Hodgkin-Huxley dynamics 13

instead of the LIF neuron model. 14

4.1 Relation to the synfire chain 15

In a synfire chain, synchronous spikes stably propagate through feedforward layered networks (Abeles 16

et al., 1993; Diesmann et al., 1999). The stability of the synfire chain has been ascertained using 17

theoretical and experimental methods (Diesmann et al., 1999; Reyes, 2003). However, little at-18

tention has been paid to the mechanism by which the first synchrony triggers the synfire chain. 19

In previous studies, dispersed synchronous spikes have been assumed a priori and have been 20

gradually sharpened through the propagation process in the layered neural network (Reyes, 21

2003). Our result suggests the presence of a neural network located near the sensory cortex, 22

wherein a synchrony can be generated from a specific input spatiotemporal pattern. In the 23

neural network, the synapses with STDP have already learned the transformation from input 24

sensory information to the synchrony. The neural network generates the synchrony if it detects 25

a previously learned pattern. 26

The propagation speed of a synfire chain is considerably higher than that of the other in-27

formation representations using mean firing rates because the synfire chain does not need to 28

calculate the temporal average. In other words, information representation by synchronies is 29

indispensable to realize efficient information transmission. Thus, synchronies and synfire chains 30

should be used in regions where fast processing is required. From this viewpoint, a candidate 31

for the source of synfire chains is the connections between the thalamus, cortex, amygdala, and 32

hippocampus. 33

4.2 Relation to the phase coding 1

Considering the synchrony, the phase, and its property of robustness to noise, we can cor-2

relate the functions of the STDP with the phase coding in the hippocampus. In a rodent 3

hippocampus, the phases in the synchronous theta rhythm are thought to represent important 4

information (O’Keefe and Recce, 1993). For example, the place cells in CA1 encode the spa-5

tiotemporal information of the environment using the phases in the theta rhythm. Our results 6

raise the issue that the neural networks in the hippocampus may encode the information applied 7

to the hippocampus into phases through the STDP learning rule. 8

4.3 Relation to the distributed synchrony 9

We consider that different types of dynamics can operate in a neural network, depending on 10

the properties of the external inputs and the network connections. For example, we can assume 11

two dynamics: One dynamics is strongly affected by a spatiotemporal spike pattern of other 12

networks. Thus, the dynamics is driven by the external inputs. The second dynamics is 13

not affected by a spatiotemporal spike pattern of other networks, but is driven by internal 14

interactions (Araki and Aihara, 2001). We consider that the distributed synchrony of Levy 15

et al. (2001) and our global synchrony are the results of the different dynamics. The distributed 16

synchrony can be assumed to be the result of the second dynamics. Hence, the random external 17

input is assumed to be background activity. Our study assumes the first dynamics and shows 18

the occurrence of global synchrony. Our results indicate that the different dynamics in a neural 19

network result in different pattern transformation. 20

Acknowledgments

1

We thank anonymous referees for their constructive comments and suggestions. We also thank 2

Kazuyuki Aihara (Institute of Industrial Science, The University of Tokyo) for encouragement. 3

This research was partially supported by Grant-in-Aid for Scientific Research on Priority Areas 4

(Advanced Brain Science Project) (No.14016002) from MEXT to O.A. and Grant-in-Aids for 5

Scientific Research (C) (No.17500136) and (B) (No.16300072) from JSPS to T.I. 6

References

1

Abbott, L. F. and Nelson, S. B. (2000). Synaptic plasticity: taming the beast. Nature Neuro-2

science, 3:1178–1183. 3

Abeles, M. (1991). Corticonics —Neural circuits of the cerebral cortex—. Cambridge University 4

Press, Cambridge. 5

Abeles, M., Bergman, H., Margalit, E., and Vaadia, E. (1993). Spatiotemoporal firing patterns 6

in the frontal cortex of behaving monkeys. Journal of Neurophysiology, 70:1629–1638. 7

Aertsen, A., Erb, M., and Palm, G. (1994). Dynamics of functional coupling in the cerebral 8

cortex: An attempt at a model-based interpretation. Physica D, 75:103–128. 9

Aertsen, A. M. H. J., Gerstein, G. L., Habib, M. K., and Palm, G. (1989). Dynamics of neu-10

ronal firing correlation: Modulation of “effective connectivity.”. Journal of Neurophysiology, 11

61:900–917. 12

Araki, O. and Aihara, K. (2001). Dual information representation with stable firing rates 13

and chaotic spatiotemporal spike patterns in a neural network model. Neural Computation, 14

13:2799–2822. 15

Bell, C. C., Han, V. Z., Sugawara, Y., and Grant, K. (1997). Synaptic plasticity in a cerebellum-16

like structure depends on temporal order. Nature, 15:278–281. 17

Bi, G. and Poo, M. (1998). Synapic modificaion in cultured hippocampal neurons: dependence 18

on spike timing, synaptic strength, and postsynaptic cell type. Journal of Neuroscience, 19

18:10464–10472. 20

Braitenberg, V. and Sch¨uz, S. (1991). Anatomy of the cortex. Springer-Verlag, Berlin. 21

de Oliveira, S. C., Thiele, A., and Hoffmann, K. P. (1997). Synchronization of neuronal activity 22

during stimulus expectation in a direction discrimination task. Journal of Neuroscience, 23

17:9248–9260. 24

Debanne, D., Gahwiler, B. H., and Thompson, S. M. (1998). Long-term synaptic plasticity 25

between pairs of individual CA3 cell in rat hippocampal slice cultures. Journal of Physiology, 26

15:237–247. 27

Diesmann, M., Gewaltig, M. O., and Aertsen, A. (1999). Stable propagation of synchronous 28

spiking in cortical neural networks. Nature, 402:529–533. 29

Froemke, R. C. and Dan, Y. (2002). Spike-timing-dependent synaptic modification induced by 1

natural spike trains. Nature, 416:433–438. 2

Fujii, H., Ito, H., Aihara, K., Ichinose, N., and Tsukada, M. (1996). Dynamical cell assembly 3

hypothesis – theoretical possibility of spatio-temporal coding in the cortex. Neural Networks, 4

9:1303–1350. 5

Gerstner, W., Kempter, R., von Hemmen, J., and Wagner, H. (1996). A neuronal learning rule 6

for sub-millisecond temporal coding. Nature, 383:76–78. 7

Gerstner, W., Ritz, R., and von Hemmen, J. L. (1993). Why spikes? Hebbian learning and 8

retrieval of time-resolved excitation patterns. Biological Cybernetics, 69:503–515. 9

Gray, C. M., K¨onig, P., Engel, A. K., and Singer, W. (1989). Oscillatory responses in cat 10

visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties. 11

Nature, 338:334–337. 12

G¨utig, R., Aharonov, R., Rotter, S., and Sompolinsky, H. (2003). Learning input correla-13

tions through nonlinear temporally asymmetric hebbian plasticity. Journal of Neuroscience, 14

23:3697–3714. 15

Hope, A. C. A. (1968). A simplified monte carlo significance test procedure. J. R. Statist. Soc. 16

B, 30:582–598. 17

Ikegaya, Y., Aaron, G., Cossart, R., Aronov, D., Lampl, I., and Yuste, R. (2004). Synfire chains 18

and cortical songs: Tempral modules of cortical activity. Science, 34:559–564. 19

Izhikevich, E. M. (2006). Polychronization: Computation with spikes. Neural Computation, 20

18:245–282. 21

Izhikevich, E. M., Gally, J. A., and Edelman, G. M. (2004). Spike-timing dynamics of neuronal 22

groups. Cerebral Cortex, 14:933–944. 23

Levy, N., Horn, D., Meilijson, I., and Ruppin, E. (2001). Distributed synchrony in a cell 24

assembly of spiking neurons. Neural Networks, 14:815–824. 25

Liley, D. T. J. and Wright, J. J. (1994). Intracortical connectivity of pyramidal and stellate cells: 26

estimates of synaptic densities and coupling symmetriy. Computation in Neural Systems, 27

5:175–189. 28

Markram, H., L¨ubke, J., Frotscher, M., and Sakmann, B. (1997). Regulation of synaptic efficacy 1

by coincidence of postsynaptic APs and EPSPs. Science, 275:213–215. 2

Masuda, N. and Aihara, K. (2004). Self-organizing dual coding based on spike-time-dependent 3

plasticity. Neural Computation, 16:627–663. 4

Matsumoto, N. and Okada, M. (2002). Self-regulation mechanism of temporally asymmetric 5

hebbian plasticity. Neural Computation, 14:2883–2902. 6

Nadasdy, Z., Hirase, H., Czurko, A., Csicsvari, J., and Buzsaki, G. (1999). Replay and time com-7

pression of recurring spike sequences in the hippocampus. Journal of Neuroscience, 1:9497– 8

9507. 9

Nishiyama, M., Hong, K., Mikoshiba, K., Poo, M. P., and Kato, K. (2000). Calcium stores 10

regulate the polarity and input specificty of synaptic modification. Nature, 408:583–588. 11

O’Keefe, J. and Recce, M. (1993). Phase relationship between hippocampal place units and 12

the EEG theta rhythm. Hippocampus, 3:317–330. 13

Rao, R. P. N. and Sejnowski, T. J. (2001). Spike-timing-dependent hebbian plasticity as tem-14

poral difference learning. Neural Computation, 13:2221–2237. 15

Reyes, A. D. (2003). Synchrony-dependent propagation of firing rate in iteratively constructed 16

networks in vitro. Nature Neuroscience, 6:593–599. 17

Riehle, A., Gr¨un, S., Diesmann, M., and Aertsen, A. (1997). Spike synchronization and rate 18

modulation differentially involved in motor cortical function. Science, 278:1950–1953. 19

Rubin, J., Lee, D. D., and Sompolinsky, H. (2001). Equilibrium properties of temporally 20

asymmetric hebbian plasticity. Physical Review Letters, 86:364–367. 21

Song, S. and Abbott, L. F. (2001). Cortical remapping through spike timing-dependent plas-22

ticity. Neuron, 32:1–20. 23

Song, S., Miller, K. D., and Abbott, L. F. (2000). Competitive hebbian learning through 24

spike-timing-dependent synaptic plasticity. Nature Neuroscience, 3:919–926. 25

Vaadia, E., Haalman, I., Abeles, M., Bergman, H., Prut, Y., Slovin, H., and Aertsen, A. (1995). 26

Dynamics of neuronal interactions in monkey cortex in relation to behavioural events. Nature, 27

373:515–518. 28

van Rossum, M. C. W., Turigiano, G. G., and Nelson, S. B. (2000). Stable hebbian learning 1

from spike timing-dependent plasticity. Journal of Neuroscience, 22:1956–1966. 2

Whittington, M. A., Traub, R. D., Kopell, N., Ermentrout, B., and Buhl, E. H. (2000). 3

Inhibition-based rhythms: Experimental and mathematical observations on netwrok dynam-4

ics. International Journal of Phychophysiology, 38:315–336. 5

Yoshioka, M. (2001). Spike-timing-dependent learning rule to encode spatiotemporal patterns 6

in a network of spiking neurons. Physical Review E, 65:011903. 7

Zhang, L. I., Tao, H. W., Holt, C. E., Harris, W. A., and Poo, M. (1998). A critical window for 8

cooperation and competition among developing retinotectal synapses. Nature, 395(6697):37– 9

44. 10

Zhigulin, V. P., Rabinovich, M. I., Huerta, R., and Abarbanel, H. D. I. (2003). Robustness and 11

enhancement of neural synchronization by activity-dependent coupling. Physical Review E, 12

67:021901. 13

t _(ms)

a

period=50ms

b

period=100ms

c

period=200ms

d

mean ISI=50ms

e

mean ISI=100ms

f

mean ISI=200ms 0 100 200 0 100 200 0 100 200 0 1000 0 50 100 150 200 250 300 350 400 0 1000 0 100 200 300 400 Ne u ro n in dex 0 1000 0 200 400 Figure 1

Input stimulus without STDP with STDP 0 2 4 6 8 10 1000 Synchr ony coeffic ient t _(ms) original largest surrogate

e

a

Synch rony coeffic ient Mea n of weigh ts

d

f

i

T

g

h

period 0 1000 0 1000 Neuron i nde x 0 1000 0.2 0.3 0.4 0.5 0 50 100 0 10000 20000 Phase

t

(ms) 0 2 4 6 8 10 0 1000 0 100 Neu ron in dex 1900 2000

t

(ms) 19900 20000 0

b

c

t=0ms t=2,500ms t=5,000ms t=7,500ms t=10,000ms t=20,000ms 0 140000 0 1 Frequ ency 0 10000 0 1 0 12000 0 1 0 14000 0 1 0 18000 0 1 0 35000 0 1 Synaptic weights 0 40000 0 1 t=500,000ms original largest surrogate Figure 2

original

significant

insignificant

Synchrony coefficients

surrogates

a 0 1000 50500 51000 Neuron index b 0 1000 0 500 Neuron index

c artificially modified network

0 1000

500 1000

Neuron index

d artificially modified network

0 1000

1500 2000

Neuron index

e artificially modified network

0 1000

3500 4000

Neuron index

f artificially modified network

0 1000

50500 51000

Neuron index

g stimulated by a different input pattern

0 1000 50500 51000 Neuron index t (ms) h 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0 10 20 30 40 50 Weight Synapse index (initial) (Δw) (multiplied Δw) Figure 4

a

b

0 2 4 6 8 10 3000 1000 500 100 50 20 Synchrony coefficients Period 0 2 4 6 8 10 500 200 100 50 20 Synchrony coefficients Mean ISI Figure 5

a

b

0 2 4 6 8 10 0 20 40 60 80 100 Synchrony coefficients

Amount of noise during learning (%)

0 2 4 6 8 10 0 20 40 60 80 100 Synchrony coefficients

Amount of noise during recalling (%)

c

0 100 20 0 100 0 1 2 3 4 5 6 Amount of noise during recalling (%) Synchrony coefficients 40 60 80 40 20 80 60

Amount of noise during learning (%)

a

b

0 2 4 6 8 10 1 1.5 2 2.5 3 Synchrony coefficients α 0 2 4 6 8 10 0 0.2 0.4 0.6 0.8 1 Synchrony coefficients µ Figure 7

a

t=0ms t=1,000ms t=1,500ms t=2,000ms t=2,500ms t=20,000ms Frequency Synaptic weights 0 14000 0 1 0 140000 0 1 0 12000 0 1 0 40000 0 1 0 60000 0 1 0 80000 0 1

b

0

1000

0

500

1000

Neuron

index

t

_(ms)

Figure 8

Inputs STDP rule

additive multiplicative

random complete bimodal unimodal

(Song et al., 2000) (van Rossum et al., 2000; Rubin et al., 2001) periodic complete bimodal incomplete bimodal

(this paper) (this paper) Table 1

Caption for Fig.1 1

Examples of input spatiotemporal patterns. In the left column, the periods are (a) 50 ms, (b) 2

100 ms, and (c) 200 ms and the mean ISI is 100 ms. On the right column, the period is 100 3

ms and the mean ISIs are (d) 50 ms, (e) 100 ms, and (f) 200 ms. 4

Caption for Fig.2 5

A typical example of the output spikes. (a) Input stimulus, which comprises repetition of 6

the input spatiotemporal pattern. The period and the mean ISI of the input spatiotemporal 7

pattern are 100 ms and 100 ms, respectively. The bar indicates the input pattern period T . 8

A raster plot of the output spikes (b) without STDP learning and (c) with STDP learning. 9

(d) Enlargements of c, from t = 0 to t = 100 ms (left), t = 1900 to t = 2000 ms (center), 10

and t = 19900 to t = 20000 ms (right). (e) The synchrony coefficients of the original raster 11

plot (filled circles) and the largest synchrony coefficient of the surrogate raster plots (open 12

squares). The synchrony coefficients at t =100 ms are calculated using the raster plot from 13

t = 0 to t = 100 ms, and those at other moments are calculated in the same manner. (f) The 14

same as e, but up to t = 20, 000 ms. Temporal changes of (g) mean synaptic weight of all 15

the connections, and (h) the phase of the synchrony. (i) Histograms of the synaptic weights of 16

excitatory-to-excitatory connections at t =0, 2500, 5000, 7500, 10,000, 20,000, and 500,000 ms. 17

Caption for Fig.3 18

Monte-Carlo significance test. The closed circles indicate the synchrony coefficient of the orig-19

inal raster plot; the open circles, the synchrony coefficients of the surrogate raster plots; and 20

the open square, the largest synchrony coefficient of the surrogate raster plots. In Figs.2e, 2f, 21

5, 6, and 7, only the largest synchrony coefficients of the surrogate raster plots are displayed. 22

Caption for Fig.4 23

A mechanism for synchrony generation. The period of the input pattern is 500 ms. (a) The 24

raster plot after 100 applications of the input pattern. (b) The raster plot with the initial 25

synaptic weights. (c ∼ f) The raster plots with a set of modified synaptic weights from 500 26

to 1000 ms, 1500 to 2000 ms, 3500 to 4000 ms, and 50,500 to 51,000 ms, respectively. (g) 27

The raster plot after 100 applications of the different input pattern from that in a ∼ f without 28

the weight modification. (h) A part of the synaptic weights (50 of the 128,000 excitatory-to-29

excitatory connections): (initial) initial condition (t = 0 ms); (∆w) after one-time application 30

of the input pattern; and (multiplied ∆w) that obtained by multiplying the difference between 31

(∆w) and (initial) with 11. 32

Caption for Fig.5 1

Each graph shows the synchrony coefficients when (a) the period and (b) the mean ISI of the 2

input patterns are altered. After 100 applications of the input pattern, the input spatiotemporal 3

patterns were reapplied and the synchrony coefficients were then calculated. For both the 4

original raster plot (closed circles) and the surrogate raster plots (open squares), the points 5

and error bars indicate the mean and standard deviation of the synchrony coefficient calculated 6

over 100 simulations, respectively. 7

Caption for Fig.6 8

The same as Fig.5, but the learned spatiotemporal pattern is reapplied after the STDP learn-9

ing. Noise is added (a) during the learning, (b) during the recalling (after the learning), and 10

(c) during both the learning and recalling. 11

Caption for Fig.7 12

The same as Fig.5, but (a) α and (b) µ are altered. 13

Caption for Fig.8 14

(a) Temporal evolution of the distribution of the synaptic weights of excitatory-to-excitatory 15

connections at t =0, 1000, 1500, 2000, 2500, and 20,000 ms; and (b) the outputs of the spiking 16

neural network with the additive STDP rule (µ = 0). The same parameters and conditions 17

were used as those of the plots in Fig.2. 18

Caption for Table 1 19

Shape of the distribution of synaptic weights resulting from STDP rule and input statistics. 20