Different noise levels are considered, with input jitters varying from 0 ms to 4 ms.
Table 5: Summarized Results for the Iris Data Set.
Successful Average Number Accuracy on the Accuracy on the Subconnections Trials of Iterations Training Set (%) Testing Set (%)
8 68 125± 12 97± 0.17 89± 0.69
9 80 174± 16 96± 0.00 94± 0.79
10 80 114± 13 97± 0.00 89± 0.47
11 74 140± 15 96± 0.16 86± 0.49
12 68 183± 21 96± 0.17 91± 0.69
Table 6: Summarized Results for a Linearly Nonseparable Problem.
Hidden Successful Average Number
Neurons Trials (%) of Iterations
50 70 293± 59 60 54 301± 66 70 56 327± 91 80 60 469± 87 90 76 247± 42 100 76 439± 73
Table 7: Summarized Results.
Number of Successful Average Number Number of Successful Average Number Hidden Units Trials (%) of Iterations Patterns Trials (%) of Iterations
200 50 5± 0.8 5 100 7± 0.7 210 52 6± 1.2 6 92 5± 0.6 220 78 5± 0.6 7 96 5± 1.2 230 76 6± 1.1 8 92 8± 1.5 240 80 5± 0.6 9 88 7± 0.9 250 74 7± 0.8 10 90 6± 0.6 260 90 5± 0.7 11 72 6± 0.7 270 88 4± 0.5 12 72 6± 0.7 280 80 7± 2.4 13 58 5± 0.9 290 90 4± 0.6 14 40 6± 0.9 300 90 4± 0.4 15 34 5± 1.0
Notes: (Left) The network is trained with 10 pattern pairs, where the size of the hidden layer is varied in order to determine the best network architecture. (Right) A neural network with a hidden layer containing 260 neurons is trained with different numbers of pattern pairs.
Table 8: Summarized Results for Learning with Noisy Patterns. Input Jitter Successful Average Number During Learning Trials (%) of Iterations
0 96 10± 1.2
1 98 12± 1.1
2 95 19± 2.3
3 66 26± 5.6
4 64 115± 51
Note: The input patterns jitter is varied between 0 ms and 4 ms, while the target jitter is always 1 ms.
Acknowledgments
We thank two anonymous reviewers whose comments helped improve the paper signficantly. A.G. was supported in part by Engineering and Physical Sciences Research Council (UK) grant EP/I014934/1.
References
Bohte, S. (2011). Error-backpropagation in networks of fractionally predictive spik- ing neurons. In Proceedings of the 21th International Conference on Artificial Neural
Networks (pp. 60–68). Berlin: Springer-Verlag.
Bohte, S., Kok, J., & Poutr´e, H. L. (2002). Error backpropagation in temporally en- coded networks of spiking neurons. Neurocomputing, 48, 17–37.
Bohte, S., & Rombouts, J. (2010). Fractionally predictive spiking neurons. In J. Laf- ferty, C.K.I. Williams, J. Shawe-Taylor, R. S. Zemel, & A. Culotta (Eds.), Advances
in neural information processing, systems, 23 (pp. 253–261). Red Hook, NY: Curran.
Booij, O., & Nguyen, H. (2005). A gradient descent rule for spiking neurons emitting multiple spikes. Information Processing Letters, 95, 552–558.
deCharms, R. C., & Merzenich, M. M. (1996). Primary cortical representation of sounds by the coordination of action-potential timing. Nature, 381, 610–613. Elias, J. G., & Northmore, D.P.M. (2002). Building silicon nervous systems with
dendritic tree neuromorphs. In W. Maass & C. M. Bishop (Eds.), Pulsed neural
networks. Cambridge, MA: MIT Press.
Fisher, R. A. (1936). The use of multiple Measurements in taxonomic problems.
Annals of Eugenics, 7, 179–188.
Fitzsimonds, R. M., Song, H., & Poo, M. (1997). Propagation of activity dependent synaptic depression in simple neural networks. Nature, 388, 439–448.
Fujita, M., Takase, H., Kita, H., & Hayashi, T. (2008). Shape of error surfaces in SpikeProp. In Proceedings of IEEE International Joint Conference on Neural Networks (pp. 840–844). Piscataway, NJ: IEEE.
Gerstner, W. (2001). A framework for spiking neuron models: The spike response model. In F. Moss & S. Gielen (Eds.), The handbook of biological physics, Vol. 4 (pp. 469–516). Amsterdam: North-Holland.
Gerstner, W., & Kistler, W. M. (2002). Spiking neuron models: Single neurons, populations,
plasticity. Cambridge: Cambridge University Press.
Ghosh-Dastidar, S., & Adeli, H. (2009). A new supervised learning algorithm for multiple spiking neural networks with application in epilepsy and seizure detec- tion. Neural Networks, 22, 1419–1431.
Glackin, C., Maguire, L., McDaid, L., & Sayers, H. (2011). Receptive field optimisation and supervision of a fuzzy spiking neural network. Neural Networks, 24, 247–256. Gr ¨uning, A. (2007). Elman backpropagation as reinforcement for simple recurrent
networks. Neural Computation, 19, 3108–3131.
Gr ¨uning, A., & Sporea, I. (2012). Supervised learning of logical operations in layered spiking neural networks with spike train encoding. Neural Processing Letters, 36, 117–134. doi:10.1007/s11063-012-9225-1.
G ¨utig, R., & Sompolinsky, H. (2006). The tempotron: A neuron that learns spike timing-based decisions. Nature Neuroscience, 9, 420–428.
Harris, K. D. (2008). Stability of the fittest: Organizing learning through retroaxonal signals. Trends in Neuroscience, 31, 130–136.
Hebb, D. O. (1949). The organization of behavior. New York: Wiley.
Johansson, R. S., & Birznieks, I. (2004). First spikes in ensembles of human tactile afferents code complex spatial fingertip events. Nature Neuroscience, 7, 170–177. Legenstein, R., Naeger, C., & Maass, W. (2005). What can a neuron learn with spike-
timing-dependent plasticity? Neural Computation, 17, 2337–2382.
Knudsen, E. I. (1994). Supervised learning in the brain. Journal of Neuroscience, 14, 3985–3997.
Knudsen, E. I. (2002). Instructed learning in the auditory localization pathway of the barn owl. Nature, 417(6886), 322–328.
Maass, W. (1997a). Networks of spiking neurons: The third generation of neural network models. Transactions of the Society for Computer Simulation International,
14, 1659–1671.
Maass, W. (1997b). Fast sigmoidal networks via spiking neurons. Neural Computation,
9, 279–304.
Mauk, M. D., & Buonomano, D. V. (2004). The neural basis of temporal processing.
Annual Rev. Neuroscience, 27, 304–340.
McKennoch, S., Voegtlin, T., & Bushnell, L. (2009). Spike-timing error backpropaga- tion in theta neuron networks. Neural Computation, 21, 9–45.
Neuenschwander, S., & Singer, W. (1996). Long-range synchronization of oscillatory light responses in the cat retina and lateral geniculate nucleus. Nature, 379, 728– 733.
Ponulak, F. (2006). ReSuMe–Proof of convergence. http://d1.cie.put.poznan.pl/dav/ fp/FP_ConvergenceProof_TechRep.pdf
Ponulak, F. (2008). Analysis of the ReSuMe learning process for spiking neural networks. International Journal of Applied Mathematics and Computer Science, 18, 117–127.
Ponulak, F., & Kasi ´nski, A. (2010). Supervised learning in spiking neural networks with ReSuMe: Sequence learning, classification, and spike shifting. Neural Com-
putation, 22, 467–510.
Roelfsema, P. R., & van Ooyen, A. (2005). Attention-gated reinforcement learning of internal representations for classification. Neural Computation, 17, 1–39.
Rostro-Gonzalez, H., Vasquez-Betancour, J. C., Cessac, B., & Vi´eville, T. (2010).
Reverse-engineering in spiking neural network parameters: Exact deterministic param- eter estimation. Sophia Antipolis, France: INRIA.
Ruf, B., & Schmitt, M. (1997). Learning temporally encoded patterns in networks of spiking neurons. Neural Processing Letters, 5(1), 9–18.
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning internal representa- tions by error propagation. In D. E. Rumelhart & J. L. McClelland (Eds.), Parallel
distributed processing: Explorations in the microstructure of cognition. Cambridge,
MA: MIT Press.
Schrauwen, B., & van Campenhout, J. (2004). Improving spike-prop: Enhancements to an error-backpropagation rule for spiking neural networks. In Proceedings of
the 15th ProRISC Workshop.
Shepard, J. D., Rumbaugh, G., Wu, J., Chowdhiry, S., Plath, N., Kuhl, D., et al. (2006). Arc mediates homoeostatic synaptic scaling of AMPA receptors. Neuron,
52, 475–484.
Sporea, I., & Gr ¨uning, A. (2011). Reference time in SpikeProp. In Proceedings of IEEE
International Joint Conference Neural Networks (pp. 1090–1092). Piscataway, NJ:
IEEE.
Takase, H., Fujita, M., Kawanaka, H., Tsuruoka, S., Kita, H., & Hayashi, T. (2009). Obstacle to training SpikeProp networks: Cause of surges in training process. In
Proceedings of IEEE International Joint Conference Neural Networks (pp. 3062–3066).
Piscataway, NJ: IEEE.
Tao, H. W., Zhang, L. I, Bi, G. Q., & Poo, M. (2000). Selective presynaptic propagation of long-term potentiation in defined neural Networks. Journal of Neuroscience, 20, 3233–3243.
Thorpe, S. T., & Imbert, M. (1989). Biological constraints on connectionist modelling. In , R. Pfeifer, Z. Schreter, F. Fogelman-Souli, & L. Steels (Eds.), Connectionism in
perspective (pp. 63–92). New York: Elsevier Science.
Ti ˇno, P., & Mills, A. J. (2005). Learning beyond finite memory in recurrent networks of spiking neurons. In L. Wang, K. Chen, & Y. Ong (Eds.), Advances in natural
computation (pp. 666–675). Berlin: Springer-Verlag.
Urbanczik, R., & Senn, W. (2009). Reinforcement learning in populations of spiking neurons. Nature Neuroscience, 12, 250–252.
van Rossum, M. C. (2001). A novel spike distance. Neural Computation, 13, 751–763. Wade, J. J., McDaid, L. J., Santos, J. A., & Sayers, H. M. (2010). SWAT: A spiking
neural network training algorithm for classification problems. IEEE Transactions
on Neural Networks, 21, 1817–1829.
Watt, A. J., & Desai, N. S. (2010). Homeostatic plasticity and STDP: Keeping a neu- ron’s cool in a fluctuating world. Frontiers in Synaptic Neuroscience, 2(5). Wehr, M., & Laurent, G. (1996). Odour encoding by temporal sequences of firing in
oscillating neural assemblies. Nature, 384, 162–166.
Xin, J., & Embrechts, M. J. (2001). Supervised learning with spiking neuron networks. In Proceedings of IEEE International Joint Conference Neural Networks (pp. 1772– 1777). Piscataway, NJ: IEEE.