A spiking binary neuron --- detector of causal links

M. V. Kiselev ; D. A. Larionov ; A. M. Urusov

Izvestiya VUZ. Applied Nonlinear Dynamics, Tome 32 (2024) no. 5, pp. 589-605.

Résumé

Purpose. Causal relationship recognition is a fundamental operation in neural networks aimed at learning behavior, action planning, and inferring external world dynamics. This operation is particularly crucial for reinforcement learning (RL). In the context of spiking neural networks (SNNs), events are represented as spikes emitted by network neurons or input nodes. Detecting causal relationships within these events is essential for effective RL implementation. Methods. This research paper presents a novel approach to realize causal relationship recognition using a simple spiking binary neuron. The proposed method leverages specially designed synaptic plasticity rules, which are both straightforward and efficient. Notably, our approach accounts for the temporal aspects of detected causal links and accommodates the representation of spiking signals as single spikes or tight spike sequences (bursts), as observed in biological brains. Furthermore, this study places a strong emphasis on the hardware-friendliness of the proposed models, ensuring their efficient implementation on modern and future neuroprocessors. Results. Being compared with precise machine learning techniques, such as decision tree algorithms and convolutional neural networks, our neuron demonstrates satisfactory accuracy despite its simplicity. Conclusion. We introduce a multi-neuron structure capable of operating in more complex environments with enhanced accuracy, making it a promising candidate for the advancement of RL applications in SNNs.

Détail

Keywords: spiking neural network, binary neuron, spike timing dependent plasticity, dopamine-modulated plasticity, anti-Hebbian plasticity, reinforcement learning, neuromorphic hardware

Bibliographie

[1] Moreno-Bote R., Drugowitsch J., “Causal Inference and Explaining Away in a Spiking Network”, Scientific Reports, 5 (2015), 17531 | DOI

[2] Lansdell B. J., Kording K. P., “Neural spiking for causal inference and learning”, PLoS Computational Biology, 19:4 (2023), e1011005 | DOI

[3] Skatchkovsky N., Jang O., Simeone O., “Bayesian continual learning via spiking neural networks”, Frontiers in Computational Neuroscience, 16 (2022), 1037976 | DOI

[4] Friston K., “The history of the future of the bayesian brain”, Neuroimage, 62:2 (2012), 1230–1233 | DOI

[5] Kasabov N., Scott N. M., Tu E., Marks S., Sengupta N., Capecci E., Othman M., Doborjeh M. G., Murli N., Hartono R., Espinosa-Ramos J. I., Zhou L., Alvi F. B., Wang G., Taylor D., Feigin V., Gulyaev S., Mahmoud M., Hou Z.-G., Yang J., “Evolving spatio-temporal data machines based on the NeuCube neuromorphic framework: Design methodology and selected applications”, Neural Networks, 78 (2016), 1–14 | DOI | Zbl

[6] Kasabov N. K., “NeuCube: a spiking neural network architecture for mapping, learning and understanding of spatio-temporal brain data”, Neural Networks, 52 (2014), 62–76 | DOI

[7] Schliebs S., Fiasché M., Kasabov N., “Constructing robust liquid state machines to process highly variable data streams”, International Conference on Artificial Neural Networks ICANN 2012: Artificial Neural Networks and Machine Learning — ICANN, 2012, 604–611 | DOI

[8] Doborjeh M., Doborjeh Z., Merkin A., Krishnamurthi R., Enayatollahi R., Feigin V., Kasabov N., “Personalized Spiking Neural Network Models of Clinical and Environmental Factors to Predict Stroke”, Cognitive Computation, 14 (2022), 2187–2202 | DOI

[9] Fernando C., “From blickets to synapses: Inferring temporal causal networks by observation”, Cognitive Science, 37:8 (2013), 1426–1470 | DOI

[10] Markram H., Gerstner W., Sjöström P. J., “A history of spike-timing-dependent plasticity”, Frontiers in Synaptic Neuroscience, 3 (2011), 4 | DOI

[11] Kerr R. R., Grayden D. B., Thomas D. A., Gilson M., Burkitt A. N., “Coexistence of Reward and Unsupervised Learning During the Operant Conditioning of Neural Firing Rates”, PLoS ONE, 9:1 (2014), e87123 | DOI

[12] Yuan M., Wu X., Yan R., Tang H., “Reinforcement Learning in Spiking Neural Networks with Stochastic and Deterministic Synapses”, Neural Computation, 31:12 (2019), 2368–2389 | DOI | MR | Zbl

[13] Mozafari M., Ganjtabesh M., Nowzari-Dalini A., Thorpe S. J., Masquelier T., “Bio-Inspired Digit Recognition Using Reward-Modulated Spike-Timing-Dependent Plasticity in Deep Convolutional Networks”, Pattern Recognition, 94 (2019), 87–95 | DOI

[14] Frémaux N., Sprekeler H., Gerstner W., “Functional Requirements for Reward-Modulated Spike-Timing-Dependent Plasticity”, The Journal of Neuroscience, 30:40 (2010), 13326–13337 | DOI

[15] Juarez-Lora A., Ponce-Ponce V. H., Sossa H., Rubio-Espino E., “R-STDP Spiking Neural Network Architecture for Motion Control on a Changing Friction Joint Robotic Arm”, Frontiers in Neurorobotics, 16 (2022), 904017 | DOI

[16] Ivanov D., Chezhegov A., Kiselev M., Grunin A., Larionov D., “Neuromorphic artificial intelligence systems”, Frontiers in Neuroscience, 16 (2022), 959626 | DOI

[17] Kiselev M., Ivanov A., Ivanov D., “Approximating Conductance-Based Synapses by Current-Based Synapses”, Advances in Neural Computation, Machine Learning, and Cognitive Research IV. Neuroinformatics 2020. Studies in Computational Intelligence, 925 (2020), 394–402 | DOI

[18] Kiselev M. V., Proceedings of International Joint Conference on Neural Networks, 2017, 3806–3813 | DOI

[19] Ho V. M., Lee J. A., Martin K. C., “The cell biology of synaptic plasticity”, Science, 334:6056 (2011), 623–628 | DOI | MR

[20] Citri A., Malenka R. C., “Synaptic Plasticity: Multiple Forms, Functions, and Mechanisms”, Neuropsychopharmacology Reviews, 33 (2008), 18–41 | DOI

[21] Roberts P. D., Leen T. K., “Anti-hebbian spike-timing-dependent plasticity and adaptive sensory processing”, Frontiers in Computational Neuroscience, 4 (2010), 156 | DOI

[22] Jiajun F., “A Review for Deep Reinforcement Learning in Atari: Benchmarks, Challenges, and Solutions”, 2022, arXiv: abs/2112.04145