Spiking neural network with local plasticity and sparse connectivity for audio classification

R. B. Rybka ; D. S. Vlasov ; A. I. Manzhurov ; A. V. Serenko ; A. G. Sboev

Izvestiya VUZ. Applied Nonlinear Dynamics, Tome 32 (2024) no. 2, pp. 239-252.

Résumé

Purpose. Studying the possibility of implementing a data classification method based on a spiking neural network, which has a low number of connections and is trained based on local plasticity rules, such as Spike-Timing-Dependent Plasticity. Methods. As the basic architecture of a spiking neural network we use a network included an input layer and layers of excitatory and inhibitory spiking neurons (Leaky Integrate and Fire). Various options for organizing connections in the selected neural network are explored. We have proposed a method for organizing connectivity between layers of neurons, in which synaptic connections are formed with a certain probability, calculated on the basis of the spatial arrangement of neurons in the layers. In this case, a limited area of connectivity leads to a higher sparseness of connections in the overall network. We use frequency-based coding of data into spike trains, and logistic regression is used for decoding. Results. As a result, based on the proposed method of organizing connections, a set of spiking neural network architectures with different connectivity coefficients for different layers of the original network was implemented. A study of the resulting spiking network architectures was carried out using the Free Spoken Digits dataset, consisting of 3000 audio recordings corresponding to 10 classes of digits from 0 to 9. Conclusion. It is shown that the proposed method of organizing connections for the selected spiking neural network allows reducing the number of connections by up to 60

Détail

Keywords: spiking neural network, STDP, sparse connectivity, free spoken digits dataset, audio classification

Bibliographie

[1] Davies M., Srinivasa N., Lin T.-H., Chinya G., Cao Y., Choday S.H., Dimou G., Joshi P., Imam N., Jain S., Liao Y., Lin C.-K., Lines A., Liu R., Mathaikutty D., McCoy S., Paul A., Tse J., Venkataramanan G., Weng Y.-H., Wild A., Yang Y., Wang H., “Loihi: A neuromorphic manycore processor with on-chip learning”, IEEE Micro, 38:1 (2018), 82–99 | DOI

[2] Merolla P.A., Arthur J.V., Alvarez-Icaza R., Cassidy A.S., Sawada J., Akopyan F., Jackson B.L., Imam N., Guo C., Nakamura Y., Brezzo B., Vo I., Esser S.K., Appuswamy R., Taba B., Amir A., Flickner M.D., Risk W.P., Manohar R., Modha D.S., “A million spiking-neuron integrated circuit with a scalable communication network and interface”, Science, 345:6197 (2014), 668–673 | DOI

[3] Matsukatova A.N., Vdovichenko A.Y., Patsaev T.D., Forsh P.A., Kashkarov P.K., Demin V.A., Emelyanov A.V., “Scalable nanocomposite parylene-based memristors: Multifilamentary resistive switching and neuromorphic applications”, Nano Research, 16:2 (2023), 3207–3214 | DOI

[4] Matsukatova A.N., Iliasov A.I., Nikiruy K.E., Kukueva E.V., Vasiliev A.L., Goncharov B.V., Sitnikov A.V., Zanaveskin M.L., Bugaev A.S., Demin V.A., Rylkov V.V., Emelyanov A.V., “Convolutional neural network based on crossbar arrays of (Co-Fe-B)$\times$(LiNbO3)100-x nanocomposite memristors”, Nanomaterials, 12:19 (2022), 3455 | DOI

[5] Bordanov I., Antonov A., Korolev L., “Simulation of calculation errors in memristive crossbars for artificial neural networks”, 2023 International Conference on Industrial Engineering, Applications and Manufacturing, ICIEAM (15-19 May 2023, Sochi, Russian Federation), IEEE, 2023, 1008–1012 | DOI

[6] Vlasov D., Minnekhanov A., Rybka R., Davydov Y., Sboev A., Serenko A., Ilyasov A., Demin V., “Memristor-based spiking neural network with online reinforcement learning”, Neural Networks, 2023, no. 166, 512–523 | DOI

[7] Tao T., Li D., Ma H., Li Y., Tan S., Liu E.-X., Schutt-Aine J., Li E.-P., “A new pre-conditioned STDP rule and its hardware implementation in neuromorphic crossbar array”, Neurocomputing, 557 (2023), 126682 | DOI

[8] Sboev A., Serenko A., Rybka R., Vlasov D., “Solving a classification task by spiking neural network with STDP based on rate and temporal input encoding”, Mathematical Methods in the Applied Sciences, 43:13 (2020), 7802–7814 | DOI | MR | Zbl

[9] Sboev A., Vlasov D., Rybka R., Davydov Y., Serenko A., Demin V., “Modeling the dynamics of spiking networks with memristor-based STDP to solve classification tasks”, Mathematics, 9:24 (2021), 3237 | DOI

[10] Diehl P.U., Cook M., “Unsupervised learning of digit recognition using spike-timing-dependent plasticity”, Frontiers in Computational Neuroscience, 9 (2015), 99 | DOI | MR

[11] Sboev A., Davydov Y., Rybka R., Vlasov D., Serenko A., “A comparison of two variants of memristive plasticity for solving the classification problem of handwritten digits recognition”, Biologically Inspired Cognitive Architectures 2021, v. 1032, Studies in Computational Intelligence, eds. Klimov V.V., Kelley D.J., Springer, Cham, 2022, 438–446 | DOI

[12] Vlasov D., Davydov Y., Serenko A., Rybka R., Sboev A., “Spoken digits classification based on spiking neural networks with memristor-based STDP”, 2022 International Conference on Computational Science and Computational Intelligence, CSCI (14-16 December 2022, Las Vegas, NV, USA), IEEE, 2022, 330–335 | DOI

[13] Lien H.-H., Chang T.-S., “Sparse compressed spiking neural network accelerator for object detection”, IEEE Transactions on Circuits and Systems I: Regular Papers, 69:5 (2022), 2060–2069 | DOI

[14] Tsai C.-C., Yang Y.-H., Lin H.-W., Wu B.-X., Chang E.C., Liu H.Y., Lai J.-S., Chen P.Y., Lin J.-J., Chang J.S., Wang L.-J., Kuo T.T., Hwang J.-N., Guo J.-I., “The 2020 embedded deep learning object detection model compression competition for traffic in Asian countries”, 2020 IEEE International Conference on Multimedia Expo Workshops, ICMEW (06-10 July 2020, London, UK), IEEE, 2020, 1–6 | DOI

[15] Han B., Zhao F., Zeng Y., Pan W., Adaptive sparse structure development with pruning and regeneration for spiking neural networks, 2022, 9 pp., arXiv: 2211.12219 | DOI

[16] Amiri M., Jafari A.H., Makkiabadi B., Nazari S., “Recognizing intertwined patterns using a network of spiking pattern recognition platforms”, Scientific Reports, 12:1 (2022), 19436 | DOI

[17] Timcheck J., Shrestha S.B., Rubin D.B.D., Kupryjanow A., Orchard G., Pindor L., Shea T., Davies M., “The intel neuromorphic DNS challenge”, 2023, 13 pp., arXiv: 2303.09503 | DOI

[18] McFee B., Raffel C., Liang D., Ellis D.P.W., McVicar M., Battenberg E., Nieto O., “librosa: Audio and music signal analysis in python”, Proceedings of the 14th Python in Science Conference, SciPy, 2015, 18–24 | DOI

[19] Morrison A., Diesmann M., Gerstner W., “Phenomenological models of synaptic plasticity based on spike timing”, Biological Cybernetics, 98:6 (2008), 459–478 | DOI | MR | Zbl

[20] Bergstra J., Yamins D., Cox D.D., “Hyperopt: a Python library for model selection and hyperparameter optimization”, Proceedings of the 12th Python in Science Conference, SciPy, 2013, 13–19 | DOI