Autoassociative Hamming Neural Network

E. S. Antipova ; S. A. Rashkovskiy

Russian journal of nonlinear dynamics, Tome 17 (2021) no. 2, pp. 175-193.

Résumé

An autoassociative neural network is suggested which is based on the calculation of Hamming distances, while the principle of its operation is similar to that of the Hopfield neural network. Using standard patterns as an example, we compare the efficiency of pattern recognition for the autoassociative Hamming network and the Hopfield network. It is shown that the autoassociative Hamming network successfully recognizes standard patterns with a degree of distortion up to 40% and more than 60%, while the Hopfield network ceases to recognize the same patterns with a degree of distortion of more than 25% and less than 75%. A scheme of the autoassociative Hamming neural network based on McCulloch–Pitts formal neurons is proposed. It is shown that the autoassociative Hamming network can be considered as a dynamical system which has attractors that correspond to the reference patterns. The Lyapunov function of this dynamical system is found and the equations of its evolution are derived.

Détail

Keywords: autoassociative Hamming network, Hopfield network, iterative algorithm, pattern recognition, dynamical system, neurodynamics, attractors, stationary states.

Bibliographie

[1] Hopfield, J. J., “Neural Networks and Physical Systems with Emergent Collective Computational Abilities”, Proc. Natl. Acad. Sci. USA, 79:9 (1982), 2554–2558 | DOI | MR | Zbl

[2] Haykin, S., Neural Networks: A Comprehensive Foundation, Macmillan, New York, 1994, 696 pp. | Zbl

[3] Lippmann, R. P., “An Introduction to Computing with Neural Nets”, IEEE ASSP Magazine, 4:2 (1987), 4–22 | DOI

[4] Ikeda, N., Watta, P., Artiklar, M., and Hassoun, M. H., “A Two-Level Hamming Network for High Performance Associative Memory”, Neural Netw., 14:9 (2001), 1189–1200 | DOI

[5] Yongsheng, H., HuaMei, D., and Zhongbin, T., “A Logistical Model Based on the Hamming Competitive Neural Network Algorithm”, J. Appl. Sci., 14:2 (2014), 129–136 | DOI

[6] Fan, L., “Revisit Fuzzy Neural Network: Demystifying Batch Normalization and ReLU with Generalized Hamming Network”, 31st Conf. on Neural Information Processing Systems (Long Beach, Calif., 2017), 1923–1932

[7] Koutroumbas, K. and Kalouptsidis, N., “Generalized Hamming Networks and Applications”, Neural Netw., 18:7 (2005), 896–913 | DOI | Zbl

[8] Schmid, A., Leblebici, Y., and Mlynek, D., “Hardware Realization of a Hamming Neural Network with On-Chip Learning”, Proc. of the 1998 IEEE Internat. Symp. on Circuits and Systems (ISCAS'98), v. 3, 98CH36187, 191–194

[9] Norouzi, M., Fleet, D. J., and Salakhutdinov, R. R., “Hamming Distance Metric Learning”, NIPS'12: Proc. of the 25th Internat. Conf. on Neural Information Processing Systems, v. 1, 1061–1069

[10] Khristodulo, O. I., Makhmutov, A. A., and Sazonova, T. V., “Use Algorithm Based at Hamming Neural Network Method for Natural Objects Classification”, Procedia Comput. Sci., 103 (2017), 388–395 | DOI

[11] Kovačević, V. B., Gavrovska, A. M., and Paskaš, M. P., “High-Speed Implementation of Hamming Neural Network”, Proc. of the 10th Symp. on Neural Network Applications in Electrical Engineering (Belgrade, Serbia, Sept 2010), 167–170

[12] Lu, W., Li, Z., and Shi, B., “A Modified Hamming Neural Network”, Proc. of the 4th Internat. Conf. on Solid-State and Integrated Circuit Technology (Beijing, China, Oct 24–28, 1995), 694–696 | Zbl

[13] Klimov, V. S., Klimov, A. S., and Mkrtychev, S. V., “Computer Diagnostics of Resistance Spot Welding Based on Hamming Neural Network”, J. Phys. Conf. Ser., 1333:4 (2019), 042015, 6 pp. | DOI

[14] Denker, J. S., “Neural Network Models of Learning and Adaptation”, Phys. D, 22:1–3 (1986), 216–232 | DOI | MR