Reduction of synapses in the Hopfield autoassociative memory

M. S. Tarkov

Prikladnaâ diskretnaâ matematika, no. 3 (2017), pp. 107-113

Résumé

The auto-associative Hopfield network is a set of neurons in which the output of each neuron is the input of all other neurons, i.e. the inter-neuronal connections graph of the Hopfield network is complete. The large number of inter-neuronal connections is one of the problems of the Hopfield networks hardware implementation. A solution is the reduction (exclusion) of insignificant connections. In this paper, based on the analogy with oscillator networks, the connections number reducing effect on the auto-associative Hopfield network behavior is investigated. It is shown that the exclusion of connections with weights whose absolute values are strictly less than the maximum for a given neuron substantially improves the operation quality of the Hopfield network trained according to the Hebb's rule. As the dimension of the stored vectors increases, not only the chimeras disappear but the permissible input data noise level also increases. At the same time, the network connections number is reduced by 13–17 times. The reduction of connections in the Hopfield network, trained by the projection method, worsens its functioning quality, namely: in the network output data, there are distortions even while the reference vectors are entered. With the stored vectors dimension increasing, the allowable noise level for the reduced Hopfield–Hebb networks approaches the corresponding index for the Hopfield projection networks. Thus, given the much smaller number of connections in the reduced Hopfield–Hebb networks, these networks can successfully compete with the Hopfield projection networks for a sufficiently large stored vectors dimension.

Détail

Keywords: auto-associative Hopfield memory, Hebb's rule, projection method, oscillatory network, reduction of connections.

Bibliographie

[1] Osovskiy S., Neural networks for data processing, Finansy i Statistika, Moscow, 2002, 344 pp. (in Russian)

[2] Hopfield J., “Neural networks and physical systems with emergent collective computational abilities”, Proc. National Academy of Science USA, 79 (1982), 2554–2558 | DOI | MR | Zbl

[3] Personnaz L., Guyon I., Dreyfus G., “Collective computational properties of neural networks: New learning mechanisms”, Phys. Rev. A, 34:5 (1986), 4217–4228 | DOI | MR

[4] Michel A. N., Liu D., Qualitative Analysis and Synthesis of Recurrent Neural Networks, Marcel Dekker Inc., N.Y., 2002, 504 pp. | MR | Zbl

[5] Hu S. G., Liu Y., Liu Z., et al., “Associative memory realized by a reconfigurable memristive Hopfield neural network”, Nature Commun., 6 (2015), Article no. 7522

[6] Tarkov M. S., “Hopfield Network with interneuronal connections on the base of memristor bridges”, Proc. NEIROINFORMATIKA-2016, Part 3, MEPhI Publ., Moscow, 2016, 91–100 (in Russian)

[7] Tarkov M. S., “Hopfield network with interneuronal connections based on memristor bridges”, LNCS, 9719, 2016, 196–203

[8] Tarkov M. S., “Oscillatory neural associative memories with synapses based on memristor bridges”, Optical Memory and Neural Networks (Information Optics), 25:4 (2016), 219–227 | DOI

[9] Gorban A. N., Rossiev D. A., Neural networks on personal computer, Nauka Publ., Novosibirsk, 1996, 276 pp. | MR

[10] Maffezzoni P., Bahr B., Zhang Z., Daniel L., “Analysis and design of Boolean associative memories made of resonant oscillator arrays”, IEEE Trans. Circuits Systems I: Regular Papers, 63:8 (2016), 1964–1973 | DOI | MR