Geodesic
Informational capacity of the Hopfield network with~quantized weights
Prikladnaâ diskretnaâ matematika, no. 3 (2019), pp. 97-103.

Voir la notice de l'article provenant de la source Math-Net.Ru

The use of binary and multilevel memristors in the hardware neural networks implementation necessitates their weight coefficients quantization. In this paper, we investigate the Hopfield network weights quantization influence on its information capacity and resistance to input data distortions. It is shown that, for a weight level number of the order of tens, the capacitance of Hopfield–Hebb network with the quantized weights approximates the capacitance of its version with continuous weights. For a Hopfield projection network, similar result can be achieved only for a weight levels number of the order of hundreds. Experiments show that: 1) binary memristors should be used in Hopfield–Hebb networks, reduced by zeroing all weights in a given row, in which absolute values are strictly less than the maximum weight in the row; 2) in the Hopfield projection networks with quantized weights, multilevel memristors with a weight levels number significantly more than two should be used, with a specific levels number depending on the stored reference vectors dimension, their particular set and the permissible input data noise level.
Keywords: Hopfield–Hebb networks, projection Hopfield networks, information capacity, weight quantization, binary and multilevel memristors. 
@article{PDM_2019_3_a11,
     author = {M. S. Tarkov},
     title = {Informational capacity of the {Hopfield} network with~quantized weights},
     journal = {Prikladna\^a diskretna\^a matematika},
     pages = {97--103},
     publisher = {mathdoc},
     number = {3},
     year = {2019},
     language = {ru},
     url = {http://geodesic.mathdoc.fr/item/PDM_2019_3_a11/}
}
TY  - JOUR
AU  - M. S. Tarkov
TI  - Informational capacity of the Hopfield network with~quantized weights
JO  - Prikladnaâ diskretnaâ matematika
PY  - 2019
SP  - 97
EP  - 103
IS  - 3
PB  - mathdoc
UR  - http://geodesic.mathdoc.fr/item/PDM_2019_3_a11/
LA  - ru
ID  - PDM_2019_3_a11
ER  - 
%0 Journal Article
%A M. S. Tarkov
%T Informational capacity of the Hopfield network with~quantized weights
%J Prikladnaâ diskretnaâ matematika
%D 2019
%P 97-103
%N 3
%I mathdoc
%U http://geodesic.mathdoc.fr/item/PDM_2019_3_a11/
%G ru
%F PDM_2019_3_a11
M. S. Tarkov. Informational capacity of the Hopfield network with~quantized weights. Prikladnaâ diskretnaâ matematika, no. 3 (2019), pp. 97-103. http://geodesic.mathdoc.fr/item/PDM_2019_3_a11/

[1] Haykin S., Neural Networks: A Comprehensive Foundation, Prentice Hall, Inc., 1999

[2] Hopfield J., “Neural networks and physical systems with emergent collective computational abilities”, Proc. NAS USA, 79 (1982), 2554–2558 | DOI | MR | Zbl

[3] Personnaz L., Guyon I., Dreyfus G., “Collective computational properties of neural networks: New learning mechanisms”, Phys. Rev. Ser. A, 34:5 (1986), 4217–4228 | DOI | MR

[4] Michel A. N., Liu D., Qualitative Analysis and Synthesis of Recurrent Neural Networks, Marcel Dekker Inc., N.Y., 2002 | MR | Zbl

[5] Chua L., “Memristor — the missing circuit element”, IEEE Trans. Circuit Theory, 18 (1971), 507–519 | DOI

[6] Strukov D. B., Snider G. S., Stewart D. R., Williams R. S., “The missing memristor found”, Nature, 453 (2008), 80–83 | DOI

[7] He W., Sun H., Zhou Y., et al., “Customized binary and multi-level HfO$_{2-x}$-based memristors tuned by oxidation conditions”, Scientific Reports, 7 (2017), 10070 | DOI

[8] Yu S., Gao B., Fang Z., et al., “A low energy oxide-based electronic synaptic device for neuromorphic visual systems with tolerance to device variation”, Adv. Mater., 25 (2013), 1774–1779 | DOI

[9] Tarkov M. S., “Construction of a Hamming network based on a crossbar with binary memristors”, Prikladnaya Diskretnaya Matematika, 2018, no. 40, 105–113 (in Russian) | MR

[10] Tarkov M. S., “Reduction of synapses in the Hopfield autoassociative memory”, Prikladnaya Diskretnaya Matematika, 2017, no. 37, 107–113 (in Russian)

[11] Folli V., Leonetti M., Ruocco G., “On the maximum storage capacity of the Hopfield model”, Frontiers in Comput. Neuroscience, 10 (2017), 144 http://www.frontiersin.org | DOI