Associative memory based on cellular neural networks with bipolar stepwise activation function

M. S. Tarkov

Prikladnaâ diskretnaâ matematika, no. 2 (2020), pp. 100-108.

Résumé

Cellular neural networks (CNN) with a bipolar stepwise activation function are considered as obtained by training on a given set of binary reference images. The trained CNN versions with the different cell neighborhood sizes were tested in solving the problem of filtering noisy reference images. It has been established that global training methods (the Hebb method and the projection method), traditionally used in Hopfield networks, generate high-level noise (tens of percent) at the output of cellular networks even in the absence of input noise. A local analogue of the projection method is proposed that provides the filtering of noisy images significantly better than the classical local perceptron learning algorithm. The local Hebb method works better than the above two methods only with minimal neighborhood and high noise levels ($70 \%$ at least). The influence of the CNN weights quantization levels number on the CNN information capacity is investigated. It is shown that: 1) with the number of quantization levels greater than $8$ and the number of neurons $16\times 16$, the capacity of the CNN with quantized weights, trained according to the local Hebb rule, approximates the capacity of the CNN with continuous weights; 2) when using the local projection method, a similar result is achieved with a number of levels not less than $64$.

Détail

Keywords: cellular neural networks, noise filtering, perceptron training algorithm, local projection method, cell neighborhood, informational capacity of cellular neural network, weight quantization.

Bibliographie

[1] Osowski S., Neural Networks for Information Processing, Finansy i Statistika, M., 2002 (in Russian)

[2] Hopfield J., “Neural networks and physical systems with emergent collective computational abilities”, Proc. NAS USA, 79 (1982), 2554–2558 | DOI | MR | Zbl

[3] Personnaz L., Guyon I., Dreyfus G., “Collective computational properties of neural networks: new learning mechanisms”, Phys. Rev. Ser. A, 34:5 (1986), 4217–4228 | DOI | MR

[4] Michel A. N., Liu D., Qualitative Analysis and Synthesis of Recurrent Neural Networks, Marcel Dekker Inc., N.Y., 2002 | MR | Zbl

[5] Tarkov M. S., “Synapses reduction in autoassociative Hopfield network”, Proc. Int. IEEE Conf. SIBIRCON-2017 (Novosibirsk, Sept. 18–22, 2017), 158–160 https://ieeexplore.ieee.org/xpl/conhome/8099466/proceeding

[6] Chua L. O., Yang L., “Cellular neural networks: theory and application”, IEEE Trans. Circuits and Systems, CAS-35 (1988), 1257–1290 | DOI | MR

[7] Pudov S. G., “Learning of cellular neural networks”, Future Generation Computer Systems, 17 (2001), 689–697 | DOI | MR | Zbl

[8] Park J., Kim H.-Y., Park Y., Leed S.-W., “A synthesis procedure for associative memories based on space-varying cellular neural networks”, Neural Networks, 14 (2001), 107–113 | DOI

[9] Li H., Liao X., Li C., et al., “Edge detection of noisy images based on cellular neural networks”, Commun. Nonlinear Sci. Numer. Simulat., 16 (2011), 3746–3759 | DOI | MR | Zbl

[10] Duan S., Hu X., Dong Z., et al., “Memristor-based cellular nonlinear/neural network: design, analysis, and applications”, IEEE Trans. Neural Networks and Learning Systems, 26:6 (2015), 1202–1213 | DOI | MR

[11] Rosenblatt F., Principles of Neurodynamics, Spartan, Washington, 1959

[12] Chua L., “Memristor — the missing circuit element”, IEEE Trans. Circuit Theory, 18 (1971), 507–519 | DOI

[13] Strukov D. B., Snider G. S., Stewart D. R., Williams R. S., “The missing memristor found”, Nature, 453 (2008), 80–83 | DOI

[14] He W., Sun H., Zhou Y., et al., “Customized binary and multi-level HfO$_{2-x}$-based memristors tuned by oxidation conditions”, Scientific Reports, 7 (2017), 10070 | DOI