Method of automatic search for the structure and parameters of~neural networks for solving information processing problems

A. D. Obukhov

Izvestiya of Saratov University. Mathematics. Mechanics. Informatics, Tome 23 (2023) no. 1, pp. 113-125.

Résumé

Neural networks are actively used in solving various applied problems of data analysis, processing and generation. When using them, one of the difficult stages is the selection of the structure and parameters of neural networks (the number and types of layers of neurons, activation functions, optimizers, and so on) that provide the greatest accuracy and, therefore, the success of solving the problem. Currently, this issue is being solved by analytical selection of the neural network architecture by a researcher or software developer. Existing automatic tools (AutoKeras, AutoGAN, AutoSklearn, DEvol and others) are not universal and functional enough. Therefore, within the framework of this work, a method of automatic search for the structure and parameters of neural networks of various types (multilayer dense, convolutional, generative-adversarial, autoencoders, and others) is considered for solving a wide class of problems. The formalization of the method and its main stages are presented. The approbation of the method is considered, which proves its effectiveness in relation to the analytical solution in the selection of the architecture of the neural network. A comparison of the method with existing analogues is carried out, its advantage is revealed in terms of the accuracy of the formed neural networks and the time to find a solution. The research results can be used to solve a large class of data processing problems for which it is required to automate the selection of the structure and parameters of a neural network.

Bibliographie

[1] Beskrovny A. S., Bessonov L. V., Ivanov D. V., Kirillova I. V., Kossovich L. Yu., “Using the mask-RCNN convolutional neural network to automate the construction of two-dimensional solid vertebral models”, Izvestiya of Saratov University. Mathematics. Mechanics. Informatics, 20:4 (2020), 502–516 (in Russian) | DOI | MR

[2] Li H., Yuan D., Ma X., Cui D., Cao L., “Genetic algorithm for the optimization of features and neural networks in ECG signals classification”, Scientific Reports, 7:1 (2017), 1–12 | DOI

[3] He X., Zhao K., Chu X., “AutoML: A Survey of the State-of-the-Art”, Knowledge-Based Systems, 212 (2021), 106622 | DOI

[4] Jin H., Song Q., Hu X., “Auto-keras: An efficient neural architecture search system”, Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2019, 1946–1956 | DOI

[5] Real E., Aggarwal A., Huang Y., Le Q. V., “Regularized evolution for image classifier architecture search”, Proceedings of the AAAI Conference on Artificial Intelligence, 33 (2019), 4780–4789 | DOI

[6] Budjac R., Nikmon M., Schreiber P., Zahradnikova B., Janacova D., “Automated machine learning overview”, Research Papers Faculty of Materials Science and Technology Slovak University of Technology, 27:45 (2019), 107–112 | DOI

[7] Feurer M., Klein A., Eggensperger K., Springenberg J., Blum M., Hutter F., “Efficient and robust automated machine learning”, Automated Machine Learning, 2019, 113–134 | DOI | MR

[8] Le Q., Zoph B., Using machine learning to explore neural network architecture, Google Research Blog, , 2017 (accessed 5 April 2022) https://research.googleblog.com/2017/05/using-machine-learning-to-explore.html

[9] Gong X., Chang S., Jiang Y., Wang Z., “Autogan: Neural architecture search for generative adversarial networks”, Proceedings of the IEEE International Conference on Computer Vision, 2019, 3224–3234 | DOI

[10] Liu C., Zoph B., Neumann M., Shlens J., Hua W., Li L. J., Murphy K., “Progressive Neural Architecture Search”, Computer Vision - ECCV 2018, ECCV 2018, Lecture Notes in Computer Science, 11205, eds. V. Ferrari, M. Hebert, C. Sminchisescu, Y. Weiss, Springer, Cham, 2018, 19–34 | DOI

[11] Cai H., Gan C., Han S., Once for all: Train one network and specialize it for efficient deployment, arXiv: 1908.09791

[12] Truong A., Walters A., Goodsitt J., Hines K., Bruss C. B., Farivar R., “Towards automated machine learning: Evaluation and comparison of AutoML approaches and tools”, 2019 IEEE 31st International Conference on Tools with Artificial Intelligence, 2019, 1471–1479 | DOI

[13] Moen E., Bannon D., Kudo T., Graf W., Covert M., Van Valen D., “Deep learning for cellular image analysis”, Nature Methods, 16:12 (2019), 1233–1246 | DOI

[14] Obukhov A., Krasnyanskiy M., “Quality assessment method for GAN based on modified metrics inception score and Frechet inception distance”, Proceedings of the Computational Methods in Systems and Software, 2020, 102–114 | DOI

[15] Yann LeCun, Corinna Cortes, Chris Burges, MNIST handwritten digit database, (accessed 5 April 2022) http://yann.lecun.com/exdb/mnist/

[16] UCI Machine Learning Repository, (accessed 5 April 2022) https://archive.ics.uci.edu/ml/datasets.php

[17] Obukhov A., Siukhin A., Dedov D., “The model of the automatic control system for a treadmill based on neural networks”, 2020 International Multi-Conference on Industrial Engineering and Modern Technologies (FarEastCon), 2020, 1–5 | DOI

[18] Obukhov A., Krasnyanskiy M., “Neural network method of data processing and transmission in adaptive information systems”, Vestnik Udmurtskogo Universiteta. Matematika. Mekhanika. Komp'yuternye Nauki, 31:1 (2021), 149–164 (in Russian) | DOI | MR