Prediction of the integrated indicator of quality of a new object under the conditions of multicollinearity of reference data

S. B. Achlyustin   ; A. V. Melnikov   ; R. A. Zhilin

Vestnik Ûžno-Uralʹskogo gosudarstvennogo universiteta. Seriâ, Matematičeskoe modelirovanie i programmirovanie, Tome 13 (2020) no. 4, pp. 66-80

Résumé

Prediction of a new object state at a lack of the known characteristics and estimates of quality indicators of a number of studied objects (a set of reference data) often leads to the problem of multicollinearity of basic data. We propose the following three ways to overcome this problem relating to the sphere of data mining: use a ridge regression, train with the teacher a two-layer neural network, consecutive adapt a single-layer neural network. Also, we compare characteristics of the proposed ways. In the ridge regression method, the introduction of a regularizing term into the LMS equation gives an approximate solution with a sufficient degree of accuracy. A disadvantage of use of the two-layer neural network “feed-forward backprop” and the procedure of training with the teacher “train” is that adjusted weights of the neural network take chaotic (and even negative) values that contradicts a common practice of examination. The following features are revealed: considerable dispersion of weights and shifts of a neural network, ambiguity of the solution due to the choice of random initial conditions, strong dependence on a training algorithm. In order to overcome this shortcoming, we propose a transition to consecutive adaptation of a single-layer neural network with fixing shifts of neurons at zero level.

Détail

Keywords: examination of objects, prediction, ridge regression, regularization, neural network, training of model
Mots-clés : multiple regression, adaptation.

Bibliographie

[1] Bukharin S. V., Melnikov A. V., Menshih V. V., “Method for Expanding the Object Base of Examination by Stitching Solutions of Hierarchy Analysis Method”, Bulletin of the South Ural State University. Series: Mathematical Modelling, Programming and Computer Software, 10:2 (2017), 74–82 | DOI

[2] Esbensen K. H., Multivariable Data Analysis in Practice, CAMO Software AS, Oslo, 2019

[3] Dreyper N., Smit G., Applied Regression Analysis, John Wiley and Sons, N.Y., 2007 | MR

[4] Ayvazyan S. A. et al., Application Statistics. Classification and Reduction of Dimension, Finance and Statistics, M., 1989 | MR

[5] Magnus Ya.R., Katyshev P. K., Peresetsky A. A., Econometrics, Delo, M., 1997 | MR

[6] Witten I. H., Frank E., Hall M. A., Data Mining: Practical Machine Learning Tools and Techniques, Morgan Kaufmann, N.Y., 2011

[7] Bukharin S., Navoev V., Methods of the Theory of Neural Networks in the Examination of Technical and Economic Objects, Scientific Book, Voronezh, 2015 (in Russian)

[8] Akhlyustin S. et al., “Fuzzy-Multiple Modeling of Generalized Hazard Indicators of Fuel and Energy Complex Facilities”, Bulletin of the Voronezh Institute of the Ministry of Internal Affairs of Russia, 2018, no. 3, 44–55 (in Russian)

[9] Melnikov A., Akhlyustin S., “Determination of the Particular Criterion of Integrated Security Systems by the Symmetric Method of Hierarchy Analysis”, Bulletin of the Voronezh Institute of the Ministry of Internal Affairs of Russia, 2018, no. 2, 37–44 (in Russian)

[10] Melnikov A. et al., “Neural Modeling of the Generalized Quality Indicator of the Repeaters of the Notification Transmission Systems”, Bulletin of the Voronezh Institute of the Ministry of Internal Affairs of Russia, 2015, no. 1, 17–26 (in Russian)

[11] T. L. Saaty, “Relative Measurement and Its Generalization in Decision Making Why Pairwise Comparisons Are Central in Mathematics for the Measurement of Intangible Factors the Analytic Hierarchy/Network Process”, Revista de la Real Academia de Ciencias Exactas, F$\acute{\text{i}}$sicas y Naturales. Serie A. Matem$\acute{a}$ticas, 102:2 (2008), 251–318 | DOI | MR | Zbl

[12] Akhlyustin S. et al., “Formation of a Complex Indicator of the Quality of Objects with Poorly Formalized Signs”, Digitization of the Agro-Industrial Complex, Tambov State Technical University, Tambov, 2018, 185–187 (in Russian)

[13] Tikhonov A., Arsenin V., Methods for Solving Ill-Posed Problems, Nauka, M., 1986 (in Russian)

[14] Haykin S., Neural Networks: a Comparison Foundation, Prentice Hall, New Jersey, 2001

[15] Medvedev V., Potemkin V., Neural Networks. Matlab 6, Dialog-MYTHI, M., 2002 (in Russian)

[16] Yakhyayeva G., Fuzzy Sets and Neural Networks, Internet University of Information Technologies, M., 2011 (in Russian)

[17] Diligensky N., Dymova L., Sevastyanov P., Fuzzy Modeling and Multi-Criteria Optimization of Production Systems in Conditions of Uncertainty: Technology, Economics, Ecology, Mashinostroenie, M., 2004 (in Russian)

[18] Bukharin S. et al., “Prediction of a Generalized Indicator of the Quality of Protection Devices Based on Ridge Regression and Neural Modeling”, Bulletin of the Voronezh Institute of the Ministry of Internal Affairs of Russia, 2014, no. 1, 57–65 (in Russian)

[19] Avsentiev O. S., Meshcheryakova T. V., Navoev V. V., “Sequential Application of the Hierarchy Analysis Method and Associative Training of Neural Network in Examination Problems”, Bulletin of the South Ural State University. Series: Mathematical Modelling, Programming and Computer Software, 10:3 (2017), 142–147 | DOI

[20] Boyarkin D. A., Krupenev D. S., Yakubovsky D. V., “Using Machine Training Methods for Assessing the Reliability of Electric Power Systems by the Monte–Carlo Method”, Bulletin of the South Ural State University. Series: Mathematical Modelling, Programming and Computer Software, 11:4 (2018), 146–153 (in Russian) | DOI