The method of successive approximations for constructing a model of dynamic polynomial regression

A. G. Golovkina ; V. A. Kozynchenko ; I. S. Klimenko

Vestnik Sankt-Peterburgskogo universiteta. Prikladnaâ matematika, informatika, processy upravleniâ, Tome 18 (2022) no. 4, pp. 487-500

Résumé

Predicting the behavior of a certain process in time is an important task that arises in many applied areas, and information about the system that generated this process can either be completely absent or be partially limited. The only available knowledge is the accumulated data on past states and process parameters. Such a task can be successfully solved using machine learning methods, but when it comes to modeling physical experiments or areas where the ability of a model to generalize and interpretability of predictions are important, then the most machine learning methods do not fully satisfy these requirements. The forecasting problem is solved by building a dynamic polynomial regression model, and a method for finding its coefficients is proposed, based on the connection with dynamic systems. Thus, the constructed model corresponds to a deterministic process, potentially described by differential equations, and the relationship between its parameters can be expressed in an analytical form. As an illustration of the applicability of the proposed approach to solving forecasting problems, we consider a synthetic data set generated as a numerical solution of a system of differential equations that describes the Van der Pol oscillator.

Détail

Mots-clés : polynomial regression
Keywords: dynamic systems, Taylor map.

Bibliographie

[1] Jansson M., Wahlberg B., “A linear regression approach to state-space subspace system identification”, Signal Processing, 2 (1996), 103–129 | DOI

[2] Herceg S., Ujevic Z., Bolf A. N., “Development of soft sensors for isomerization process based on support vector machine regression and dynamic polynomial models”, Chemical Engineering Research and Design, 149 (2019), 95–103 | DOI

[3] Dette H., “Optimal designs for identifying the degree of a polynomial regression”, Annals of Statistics, 23:4 (1995), 1248–1266 | DOI | MR

[4] Kim B., Ko Ch.-Y., Wong N., “Tensor network subspace identification of polynomial state space models”, Automatica, 95 (2018), 187–196 | DOI | MR

[5] Blondel M., Ishihata M., Fujino A., Ueda N., “Polynomial networks and factorization machines: New insights and efficient training algorithms”, Proceedings of ICML 2016 (New York City, 2016), 850–858

[6] Blondel M., Niculae V., Otsuka T., Ueda N., “Multi-output polynomial networks and factorization machines”, Proceedings of the 31$^{\rm st}$ International Conference on Neural Information Processing Systems, NIPS'17 (Long Beach, 2017), 3351–3361

[7] Chen T., Guestrin C., “XGBoost: A Scalable tree boosting system”, Proceedings of the 22$^{\rm nd}$ ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD'16 (San Francisco, 2016), 785–794 | DOI

[8] Hornik K., Stinchcombe M., White H., “Multilayer feedforward networks are universal approximators”, Neural Networks, 2 (1989), 359–366 | DOI | MR

[9] Rao S., Sethuraman S., Ramamurthi V., “A recurrent neural network for nonlinear time series prediction: a comparative study”, IEEE Workshop on Neural Networks for Signal Processing (NNSP'92) (Helsingoer, 1992), 531–539

[10] Kaheman K., Kutz J. N., Brunton S. L., “SINDy-PI: a robust algorithm for parallel implicit sparse identification of nonlinear dynamics”, Proceedings of the Royal Society A, 476:2242 (2020), 20200279 | DOI | MR

[11] Brunton S. L., Joshua L. P., Kutz N., “Discovering governing equations from data by sparse identification of nonlinear dynamical systems”, Proceedings of the National Academy of Sciences, 113:15 (2016), 3932–3937 | DOI | MR

[12] Andrianov S. N., Dynamical modeling of control systems for particle beams, St Petersburg University Press, St Petersburg, 2004, 368 pp.

[13] Dragt A., Lie methods for nonlinear dynamics with applications to accelerator physics, 2011 (accessed: August 10, 2022) inspirehep.net/record/955313/files/TOC28Nov2011.pdf

[14] Andrianov S., “Symbolic computation of approximate symmetries for ordinary differential equations”, Mathematics and Computers in Simulation, 57:3–5 (2001), 147–154 | DOI | MR

[15] Andrianov S., “A matrix representation of the Lie transformation”, Proceedings of the Abstracts of the International Congress on Computer Systems and Applied Mathematics (St Petersburg, 1993), v. 14, 19–23

[16] Ivanov A., Golovkina A., Iben U., “Polynomial neural networks and taylor maps for dynamical systems simulation and learning”, 24$^{\rm th}$ European Conference on Artificial Intelligence, including 10$^{\rm th}$ Conference on Prestigious Applications of Artificial Intelligence, PAIS 2020, Proceedings, Frontiers in Artificial Intelligence and Applications, IOS Press, 2020, 1230–1237 | DOI | MR

[17] Golovkina A., Kozynchenko V., Kulabukhova N., “Reconstruction and identification of dynamical systems based on taylor maps”, Computational Science and its Applications, 21$^{\rm st}$ International Conference, Proceedings, v. VIII, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Springer Nature, Switzerland, 2021, 360–369 | DOI

[18] Golovkina A. G., Kozynchenko V. A., Kulabukhova N. V., “Reconstruction of ordinary differential equations from irregularly distributed time-series data”, Proceedings of the 9$^{\rm th}$ International Conference “Distributed Computing and Grid Technologies in Science and Education”, GRID'2021 (Dubna, Russia, July 5–9, 2021), CEUR Workshop Proceedings, 3041, 342–347

[19] Golovkina A. G., Ganaeva D. D., “Method of nonlinear dynamical systems reconstruction based on time series data”, Control Processes and Stability, 9:1 (2022), 197–201 (In Russian)

[20] Klimenko I. S., “Implementation of matrix map method for solving a system of ordinary equations”, Control Processes and Stability, 9:1 (2022), 53–57 (In Russian)

[21] Cartwright M. L., “Van der Pol's equation for relaxation oscillations”, Contributions to the theory of nonlinear oscillations. II, Princeton Ann. Math. Stud., Princeton University Press, Princeton, 1952, 3–18 | MR

[22] Abrevaya G., Rish I., Aravkin A. Y., Cecchi G., Kozloski J., Polosecki P., Zheng P., Dawson S. R., Rhee J., Cox D., Learning nonlinear brain dynamics: Van der Pol Meets LSTM, bioRxiv 330548 | DOI

[23] Ordinary differential equation solver for stiff or non-stiff system, LSODA, September 2005 (accessed: August 10, 2022) http://www.nea.fr/abs/html/uscd1227.html