Neural network optimal control for nonlinear system based on zero-sum differential game

Xingjian, Fu ; Zizheng, Li

Kybernetika, Tome 57 (2021) no. 3, pp. 546-566

Abstract (VO)

In this paper, for a class of the complex nonlinear system control problems, based on the two-person zero-sum game theory, combined with the idea of approximate dynamic programming(ADP), the constrained optimization control problem is solved for the nonlinear systems with unknown system functions and unknown time-varying disturbances. In order to obtain the approximate optimal solution of the zero-sum game, the multilayer neural network is used to fit the evaluation network, the execution network and the disturbance network of ADP respectively. The Lyapunov stability theory is used to prove the uniform convergence, and the system control output converges to the neighborhood of the target reference value. Finally, the simulation example verifies the effectiveness of the algorithm.

Détail

DOI : 10.14736/kyb-2021-3-0546

Classification : 91A80, 93C10, 93D21
Keywords: zero-sum game; nonlinear system; neural network; approximate dynamic programming

Bibliographie

[1] L'Afflitto, A.: Differential games, continuous Lyapunov functions, and stabilization of non-linear dynamical systems. IET Control Theory Appl. 11 (2017), 2486-2496. | DOI

[2] Bellman, R. E.: Dynamic Programming. Princeton University Press, Princeton 1957. | Zbl

[3] Bian, T., Jiang, Y., Jiang, Z. P.: Adaptive dynamic programming and optimal control of nonlinear nonaffine systems. Automatica 50 (2014), 2624-2632. | DOI

[4] Chai, Y., Luo, J.-J., Han, N., Xie, J.-F.: Attitude takeover control of failed spacecraft using SDRE based differential game approach. J. Astronaut. 41 (2020), 191-198.

[5] El-Sousy, F. F. M., Abuhasel, K. A.: Nonlinear robust optimal control via adaptive dynamic programming of permanent-magnet linear synchronous motor drive for uncertain two-axis motion control system. IEEE Trans. Industry Appl. 56 (2020), 1940-1952. | DOI

[6] Federico, S., Tacconi, E.: Dynamic programming for optimal control problems with delays in the control variable. SIAM J. Control Optim. 52 (2014), 1203-1236. | DOI

[7] Garcia, Y. H., Gonzalez-Hernandez, J.: Discrete-time Markov control processes with recursive discount rates. Kybernetika 52 (2016), 403-426.

[8] Gromov, D., Gromova, E.: On a class of hybrid differential games. Dynamic Games Appl. 7 (2017), 266-288. | DOI

[9] Hua, W., Meng, Q., Zhang, J.: Differential game guidance law for dual and bounded controlled missiles. J. Bejing Univ. Aeronaut. Astronaut. 42 (2016), 1851-1856.

[10] Isaacs, R.: Differential Games. John Wiley and Sons, New York 1965.

[11] Krasnosielska-Kobos, A.: Construction of Nash equilibrium based on multiple stopping problem in multi-person game. Math. Methods Oper. Res. 83 (2016), 53-70. | DOI

[12] Lei, L., Yan-Jun, L., Aiqing, Ch., Shaocheng, T., Chen, C. L. P.: Integral barrier Lyapunov function based adaptive control for switched nonlinear systems. Science China Inform. Sci. 63 (2020), 132203. | DOI

[13] Lei, L., Yan-Jun, J., Dapeng, L., Shaocheng, T., Zhanshan, W.: Barrier Lyapunov function based adaptive fuzzy FTC for switched systems and its applications to resistance inductance capacitance circuit system. IEEE Trans. Cybernet. 50 (2020), 3491-3502. | DOI

[14] Li, J.-M., Zhu, H.-N.: Nash differential games for delayed nonlinear stochastic systems with state-and control-dependent noise. J. Guangdong Univ. Technol. 35 (2018), 41-45.

[15] Liu, D. R., Li, H. L., Wang, D.: Neural-network-based zero-sum game for discrete time nonlinear systems via iterative adaptive dynamic programming algorithm. Neurocomputing 110 (2013), 92-100. | DOI

[16] Majid, M., Sani, Seyed Kamal Hosseini: A Novel distributed optimal adaptive control algorithm for nonlinear multi-agent differential graphical games. IEEE/CAA J. Automat. Sinica 5 (2018), 331-341. | DOI

[17] Marzieh, M., Karimi, B., Mahootchi, M.: A differential Stackelberg game for pricing on a freight transportation network with one dominant shipper and multiple oligopolistic carriers. Scientia Iranica 23 (2016), 2391-2406. | DOI

[18] Moon, J.: Necessary and sufficient conditions of risk - sensitive optimal control and differential games for stochastic differential delayed equations. Int. J. Robust Nonlinear Control 29 (2019), 4812-4827. | DOI

[19] Mu, C., Wang, K.: Approximate-optimal control algorithm for constrained zero-sum differential games through event-triggering mechanism. Nonlinear Dynamics 95 (2019), 2639-2657. | DOI

[20] Nash, J. F.: Equilibrium points in n-person games. Proc. Nat. Acad. Sci. USA 36 (1950), 1, 48-49. | DOI

[21] Nash, J.: Non-cooperative games. Ann. Math. 54 (1951), 286-295. | DOI

[22] Neumann, J. von, Morgenstern, O.: Theory of Games and Economic Behavior. Princeton University Press, Princeton 1944.

[23] Pham, H., Wei, X.: Dynamic programming for optimal control of stochastic McKean-Vlasov dynamics. SIAM J. Control Optim. 55 (2016), 1069-1101. | DOI

[24] Qin, Ch.: Research on Optimal Control Based on Approximate Dynamic Programming Application in Power System. Doctoral dissertation, Northeastern University 2014.

[25] Rui-Feng, C., Wei-Dong, L., Li, E. G.: Differential game guidance of underwater nonlinear tracking control based on continuous time generalized predictive correction. Acta Physica Sinica 67 (2018), 050501. | DOI

[26] Song, R., Li, J., Lewis, F. L.: Robust optimal control for disturbed nonlinear zero-sum differential games based on single NN and least qquares. IEEE Trans. Systems Man Cybernet. Systems PP99 (2019), 4009-4019. | DOI

[27] Wei, Q., Liu, D., Lin, H.: Value iteration adaptive dynamic programming for optimal control of discrete-time nonlinear systems. IEEE Trans Cybernet. 46 (2016), 840-853. | DOI

[28] Werbos, P. J.: Advances forecasting methods for global crisis warning and models of intelligence. General Systems Yearbook 22 (1977), 25-38.

[29] Yang, B., Xuesong, C. L.: Heuristic dynamic programming based optimal control for multiple time delay systems. J. Theoret. Appl. Inform. Technol. 48 (2013), 876-881.

[30] Zhang, H., Qin, Ch., Jiang, B., Luo, Y.: Online adaptive policy learning algorithm for Hinf state feedback control of unknown affine nonlinear discrete-time systems. IEEE Trans. Cybernet. 44 (2014), 2706-2718. | DOI

[31] Zhang, F., Shan, G. S., Gao, H.: Rheumatoid arthritis analysis by Nash equilibrium game analysis. J. Medical Imaging Health Inform. 9 (2019), 1382-1385. | DOI

[32] Zhu, Q., Liu, Y., Wen, G.: Adaptive neural network output feedback control for stochastic nonlinear systems with full state constraints. ISA Trans. 101 (2020), 60-68. | DOI

[33] Zhu, Q., Wang, K., Shao, Z.: Distributed dynamic optimization for chemical process networks based on differential games. Industr. Engrg. Chem. Res. 59 (2020), 2441-2456. | DOI