Solving a class of Hamilton-Jacobi-Bellman equations using pseudospectral methods

Mehrali-Varjani, Mohsen   ; Shamsi, Mostafa   ; Malek, Alaeddin

Kybernetika, Tome 54 (2018) no. 4, pp. 629-647

Abstract (VO)

This paper presents a numerical approach to solve the Hamilton-Jacobi-Bellman (HJB) problem which appears in feedback solution of the optimal control problems. In this method, first, by using Chebyshev pseudospectral spatial discretization, the HJB problem is converted to a system of ordinary differential equations with terminal conditions. Second, the time-marching Runge-Kutta method is used to solve the corresponding system of differential equations. Then, an approximate solution for the HJB problem is computed. In addition, to get more efficient and accurate method, the domain decomposition strategy is proposed with the pseudospectral spatial discretization. Five numerical examples are presented to demonstrate the efficiency and accuracy of the proposed hybrid method.

Détail

DOI : 10.14736/kyb-2018-4-0629

Classification : 35F21, 49J20, 65M70
Keywords: nonlinear optimal control; pseudospectral method; Hamilton–Jacobi–Bellman equation

Bibliographie

[1] Baltensperger, R., Trummer, M. R.: Spectral differencing with a twist. SIAM J. Sci. Comput. 24 (2003), 1465-1487. | DOI | MR

[2] Beard, R., Saridis, G., Wen, J.: Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation. Automatica 33 (1997), 2159-2177. | DOI | MR

[3] Ben-Asher, J. Z.: Optimal Control Theory with Aerospace Applications. American Institute of Aeronautics and Astronautics, Reston 2010. | DOI

[4] Boyd, J. P.: Chebyshev and Fourier Spectral Methods. Second revised edition. Dover Publications, New York 2001. | MR

[5] Boyd, J. P., Petschek, R.: The relationships between Chebyshev, Legendre and Jacobi polynomials: the generic superiority of Chebyshev polynomials and three important exceptions. J. Scientific Comput. 59 (2014), 1-27. | DOI | MR

[6] Canuto, C., Hussaini, M. Y., Quarteroni, A., Zang, T. A.: Spectral Methods in Fluid Dynamics. Springer-Verlag, Berlin 1987. | DOI | MR

[7] Canuto, C., Hussaini, M. Y., Quarteroni, A., Zang, T. A.: Spectral Methods: Fundamentals in Single Domains. Springer-Verlag, Berlin 2006. | MR

[8] Cristiani, E., Martinon, P.: Initialization of the shooting method via the Hamilton-Jacobi-Bellman approach. J. Optim. Theory Appl. 146 (2010), 321-346. | DOI | MR

[9] Dai, R., Jr, J. Cochran: Wavelet collocation method for optimal control problems. J. Optim. Theory Appl. 143 (2009), 265–278. | DOI | MR

[10] Elnagar, G., Kazemi, M. A., Razzaghi, M.: The pseudospectral Legendre method for discretizing optimal control problems. IEEE Trans. Automat. Control 40 (1995), 1793-1796. | DOI | MR

[11] Fahroo, F., Ross, I. M.: Direct trajectory optimization by a Chebyshev pseudospectral method. J. Guid. Control Dynam. 25 (2002), 160-166. | DOI

[12] Foroozandeh, Z., Shamsi, M., Azhmyakov, V., Shafiee, M.: A modified pseudospectral method for solving trajectory optimization problems with singular arc. Math. Methods Appl. Sci. 40 (2017), 1783-1793. | DOI | MR

[13] Funaro, D.: Polynomial Approximation of Differential Equations. Springer-Verlag, Berlin 1992. | DOI | MR

[14] Hanert, E., Piret, C.: A Chebyshev pseudospectral method to solve the space-time tempered fractional diffusion equation. SIAM J. Scientif. Comput. 36 (2014), A1797-A1812. | DOI | MR

[15] Huang, J., Lin, C. F.: Numerical approach to computing nonlinear $ H_\infty $ control laws. J. Guid. Control Dynam. 18 (1995), 989-994. | DOI

[16] Huang, C. S., Wang, S., Chen, C. S., Li, Z. C.: A radial basis collocation method for Hamilton-Jacobi-Bellman equations. Automatica 42 (2006), 2201-2207. | DOI | MR

[17] Kang, W., Bedrossian, N.: Pseudospectral optimal control theory makes debut flight, Saves {NASA} 1m in Under Three Hours. SIAM News 40 (2007).

[18] Kang, W., Gong, Q., Ross, I. M., Fahroo, F.: On the convergence of nonlinear optimal control using pseudospectral methods for feedback linearizable systems. Int. J. Robust Nonlin. 17 (2007), 1251-1277. | DOI | MR

[19] Kirk, D. E.: Optimal Control Therory: An Introduction. Prentice-Hall, New Jersey 1970.

[20] Kleinman, D.: On an iterative technique for Riccati equation computations. IEEE Trans. Automat. Control 13 (1968), 114-115. | DOI

[21] Lancaster, P., Rodman, L.: Algebraic Riccati Equations. Clarendon, Wotton-under-Edge 1995. | MR

[22] Lewis, F. L., Syrmos, V. L.: Optimal Control. John Wiley, New York 1995. | MR

[23] Liberzon, D.: Calculus of Variations and Optimal Control Theory. Princeton University Press 2012. | MR

[24] Nagy, Z. K., Braatz, R D.: Open-loop and closed-loop robust optimal control of batch processes using distributional and worst-case analysis. J. Process Control. 14 (2004), 411-422. | DOI

[25] Nik, H. S., Shateyi, S.: Application of optimal HAM for finding feedback control of optimal control problems. Math. Probl. Eng. 2013 (2013), 1-10. | DOI | MR

[26] Orszag, S. A.: Comparison of pseudospectral and spectral approximation. Stud. Appl. Math. 51 (1972), 253-259. | DOI

[27] Parand, K., Rezaei, A. R., Ghaderi, S. M.: A modified pseudospectral scheme for accurate solution of Bang-Bang optimal control problems. Comm. Nonlinear Sci. Numer. Simul. 16 (2011), 274-283. | DOI | MR

[28] Rakhshan, S. A., Effati, S., Kamyad, A. Vahidian: Solving a class of fractional optimal control problems by the Hamilton-Jacobi-Bellman equation. J. Vib. Control 1 (2016), 1-16. | MR

[29] Reisinger, C., Forsyth, P. A.: Piecewise constant policy approximations to Hamilton-Jacobi-Bellman equations. Appl. Numer. Math. 103 (2016), 27-47. | DOI | MR

[30] Ross, I. M., Fahroo, F.: Pseudospectral knotting methods for solving nonsmooth optimal control problems. J. Guid. Control Dynam. 27 (2004), 397-405. | DOI

[31] Sabeh, Z., Shamsi, M., Dehghan, M.: Distributed optimal control of the viscous Burgers equation via a Legendre pseudo-spectral approach.

[32] Taher, A. H. Saleh, Malek, A., Momeni-Masuleh, S. H.: Chebyshev differentiation matrices for efficient computation of the eigenvalues of fourth-order Sturm-Liouville problems. Appl. Math. Model. 37 (2013), 4634-4642. | DOI | MR

[33] Schafer, R. D.: An Introduction to Nonassociative Algebras. Stillwater, Oklahoma 1969.

[34] Shamsi, M.: A modified pseudospectral scheme for accurate solution of Bang-Bang optimal control problems. Optimal Control Appl. Methods 32 (2010), 668-680. | DOI | MR

[35] Shamsi, M., Dehghan, M.: Determination of a control function in three-dimensional parabolic equations by Legendre pseudospectral method. Numer. Methods Partial Differential Equations 28 (2012), 74-93. | DOI | MR

[36] Swaidan, W., Hussin, A.: Feedback control method using Haar wavelet operational matrices for solving optimal control problems. Abs. Appl. Anal. 2013 (2013), 1-8. | DOI | MR

[37] Trefethen, L. N.: Spectral Methods in Matlab. SIAM, Philadelphia 2000. | DOI | MR

[38] Vlassenbroeck, J., Doreen, R. Van: A Chebyshev technique for solving nonlinear optimal control problems. IEEE Trans. Automat. Control 33 (1988), 333-340. | DOI | MR

[39] Wang, S., Gao, F., Teo, K. L.: An upwind finite-difference method for the approximation of viscosity solutions to Hamilton-Jacobi-Bellman equations. IMA J. Math. Control I. 17 (2000), 167-178. | DOI | MR

[40] Yan, Zh., Wang, J.: Model predictive control of nonlinear systems with unmodeled dynamics based on feedforward and recurrent neural networks. IEEE Trans. Ind. Informat. 8 (2012), 746-756. | DOI

[41] Yershov, D. S., Frazzoli, E.: Asymptotically optimal feedback planning using a numerical Hamilton-Jacobi-Bellman solver and an adaptive mesh refinement. Int. J. Robot. Res. 35 (2016), 565-584. | DOI

[42] Yong, J., Zhou, X. Y.: Stochastic Controls: Hamiltonian Systems and HJB Equations. Springer-Verlag, New York 1999. | MR