On a stochastic optimality of the feedback control in the

T. A. Belkina   ; Yu. M. Kabanov   ; E. L. Presman

Teoriâ veroâtnostej i ee primeneniâ, Tome 48 (2003) no. 4, pp. 661-675

Résumé

We show that the optimal feedback control $\widehat u$ in the standard nonhomogeneous LQG-problem with infinite horizon has the following property. There is a constant $b_*$ such that, whatever $b> b_*$ is, the deficiency process of optimal control with respect to any possible control $u$, i.e., the difference $J_T(\widehat u\hspace*{0.2pt})- J_T(u)$ between the optimal cost process $J_T(\widehat u\hspace*{0.2pt})$ and the cost process corresponding to control $u$, is majorated at infinity by a deterministic function $b\log T$. In other words, $b\log T$ is an upper function for any deficiency process. This result, combined with an example of an LQG-regulator where, for certain $b>0$, the function $b\log T$ is not an upper function for certain deficiency processes, gives an answer to the long-standing open problem about the best possible rate function for sensitive probabilistic criteria. Our setting covers the optimal tracking problem.

Détail

Keywords: linear-quadratic regulator, optimality almost surely, observability, controllability, martingale law of large numbers, upper functions, Ornstein–Uhlenbeck process.
Mots-clés : Riccati equation

Bibliographie

[1] Asriev A. V., Rotar' V. I., “On asymptotic optimality in probability and almost surely in dynamic control”, Stochastics Stochastics Rep., 33 (1990), 1–16 | MR | Zbl

[2] Belkina T. A., Presman E. L., “Asimptoticheski optimalnye po raspredeleniyu upravleniya dlya lineinoi stokhasticheskoi sistemy s kvadratichnym funktsionalom”, Avtomatika i telemekhanika, 58:3 (1997), 106–115 | MR | Zbl

[3] Belkina T. A., Rotar V. I., “Ob usloviyakh asimptoticheskoi optimalnosti po veroyatnosti i pochti navernoe v modeli upravlyaemogo diffuzionnogo protsessa”, Avtomatika i telemekhanika, 60:2 (1999), 46–56 | MR | Zbl

[4] Bensoussan A., Stochastic Control of Partially Observable Systems, Cambridge University Press, Cambridge, 1992 | MR | Zbl

[5] Borkar V. S., Optimal Control of Diffusion Processes, Pitman Research Notes in Mathematics Series, 203, Longman, Harlow, 1989 | MR | Zbl

[6] Carlson D. A., Haurie A. B., Leizarowitz A., Infinite Horizon Optimal Control. Deterministic and Stochastic Systems, Springer-Verlag, Berlin, 1991

[7] Dai Pra P., Di Masi G. B., Trivellato B., “Almost sure optimality and optimality in probability for stochastic control problems over an infinite time horizon”, Ann. Oper. Res., 88 (1999), 161–171 | DOI | MR | Zbl

[8] Devis M. Kh. A., Lineinoe otsenivanie i stokhasticheskoe upravlenie, Nauka, M., 1984, 202 pp. | MR

[9] Di Masi G. B., Kabanov Yu., “On sensitive probabilistic criteria in the linear regulator problem with the infinite horizon”, Obozrenie prikladnoi i promyshlennoi matematiki, 5:2 (1998), 410–422 | Zbl

[10] Föllmer H., Majumdar M., “On the asymptotic behavior of stochastic economic processes. Two examples from intertemporal allocation under uncertainty”, J. Math. Econom., 5 (1978), 275–287 | DOI | MR | Zbl

[11] Konyukhova (Belkina) T. A., “Asimptoticheski optimalnye po veroyatnosti upravleniya v zadache o lineinom regulyatore s peremennymi parametrami”, Avtomatika i telemekhanika, 55:2 (1994), 110–120 | MR | Zbl

[12] Konyukhova (Belkina) T. A., Rotar V. I., “Upravleniya, asimptoticheski optimalnye po veroyatnosti i pochti navernoe v zadache o lineinom regulyatore”, Avtomatika i telemekhanika, 53:6 (1992), 65–78 | MR | Zbl

[13] Kwakernaak H., Sivan R, Linear Optimal Control Systems, Wiley-Interscience, New York, London, 1972 | MR | Zbl

[14] Leizarowitz A., “Infinite horizon stochastic regulation and tracking with the overtaking criteria”, Stochastics, 22 (1987), 117–150 | MR | Zbl

[15] Leizarowitz A., “On almost sure optimization for stochastic controls systems”, Stochastics, 23 (1988), 85–107 | MR | Zbl

[16] Lippman S., “Maximal average-reward policies for semi-Markov decision processes with arbitrary state and action space”, Ann. Math. Statist., 42 (1971), 1717–1726 | DOI | MR | Zbl

[17] Mandl P., “Asymptotic ordering of probability distributions for linear controlled systems with quadratic cost”, Stochastic Differential Systems, Lecture Notes in Control and Inform. Sci., eds. N. Christopeit, K. Helmes, M. Kohlmann, Springer-Verlag, Berlin, New York, 1986, 277–283 | MR

[18] Mandl P., Romera Ayllón M. R., “On controlled Markov processes with average cost criterion”, Kybernetika (Prague), 23 (1987), 433–442 | MR | Zbl

[19] Nagaev S. V., Presman E. L., “Zakon povtornogo logarifma v odnoi zadache upravleniya”, Teoriya veroyatn. i ee primen., 43:2 (1998), 364–369 | MR | Zbl

[20] Presman E. L., “Optimalnost pochti navernoe i po veroyatnosti dlya stokhasticheskogo lineino-kvadraticheskogo regulyatora”, Teoriya veroyatn. i ee primen., 42:3 (1997), 627–632 | MR | Zbl

[21] Presman E., Rotar' V., Taksar M., “Optimality in probability and almost surely. The general scheme and a linear regulator problem”, Stochastics Stochastics Rep., 43 (1993), 127–137 | MR | Zbl

[22] Revuz D., Yor M., Continuous Martingales and Brownian Motion, Springer-Verlag, Berlin, 1991 | MR | Zbl

[23] Rotar' V. I., “The recurrency property and optimality almost surely and in probability”, New Trends in Probability and Statistics, Proceedings of Bakuriany Colloquium in Honor of Yu. V. Prohorov, v. 1, VSP, Utrecht, 1990