Geodesic
Approximation and estimation in Markov control processes under a discounted criterion
Kybernetika, Tome 40 (2004) no. 6, pp. 681-690 Cet article a éte moissonné depuis la source Czech Digital Mathematics Library

Voir la notice de l'article

We consider a class of discrete-time Markov control processes with Borel state and action spaces, and $\Re ^{k}$-valued i.i.d. disturbances with unknown density $\rho .$ Supposing possibly unbounded costs, we combine suitable density estimation methods of $\rho $ with approximation procedures of the optimal cost function, to show the existence of a sequence $\lbrace \hat{f}_{t}\rbrace $ of minimizers converging to an optimal stationary policy $f_{\infty }.$
We consider a class of discrete-time Markov control processes with Borel state and action spaces, and $\Re ^{k}$-valued i.i.d. disturbances with unknown density $\rho .$ Supposing possibly unbounded costs, we combine suitable density estimation methods of $\rho $ with approximation procedures of the optimal cost function, to show the existence of a sequence $\lbrace \hat{f}_{t}\rbrace $ of minimizers converging to an optimal stationary policy $f_{\infty }.$
Classification : 90B05, 90B30, 90C40, 93E10
Keywords: Markov control processes; density estimation; discounted cost criterion 
@article{KYB_2004_40_6_a2,
     author = {Minj\'arez-Sosa, J. Adolfo},
     title = {Approximation and estimation in {Markov} control processes under a discounted criterion},
     journal = {Kybernetika},
     pages = {681--690},
     year = {2004},
     volume = {40},
     number = {6},
     mrnumber = {2120390},
     zbl = {1249.93163},
     language = {en},
     url = {http://geodesic.mathdoc.fr/item/KYB_2004_40_6_a2/}
}
TY  - JOUR
AU  - Minjárez-Sosa, J. Adolfo
TI  - Approximation and estimation in Markov control processes under a discounted criterion
JO  - Kybernetika
PY  - 2004
SP  - 681
EP  - 690
VL  - 40
IS  - 6
UR  - http://geodesic.mathdoc.fr/item/KYB_2004_40_6_a2/
LA  - en
ID  - KYB_2004_40_6_a2
ER  - 
%0 Journal Article
%A Minjárez-Sosa, J. Adolfo
%T Approximation and estimation in Markov control processes under a discounted criterion
%J Kybernetika
%D 2004
%P 681-690
%V 40
%N 6
%U http://geodesic.mathdoc.fr/item/KYB_2004_40_6_a2/
%G en
%F KYB_2004_40_6_a2
Minjárez-Sosa, J. Adolfo. Approximation and estimation in Markov control processes under a discounted criterion. Kybernetika, Tome 40 (2004) no. 6, pp. 681-690. http://geodesic.mathdoc.fr/item/KYB_2004_40_6_a2/

[1] Cavazos-Cadena R.: Nonparametric adaptive control of discounted stochastic systems with compact state space. J. Optim. Theory Appl. 65 (1990), 191–207 | DOI | MR | Zbl

[2] Devroye L., Gyorfi L.: Nonparametric Density Estimation the $L_{1}$ View. Wiley, New York 1985 | MR

[3] Dynkin E. B., Yushkevich A. A.: Controlled Markov Processes. Springer–Verlag, New York 1979 | MR

[4] Gordienko E. I.: Adaptive strategies for certain classes of controlled Markov processes. Theory Probab. Appl. 29 (1985), 504–518 | Zbl

[5] Gordienko E. I., Minjárez-Sosa J. A.: Adaptive control for discrete-time Markov processes with unbounded costs: discounted criterion. Kybernetika 34 (1998), 217–234 | MR

[6] Hasminskii R., Ibragimov I.: On density estimation in the view of Kolmogorov’s ideas in approximation theory. Ann. Statist. 18 (1990), 999–1010 | DOI | MR | Zbl

[7] Hernández-Lerma O.: Adaptive Markov Control Processes. Springer–Verlag, New York 1989 | MR

[8] Hernández-Lerma O., Cavazos-Cadena R.: Density estimation and adaptive control of Markov processes: average and discounted criteria. Acta Appl. Math. 20 (1990), 285–307 | DOI | MR

[9] Hernández-Lerma O., Lasserre J. B.: Discrete-Time Markov Control Processes: Basic Optimality Criteria. Springer–Verlag, New York 1996 | MR | Zbl

[10] Hernández-Lerma O., Lasserre J. B.: Further Topics on Discrete-Time Markov Control Processes. Springer–Verlag, New York 1999 | MR | Zbl

[11] Hernández-Lerma O., Marcus S. I.: Adaptive policies for discrete-time stochastic control systems with unknown disturbance distribution. Systems Control Lett. 9 (1987), 307–315 | DOI | MR | Zbl

[12] Hilgert N., Minjárez-Sosa J. A.: Adaptive policies for time-varying stochastic systems under discounted criterion. Math. Methods Oper. Res. 54 (2001), 491–505 | DOI | MR | Zbl

[13] Schäl M.: Conditions for optimality and for the limit of $n$-stage optimal policies to be optimal. Z. Wahrs. Verw. Gerb. 32 (1975), 179–196 | DOI | MR