Geodesic
Discounted Markov control processes induced by deterministic systems
Kybernetika, Tome 42 (2006) no. 6, pp. 647-664 Cet article a éte moissonné depuis la source Czech Digital Mathematics Library

Voir la notice de l'article

This paper deals with Markov Control Processes (MCPs) on Euclidean spaces with an infinite horizon and a discounted total cost. Firstly, MCPs which result from the deterministic controlled systems will be analyzed. For such MCPs, conditions that permit to establish the equation known in the literature of Economy as Euler’s Equation (EE) will be given. There will be also presented an example of a Markov Control Process with deterministic controlled system where, to obtain the optimal value function, EE applied to the value iteration algorithm will be used. Secondly, the MCPs which result from the perturbation of deterministic controlled systems with a random noise will be dealt with. There will be also provided the conditions which allow to obtain the optimal value function and the optimal policy of a perturbed controlled system, in terms of the optimal value function and the optimal policy of deterministic controlled system corresponding. Finally, several examples to illustrate the last case mentioned will be presented.
This paper deals with Markov Control Processes (MCPs) on Euclidean spaces with an infinite horizon and a discounted total cost. Firstly, MCPs which result from the deterministic controlled systems will be analyzed. For such MCPs, conditions that permit to establish the equation known in the literature of Economy as Euler’s Equation (EE) will be given. There will be also presented an example of a Markov Control Process with deterministic controlled system where, to obtain the optimal value function, EE applied to the value iteration algorithm will be used. Secondly, the MCPs which result from the perturbation of deterministic controlled systems with a random noise will be dealt with. There will be also provided the conditions which allow to obtain the optimal value function and the optimal policy of a perturbed controlled system, in terms of the optimal value function and the optimal policy of deterministic controlled system corresponding. Finally, several examples to illustrate the last case mentioned will be presented.
Classification : 90C40, 93E20
Keywords: discounted Markov control process; deterministic control system; Euler equation; deterministic control system perturbed by a random noise 
@article{KYB_2006_42_6_a1,
     author = {Cruz-Su\'arez, Hugo and Montes-de-Oca, Ra\'ul},
     title = {Discounted {Markov} control processes induced by deterministic systems},
     journal = {Kybernetika},
     pages = {647--664},
     year = {2006},
     volume = {42},
     number = {6},
     mrnumber = {2296506},
     zbl = {1249.90312},
     language = {en},
     url = {http://geodesic.mathdoc.fr/item/KYB_2006_42_6_a1/}
}
TY  - JOUR
AU  - Cruz-Suárez, Hugo
AU  - Montes-de-Oca, Raúl
TI  - Discounted Markov control processes induced by deterministic systems
JO  - Kybernetika
PY  - 2006
SP  - 647
EP  - 664
VL  - 42
IS  - 6
UR  - http://geodesic.mathdoc.fr/item/KYB_2006_42_6_a1/
LA  - en
ID  - KYB_2006_42_6_a1
ER  - 
%0 Journal Article
%A Cruz-Suárez, Hugo
%A Montes-de-Oca, Raúl
%T Discounted Markov control processes induced by deterministic systems
%J Kybernetika
%D 2006
%P 647-664
%V 42
%N 6
%U http://geodesic.mathdoc.fr/item/KYB_2006_42_6_a1/
%G en
%F KYB_2006_42_6_a1
Cruz-Suárez, Hugo; Montes-de-Oca, Raúl. Discounted Markov control processes induced by deterministic systems. Kybernetika, Tome 42 (2006) no. 6, pp. 647-664. http://geodesic.mathdoc.fr/item/KYB_2006_42_6_a1/

[1] Benveniste L. M., Scheinkman J. A.: On the differentiability of the value function in dynamic models of economics. Econometrica 47 (1979), 727–732 | DOI | MR | Zbl

[2] Bertsekas D. P.: Dynamic Programming: Deterministic and Stochastic Models. Prentice-Hall, Englewood Cliffs, New Jersey 1987 | MR | Zbl

[3] Cruz-Suárez D., Montes-de-Oca, R., Salem-Silva F.: Conditions for the uniqueness of optimal policies of discounted Markov decision processes. Math. Methods Oper. Res. 60 (2004), 415–436 | DOI | MR | Zbl

[4] Fuente A. De la: Mathematical Methods and Models for Economists. Cambridge University Press, New York 2000 | MR | Zbl

[5] Duffie D.: Security Markets. Academic Press, Boston 1988 | MR | Zbl

[6] Durán J.: On dynamic programming with unbounded returns. J. Econom. Theory 15 (2000), 339–352 | DOI | MR | Zbl

[7] Heer B., Maußner A.: Dynamic General Equilibrium Modelling: Computational Method and Application. Springer-Verlag, Berlin 2005 | MR

[8] Hernández-Lerma O.: Adaptive Markov Control Processes. Springer-Verlag, New York 1989 | MR

[9] Hernández-Lerma O., Lasserre J. B.: Discrete-Time Markov Control Processes: Basic Optimality Criteria. Springer-Verlag, New York 1996 | MR | Zbl

[10] Van C. Le, Morhaim L.: Optimal growth models with bounded or unbounded returns: a unifying approach. J. Econom. Theory 105 (2002), 158–187 | DOI | MR | Zbl

[11] Levhari D., Srinivasan T. N.: Optimal savings under uncertainty. Rev. Econom. Stud. 36 (1969), 153–164 | DOI

[12] Mirman L. J.: Dynamic models of fishing: a heuristic approach. In: Control Theory in Mathematical Economics (Pan-Tai Liu and J. G. Sutinen, eds.), Marcel Dekker, New York 1979, pp. 39–73 | Zbl

[13] Rincón-Zapatero J. L., Rodríguez-Palmero C.: Existence and uniqueness of solutions to the Bellman equation in the unbounded case. Econometrica 71 (2003), 1519–1555 | DOI | MR | Zbl

[14] Santos M. S.: Numerical solution of dynamic economic models. In: Handbook of Macroeconomic, Volume I (J. B. Taylor and M. Woodford, eds.), North Holland, Amsterdam 1999, pp. 311–386

[15] Stokey N. L., Lucas R. E.: Recursive Methods in Economic Dynamics. Harvard University Press, Cambridge, Mass. 1989 | MR | Zbl