Markov stopping games with an absorbing state and total reward criterion

Cavazos-Cadena, Rolando ; Rodríguez-Gutiérrez, Luis ; Sánchez-Guillermo, Dulce María

Kybernetika, Tome 57 (2021) no. 3, pp. 474-492

Abstract (VO)

This work is concerned with discrete-time zero-sum games with Markov transitions on a denumerable space. At each decision time player II can stop the system paying a terminal reward to player I, or can let the system to continue its evolution. If the system is not halted, player I selects an action which affects the transitions and receives a running reward from player II. Assuming the existence of an absorbing state which is accessible from any other state, the performance of a pair of decision strategies is measured by the total expected reward criterion. In this context it is shown that the value function of the game is characterized by an equilibrium equation, and the existence of a Nash equilibrium is established.

Détail

DOI : 10.14736/kyb-2021-3-0474

Classification : 91A10, 91A15
Keywords: non-expansive operator; monotonicity property; fixed point; equilibrium equation; hitting time; bounded rewards

Bibliographie

[1] Altman, E., Shwartz, A.: Constrained Markov Games: Nash Equilibria. In: Annals of Dynamic Games (V. Gaitsgory, J. Filar and K. Mizukami, eds. 6, Birkhauser, Boston 2000, pp. 213-221. | DOI | Zbl

[2] Atar, R., Budhiraja, A.: A stochastic differential game for the inhomogeneous $\infty$-Laplace equation. Ann. Probab.2 (2010), 498-531. | DOI

[3] Bäuerle, N., Rieder, U.: Zero-sum risk-sensitive stochastic games. Stoch. Proc. Appl. 127 (2017), 622-642. | DOI

[4] Bielecki, T., Hernández-Hernández, D., Pliska, S. R.: Risk sensitive control of finite state Markov chains in discrete time, with applications to portfolio management. Math. Methods Oper. Res. 50 (1999), 167-188. | DOI | Zbl

[5] Cavazos-Cadena, R., Hernández-Hernández, D.: Nash equilibria in a class of Markov stopping games. Kybernetika 48 (2012), 1027-1044.

[6] Filar, J. A., Vrieze, O. J.: Competitive Markov Decision Processes. Springer, Berlin 1996.

[7] Hernández-Lerma, O.: Adaptive Markov Control Processes. Springer, New York 1989. | Zbl

[8] Hernández-Lerma, O., Lasserre, J. B.: Discrete-Time Markov Control Processes: Basic Optimality Criteria. Springer, New York 1996. | Zbl

[9] Kolokoltsov, V. N., Malafeyev, O. A.: Understanding Game Theory. World Scientific, Singapore 2010. | Zbl

[10] Martínez-Cortés, V. M.: Bipersonal stochastic transient Markov games with stopping times and total reward criteria. Kybernetika 57 (2021), 1-14. | DOI

[11] Peskir, G.: On the American option problem. Math. Finance 15 (2005), 169-181. | DOI | Zbl

[12] Peskir, G., Shiryaev, A.: Optimal Stopping and Free-Boundary Problems. Birkhau\-ser, Boston 2010. | Zbl

[13] Piunovskiy, A. B.: Examples in Markov Decision Processes. Imperial College Press, London 2013.

[14] Puterman, M.: Markov Decision Processes. Wiley, New York 1994. | Zbl

[15] Shapley, L. S.: Stochastic games. Proc. Natl. Acad. Sci. USA 39 (1953), 1095-1100. | DOI | Zbl

[16] Shiryaev, A.: Optimal Stopping Rules. Springer, New York 1978. | Zbl

[17] Sladký, K.: Ramsey growth model under uncertainty. In: Proc. 27th International Conference Mathematical Methods in Economics (H. Brozová, ed.), Kostelec nad Černými lesy 2009, pp. 296-300.

[18] Sladký, K.: Risk-sensitive Ramsey growth model. In: Proc. 28th International Conference on Mathematical Methods in Economics (M. Houda and J. Friebelová, eds.) České Budějovice 2010, pp. 560-565.

[19] Sladký, K.: Risk-sensitive average optimality in Markov decision processes. Kybernetika 54 (2018), 1218-1230. | DOI

[20] White, D. J.: Real applications of Markov decision processes. Interfaces 15 (1985), 73-83. | DOI

[21] White, D. J.: Further real applications of Markov decision processes. Interfaces 18 (1988), 55-61. | DOI

[22] White, D. J.: A survey of applications of Markov decision processes. J. Opl. Res. Soc. 44 (1993), 1073-1096. | DOI

[23] Zachrisson, L. E.: Markov Games. In: Advances in Game Theory (M. Dresher, L. S. Shapley and A. W. Tucker, eds.), Princeton Univ. Press, Princeton N.J. 1964, pp. 211-253.