Denumerable Markov stopping games with risk-sensitive total reward criterion

Torres-Gomar, Manuel A. ; Cavazos-Cadena, Rolando ; Cruz-Suárez, Hugo

Kybernetika, Tome 60 (2024) no. 1, pp. 1-18

Abstract (VO)

This paper studies Markov stopping games with two players on a denumerable state space. At each decision time player II has two actions: to stop the game paying a terminal reward to player I, or to let the system to continue it evolution. In this latter case, player I selects an action affecting the transitions and charges a running reward to player II. The performance of each pair of strategies is measured by the risk-sensitive total expected reward of player I. Under mild continuity and compactness conditions on the components of the model, it is proved that the value of the game satisfies an equilibrium equation, and the existence of a Nash equilibrium is established.

Détail

DOI : 10.14736/kyb-2024-1-0001

Classification : 91A10, 91A15
Keywords: monotone operator; fixed point; equilibrium equation; Nash equilibrium; hitting time; bounded rewards

Bibliographie

[1] Alanís-Durán, A., Cavazos-Cadena, R.: An optimality system for finite average Markov decision chains under risk-aversion. Kybernetika 48 (2012), 1, 83-104. | MR

[2] Bäuerle, N., Rieder, U.: Markov Decision Processes with Applications to Finance. Springer-Verlag, New York 2011. | MR | Zbl

[3] Bäuerle, N., Rieder, U.: More risk-sensitive Markov decision processes. Math. Oper. Res. 39 (2014), 1, 105-120. | DOI | MR

[4] Balaji, S., Meyn, S. P.: Multiplicative ergodicity and large deviations for an irreducible Markov chain. Stoch. Proc. Appl. 90 (2000), 1, 123-144. | DOI | MR

[5] Bielecki, T., Hernández-Hernández, D., Pliska, S. R.: Risk sensitive control of finite state Markov chains in discrete time, with applications to portfolio management. Math. Methods Oper. Res. 50 (1999), 167-188. | DOI | MR | Zbl

[6] Borkar, V. S., Meyn, S. P.: Risk-sensitive optimal control for Markov decision process with monotone cost. Math. Oper. Res. 27 (2002), 1, 192-209. | DOI | MR

[7] Cavazos-Cadena, R., Hernández-Hernández, D.: A system of Poisson equations for a non-constant Varadhan functional on a finite state space. Appl. Math. Optim. 53 (2006), 101-119. | DOI | MR

[8] Cavazos-Cadena, R., Hernández-Hernández, D.: Nash equilibrium in a class of Markov stopping games. Kybernetika 48 (2012), 1027-1044. | MR

[9] Cavazos-Cadena, R., Rodríguez-Gutiérrez, L., Sánchez-Guillermo, D. M.: Markov stopping game with an absorbing state. Kybernetika 57 (2021), 3, 474-492. | DOI | MR

[10] Cavazos-Cadena, R., Cantú-Sifuentes, M., Cerda-Delgado, I.: Nash equilibria in a class of Markov stopping games with total reward criterion. Math. Methods Oper. Res. 94 (2021), 319-340. | DOI | MR

[11] Denardo, E. V., Rothblum, U. G.: A turnpike theorem for A risk-sensitive Markov decision process with stopping. SIAM J. Control Optim. 45 (2006), 2, 414-431. | DOI | MR

[12] Masi, G. B. Di, Stettner, L.: Risk-sensitive control of discrete-time Markov processes with infinite horizon. SIAM J. Control Optim. 38 (1999), 1, 61-78. | DOI | MR

[13] Masi, G. B. Di, Stettner, L.: Infinite horizon risk sensitive control of discrete time Markov processes with small risk. Systems Control Lett. 40 (2000), 1, 305-321. | DOI | MR | Zbl

[14] Masi, G. B. Di, Stettner, L.: Infinite horizon risk sensitive control of discrete time Markov processes under minorization property. SIAM J. Control Optim. 46 (2007), 1, 231-252. | DOI | MR

[15] Hernández-Lerma, O.: Adaptive Markov Control Processes. Springer, New York 1988. | MR | Zbl

[16] Howard, R., Matheson, J.: Risk-sensitive Markov decision processes. Management Science 18 (1972), 356-369. | DOI | MR

[17] Jaśkiewicz, A.: Average optimality for risk sensitive control with general state space. Ann. App. Probab. 17 (2007), 2, 654-675. | DOI | MR

[18] Kontoyiannis, I., Meyn, S. P.: Spectral theory and limit theorems for geometrically ergodic Markov processes. Ann. App. Probab. 13 (2003), 1, 304-362. | DOI | MR

[19] López-Rivero, J., Cavazos-Cadena, R., Cruz-Suárez, H.: Risk-sensitive Markov stopping games with an absorbing state. Kybernetika 58 (2022), 1, 101-122. | DOI | MR

[20] Martínez-Cortés, V. M.: Bipersonal stochastic transient Markov games with stopping times and total reward criteria. Kybernetika 57 (2021), 1, 1-14. | DOI | MR

[21] Pitera, M., Stettner, L.: Long run risk sensitive portfolio with general factors. Math. Meth. Oper. Res. 82 (2016), 2, 265-293. | DOI | MR

[22] Puterman, M.: Markov Decision Processes. Wiley, New York 1994. | MR | Zbl

[23] Sladký, K.: Ramsey growth model under uncertainty. In: Proc. 27th International Conference Mathematical Methods in Economics 2009 (H. Brozová, ed.), Kostelec nad Cernými lesy 2009, pp. 296-300.

[24] Sladký, K.: Risk-sensitive Ramsey growth model. In: Proce. 27th International Conference Mathematical Methods in Economics 2010 (M. Houda and J. Friebelová, eds.), Ceské Budějovice 2010, pp. 1-6.

[25] Sladký, K.: Risk-sensitive average optimality in Markov decision processes. Kybernetika 54 (2018), 1218-1230. | DOI | MR

[26] Stettner, L.: Risk sensitive portfolio optimization. Math. Meth. Oper. Res. 50 (1999), 3, 463-474. | DOI | MR