About stability of risk-seeking optimal stopping

Montes-de-Oca, Raúl ; Zaitseva, Elena

Kybernetika, Tome 50 (2014) no. 3, pp. 378-392

Abstract (VO)

We offer the quantitative estimation of stability of risk-sensitive cost optimization in the problem of optimal stopping of Markov chain on a Borel space $X$. It is supposed that the transition probability $p(\cdot |x)$, $x\in X$ is approximated by the transition probability $\widetilde{p}(\cdot |x)$, $x\in X$, and that the stopping rule $\widetilde{f}_*$ , which is optimal for the process with the transition probability $\widetilde{p}$ is applied to the process with the transition probability $p$. We give an upper bound (expressed in term of the total variation distance: $\sup_{x\in X}\|p(\cdot |x)-\widetilde{p}(\cdot |x)\|)$ for an additional cost paid for using the rule $\widetilde{f}_*$ instead of the (unknown) stopping rule $f_*$ optimal for $p$.

Détail

DOI : 10.14736/kyb-2014-3-0378

Classification : 60G40, 62L15, 90C40
Keywords: discrete-time Markov process; risk-seeking expected total cost; optimal stopping rule; stability index; total variation metric

Bibliographie

[1] Avila-Godoy, G., Fernández-Gaucherand, E.: Controlled Markov chains with exponential risk-sensitive criteria: modularity, structured policies and applications. In: Decision and Control 1998. Proc. 37th IEEE Conference. Vol. 1, IEEE, pp. 778-783.

[2] Bäuerle, N., Rieder, U.: Markov Decision Processes with Applications to Finance. Springer-Verlag, Berlin 2011. | MR | Zbl

[3] Borkar, V. S., Meyn, S. P.: Risk-sensitive optimal control for Markov decision processes with monotone cost. Math. Oper. Res. 27 (2002), 192-209. | DOI | MR | Zbl

[4] Cavazos-Cadena, R.: Optimality equations and inequalities in a class of risk-sensitive average cost Markov decision chains. Math. Methods Oper. Res. 71 (2010), 47-84. | DOI | MR | Zbl

[5] Cavazos-Cadena, R., Fernández-Gaucherand, E.: Controlled Markov chains with risk-sensitive criteria: Average costs, optimality equations, and optimal solutions. Math. Methods Oper. Res. 49 (1999), 299-324. | MR

[6] Cavazos-Cadena, R., Montes-de-Oca, R.: Optimal stationary policies in risk-sensitive dynamic programs with finite state space and nonegative rewards. Appl. Math. 27 (2000), 167-185. | MR

[7] Dijk, N. M. Van, Sladký, K.: Error bounds for nonnegative dynamic models. J. Optim. Theory Appl. 101 (1999), 449-474. | DOI | MR

[8] Devroye, L., Györfy, L.: Nonparametric Density Estimation: The $L_1$ View. John Wiley, New York 1986.

[9] Dynkin, E. B., Yushkevich, A. A.: Controlled Markov Processes. Springer Verlag, New York 1979. | MR

[10] Gordienko, E. I., Yushkevich, A. A.: Stability estimates in the problem of average optimal switching of a Markov chain. Math. Methods Oper. Res. 57 (2003), 345-365. | MR | Zbl

[11] Gordienko, E. I., Lemus-Rodríguez, E., Montes-de-Oca, R.: Average cost Markov control processes: stability with respect to the Kantorovich metric. Math. Methods Oper. Res. 70 (2009), 13-33. | DOI | MR | Zbl

[12] Gordienko, E. I., Salem, F.: Robustness inequalities for Markov control processes with unbounded costs. Syst. Control Lett. 33 (1998), 125-130. | DOI | MR

[13] Hernández-Lerma, O., Lasserre, J. B.: Further Topics on Discrete-time Markov Control Processes. Springer-Verlag, New York 1999. | MR | Zbl

[14] Jaśkiewicz, A.: Average optimality for risk-sensitive control with general state space. Ann. Appl. Probab. 17 (2007), 654-675. | DOI | MR | Zbl

[15] Kartashov, N. V.: Strong Stable Markov Chains. VSP, Utrecht 1996. | MR | Zbl

[16] Marcus, S. I., Fernández-Gaucherand, E., Hernández-Hernández, D. E., Coraluppi, S., Fard, P.: Risk sensitive Markov decision processes. Progress in System and Control Theory 22 (1997), 263-280. | MR

[17] Masi, G. B. Di, Stettner, L.: Infinite horizon risk sensitive control of discrete time Markov processes with small risk. Systems Control Lett. 40 (2000), 15-20. | DOI | MR | Zbl

[18] Meyn, S. P., Tweedie, R. L.: Markov Chains and Stochastic Stability. Springer-Verlag, London 1993. | MR | Zbl

[19] Montes-de-Oca, R., Salem-Silva, F.: Estimates for perturbations of an average Markov decision processes with a minimal state and upper bounded stochastically ordered Markov chains. Kybernetika 41 (2005), 757-772. | MR

[20] Muciek, B. K.: Optimal stopping of risk processes: model with interest rates. J. Appl. Probab. 39 (2002), 261-270. | DOI | MR

[21] Shiryaev, A. N.: Optimal Stopping Rules. Springer-Verlag, New York 1978. | MR | Zbl

[22] Shiryaev, A. N.: Essential of Stochastic Finance. Facts, Models, Theory. World Scientific Publishing Co., Inc., River Edge, N. J. 1999. | MR

[23] Sladký, K.: Bounds on discrete dynamic programming recursions I. Kybernetika 16 (1980), 526-547. | MR | Zbl

[24] Zaitseva, E.: Stability estimating in optimal stopping problem. Kybernetika 44 (2008), 400-415. | MR | Zbl