Stability estimating in optimal stopping problem

Zaitseva, Elena

Kybernetika, Tome 44 (2008) no. 3, pp. 400-415

Abstract (VO)

We consider the optimal stopping problem for a discrete-time Markov process on a Borel state space $X$. It is supposed that an unknown transition probability $p(\cdot |x)$, $x\in X$, is approximated by the transition probability $\widetilde{p}(\cdot |x)$, $x\in X$, and the stopping rule $\widetilde{\tau }_*$, optimal for $\widetilde{p}$, is applied to the process governed by $p$. We found an upper bound for the difference between the total expected cost, resulting when applying $\widetilde{\tau }_*$, and the minimal total expected cost. The bound given is a constant times $\displaystyle \sup \nolimits _{x\in X}\Vert p(\cdot |x)-\widetilde{p}(\cdot |x)\Vert $, where $\Vert \cdot \Vert $ is the total variation norm.

Détail

Classification : 60G40, 60J10
Keywords: discrete-time Markov process; optimal stopping rule; stability index; total variation metric; contractive operator; optimal asset selling

Bibliographie

[1] Allart P.: Optimal stopping rules for correlated random walks with a discount. J. Appl. Prob. 41 (2004), 483–496 | MR

[2] Bertsekas D. P.: Dynamic Programming: Deterministic and Stochastic Models. Prentice Hall, Englewood Cliffs, N. J. 1987 | MR | Zbl

[3] Bertsekas D. P., Shreve S. E.: Stochastic Optimal Control: The Discrete Time Case. Academic Press, New York 1979 | MR | Zbl

[4] Dijk N. M. Van: Perturbation theory for unbounded Markov reward process with applications to queueing systems. Adv. in Appl. Probab. 20 (1988), 99–111 | MR

[5] Dijk N. M. Van, Sladký K.: Error bounds for nonnegative dynamic models. J. Optim. Theory Appl. 101 (1999), 449–474 | MR

[6] Dynkin E. B., Yushkevich A. A.: Controlled Markov Process. Springer-Verlag, New York 1979 | MR

[7] Favero G., Runggaldier W. J.: A robustness results for stochastic control. Systems Control Lett. 46 (2002), 91–97 | MR

[8] Gordienko E. I.: An estimate of the stability of optimal control of certain stochastic and deterministic systems. J. Soviet Math. 59 (1992), 891–899. (Translated from the Russian publication of 1989) | MR

[9] Gordienko E. I., Salem F. S.: Robustness inequality for Markov control process with unbounded costs. Systems Control Lett. 33 (1998), 125–130 | MR

[10] Gordienko E. I., Yushkevich A. A.: Stability estimates in the problem of average optimal switching of a Markov chain. Math. Methods Oper. Res. 57 (2003), 345–365 | MR | Zbl

[11] Gordienko E. I., Lemus-Rodríguez E., Montes-de-Oca R.: Discounted cost optimality problem: stability with respect to weak metrics. In press in: Math. Methhods Oper. Res. (2008) | MR | Zbl

[12] Hernández-Lerma O., Lassere J. B.: Discrete-Time Markov Control Processes: Basic Optimality Criteria. Springer-Verlag, N.Y. 1996

[13] Jensen U.: An optimal stopping problem in risk theory. Scand. Actuarial J.2 (1997), 149–159 | MR | Zbl

[14] Meyn S. P., Tweedie R. L.: Markov Chains and Stochastic Stability. Springer-Verlag, London 1993 | MR | Zbl

[15] Montes-de-Oca R., Salem-Silva F.: Estimates for perturbations of an average Markov decision process with a minimal state and upper bounded by stochastically ordered Markov chains. Kybernetika 41 (2005), 757–772 | MR

[16] Montes-de-Oca R., Sakhanenko, A., Salem-Silva F.: Estimate for perturbations of general discounted Markov control chains. Appl. Math. 30 (2003), 287–304 | MR

[17] Muciek B. K.: Optimal stopping of a risk process: model with interest rates. J. Appl. Prob. 39 (2002), 261–270 | MR | Zbl

[18] Müller A.: How does the value function of a Markov decision process depend on the transition probabilities? Math. Oper. Res. 22 (1997), 872–885 | MR

[19] Schäl M.: Conditions for optimality in dynamic programming and for the limit of $n$-stage optimal policies to be optimal. Z. Wahrsch. verw. Gebiete 32 (1975), 179–196 | MR | Zbl

[20] Shiryaev A. N.: Optimal Stopping Rules. Springer-Verlag, New York 1978 | MR | Zbl

[21] Shiryaev A. N.: Essential of Stochastic Finance. Facts, Models, Theory. World Scientific Publishing Co., Inc., River Edge, N.J. 1999 | MR