Approximation, estimation and control of stochastic systems under a randomized discounted cost criterion

González-Hernández, Juan   ; López-Martínez, Raquiel R.   ; Minjárez-Sosa, J. Adolfo

Kybernetika, Tome 45 (2009) no. 5, pp. 737-754

Abstract (VO)

The paper deals with a class of discrete-time stochastic control processes under a discounted optimality criterion with random discount rate, and possibly unbounded costs. The state process $\left\{ x_{t}\right\} $ and the discount process $\left\{ \alpha _{t}\right\} $ evolve according to the coupled difference equations $x_{t+1}=F(x_{t},\alpha _{t},a_{t},\xi _{t}),$ $ \alpha _{t+1}=G(\alpha _{t},\eta _{t})$ where the state and discount disturbance processes $\{\xi _{t}\}$ and $\{\eta _{t}\}$ are sequences of i.i.d. random variables with densities $\rho ^{\xi }$ and $\rho ^{\eta }$ respectively. The main objective is to introduce approximation algorithms of the optimal cost function that lead up to construction of optimal or nearly optimal policies in the cases when the densities $\rho ^{\xi }$ and $\rho ^{\eta }$ are either known or unknown. In the latter case, we combine suitable estimation methods with control procedures to construct an asymptotically discounted optimal policy.

Détail

Classification : 90C40, 93C55, 93E10, 93E20
Keywords: discounted cost; random rate; stochastic systems; approximation algorithms; density estimation

Bibliographie

[1] H. Berument, Z. Kilinc, and U. Ozlale: The effects of different inflation risk premiums on interest rate spreads. Physica A 333 (2004), 317–324. | MR

[2] L. Devroye and L. Györfi: Nonparametric Density Estimation the $L_{1}$ View. Wiley, New York 1985. | MR

[3] E. B. Dynkin and A. A. Yushkevich: Controlled Markov Processes. Springer–Verlag, New York 1979. | MR

[4] A. Gil and A. Luis: Modelling the U. S. interest rate in terms of I(d) statistical model. Quart. Rev. Economics and Finance 44 (2004), 475–486.

[5] R. Hasminskii and I. Ibragimov: On density estimation in the view of Kolmogorov’s ideas in approximation theory. Ann. Statist. 18 (1990), 999–1010. | MR

[6] J. González-Hernández, R. R. López-Martínez, and R. Pérez-Hernández: Markov control processes with randomized discounted cost in Borel space. Math. Meth. Oper. Res. 65 (2007), 27–44.

[7] S. Haberman and J. Sung: Optimal pension funding dynamics over infinite control horizon when stochastic rates of return are stationary. Insurance Math. Econom. 36 (2005), 103–116. | MR

[8] O. Hernández-Lerma: Adaptive Markov Control Processes. Springer–Verlag, New York 1989. | MR

[9] O. Hernández-Lerma and J. B. Lasserre: Discrete-Time Markov Control Processes: Basic Optimality Criteria. Springer–Verlag, New York 1996. | MR

[10] O. Hernández-Lerma and J. B. Lasserre: Further Topics on Discrete-Time Markov Control Processes. Springer–Verlag, New York 1999. | MR

[11] O. Hernández-Lerma and W. Runggaldier: Monotone approximations for convex stochastic control problems. J. Math. Syst., Estimation, and Control 4 (1994), 99–140. | MR

[12] P. Lee and D. B. Rosenfield: When to refinance a mortgage: a dynamic programming approach. European J. Oper. Res. 166 (2005), 266–277.

[13] R. G. Newell and W. A. Pizer: Discounting the distant future: how much do uncertain rates increase valuation? J. Environmental Economic and Management 46 (2003), 52–71.

[14] B. Sack and V. Wieland: Interest-rate smooothing and optimal monetary policy: A review of recent empirical evidence. J. Econom. Business 52 (2000), 205–228.

[15] M. Schäl: Conditions for optimality and for the limit of n-stage optimal policies to be optimal. Z. Wahrsch. Verw. Gerb. 32 (1975), 179–196. | MR

[16] M. Schäl: Estimation and control in discounted stochastic dynamic programming. Stochastics 20 (1987), 51–71. | MR

[17] N. L. Stockey and R. E. Lucas, Jr.: Recursive Methods in Economic Dynamics. Harvard University Press, Cambridge, MA 1989. | MR