Estimation and control in finite Markov decision processes with the average reward criterion

Rolando Cavazos-Cadena¹ ; Raúl Montes-de-Oca²

¹ Departamento de Estadística y Cálculo Universidad Autónoma Agraria Antonio Narro Buenavista, Saltillo COAH 25315, México
² Departamento de Matemáticas Universidad Autónoma Metropolitana Campus Iztapalapa Avenida San Rafael Atlixco #186 Colonia Vicentina México 09340, D.F., México

Applicationes Mathematicae, Tome 31 (2004) no. 2, pp. 127-154

Résumé

This work concerns Markov decision chains with finite state and action sets. The transition law satisfies the simultaneous Doeblin condition but is unknown to the controller, and the problem of determining an optimal adaptive policy with respect to the average reward criterion is addressed. A subset of policies is identified so that, when the system evolves under a policy in that class, the frequency estimators of the transition law are consistent on an essential set of admissible state-action pairs, and the non-stationary value iteration scheme is used to select an optimal adaptive policy within that family.

Détail

DOI : 10.4064/am31-2-1

Keywords: work concerns markov decision chains finite state action sets transition law satisfies simultaneous doeblin condition unknown controller problem determining optimal adaptive policy respect average reward criterion addressed subset policies identified system evolves under policy class frequency estimators transition law consistent essential set admissible state action pairs non stationary value iteration scheme select optimal adaptive policy within family