An extended version of average Markov decision processes on discrete spaces under fuzzy environment

Cruz-Suárez, Hugo ; Montes-de-Oca, Raúl ; Ortega-Gutiérrez, R. Israel

Kybernetika, Tome 59 (2023) no. 1, pp. 160-178

Abstract (VO)

The article presents an extension of the theory of standard Markov decision processes on discrete spaces and with the average cost as the objective function which permits to take into account a fuzzy average cost of a trapezoidal type. In this context, the fuzzy optimal control problem is considered with respect to two cases: the max-order of the fuzzy numbers and the average ranking order of the trapezoidal fuzzy numbers. Each of these cases extends the standard optimal control problem, and for each of them the optimal solution is related to a suitable standard optimal control problem, and it is obtained that (i) the optimal policy coincides with the optimal policy of this suitable standard control problem, and (ii) the fuzzy optimal value function is of a trapezoidal shape. Two models: a queueing system and a machine replacement problem are provided in order to examplify the theory given.

Détail

DOI : 10.14736/kyb-2023-1-0160

Classification : 90C40, 93C40
Keywords: Markov decision process; average criterion; trapezoidal fuzzy cost; max-order; average ranking

Bibliographie

[1] Arapostathis, A., Borkar, V. S., Fernández-Gaucherand, E., Gosh, M. K., Marcus, S. I.: Discrete-time controlled Markov processes with average cost criterion: a survey. SIAM J. Control Optim. 32 (1993), 2, 282-344. | DOI | MR

[2] Carrero-Vera, K., Cruz-Suárez, H., Montes-de-Oca, R.: Discounted Markov decision processes with fuzzy rewards induced by non-fuzzy systems. In: Proc. 10th International Conference on Operations Research and Enterprise Systems ICORES 2021, pp. 49-59.

[3] Carrero-Vera, K., Cruz-Suárez, H., Montes-de-Oca, R.: Markov decision proceses on finite spaces with fuzzy total reward. Kybernetika 58 (2022), 2, 180-199. | DOI | MR

[4] Chen, S. J., Chen, S. M.: Fuzzy risk analysis based on the ranking of generalized trapezoidal fuzzy numbers. Appl. Intell. 26 (2007), 1, 1-11. | DOI

[5] Chung, Y. L., Tsai, Z. N.: A quantized water-filling packet scheduling scheme for downlink transmissions in LTE-advanced systems with carrier aggregation. In: SoftCOM 2010, 18th International Conference on Software, Telecommunications and Computer Networks IEEE (2010), pp. 275-279.

[6] Diamond, P., Kloeden, P.: Metric Spaces of Fuzzy Sets: Theory and Applications. World Scientific, Singapore 1994. | MR

[7] Ebrahimnejad, A.: A simplified new approach for solving fuzzy transportation problems with generalized trapezoidal fuzzy numbers. Appl. Soft Comput. 19 (2014), 171-176. | DOI | MR

[8] Furukawa, N.: Parametric orders on fuzzy numbers and their roles in fuzzy optimization problems. Optimization 40 (1997), 171-192. | DOI | MR

[9] Hernández-Lerma, O., Lasserre, J. B.: Discrete-Time Markov Control Processes: Basic Optimality Criteria. Springer-Verlag, New York, 1996. | MR | Zbl

[10] Kageyama, M.: Credibilistic Markov decision processes: the average case. J. Comput. Appl. Math. 224 (2009), 1, 140-145. | DOI | MR

[11] Kaur, A., Kumar, A.: A new approach for solving fuzzy transportation problems using generalized trapezoidal fuzzy numbers. Appl. Soft Comput. 12 (2012), 3, 1201-1213. | DOI | MR

[12] Konstantin, E., Avrachenkov, E., Sanchez, E.: Fuzzy Markov chains and decision-making. Fuzzy Optim. Decis. Making 1 (2002), 12, 143-159. | DOI | MR

[13] Kurano, M., Yasuda, M., Nakagami, J., Yoshida, Y.: A fuzzy treatment of uncertain Markov decision processes: average case. In: Proc. ASSM2000 International Conference on Applied Stochastic System Modeling, Kyoto 2000, pp. 148-157. | MR

[14] Kurano, M., Yasuda, M., Nakagami, J., Yoshida, Y.: Markov-type fuzzy decision processes with a discounted reward on a closed interval. Eur. J. Oper. Res. 92 (1996), 3, 649-662. | DOI | MR

[15] Kurano, M., Yasuda, M., Nakagami, J., Yoshida, Y.: Markov decision processes with fuzzy rewards. J. Nonlinear Convex Anal. 4 (1996), 1, 105-116. | MR

[16] Kurano, M., Yasuda, M., Nakagami, J., Yoshida, Y.: Perceptive evaluation for the optimal discounted reward in Markov decision processes. In: International Conference on Modeling Decisions for Artificial Intelligence, Springer 2005, pp. 283-293.

[17] Kurano, M., Yasuda, M., Nakagami, J., Yoshida, Y.: A fuzzy approach to Markov decision processes with uncertain transition probabilities. Fuzzy Sets and Systems 157 (2006), 19, 2674-2682. | DOI | MR

[18] López-Díaz, M., Ralescu, D. A.: Tools for fuzzy random variables: embeddings and measurabilities. Comput. Statist. Data Anal. 51 (2006), 109-114. | DOI | MR

[19] Puri, M. L., Ralescu, D. A.: Fuzzy random variable. J. Math. Anal. Appl. 114 (1986), 402-422. | DOI | MR

[20] Puterman, M.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, 1994. | MR | Zbl

[21] Rani, D., Gulati, T. R.: A new approach to solve unbalanced transportation problems in imprecise environment. J. Transp. Secur. 7 (2014), 3, 277-287. | DOI

[22] Rani, D., Gulati, T. R., Kumar, A.: A method for unbalanced transportation problems in fuzzy environment. Sadhana 39 (2014),3, 573-581. | DOI | MR

[23] Rezvani, S., Molani, M.: Representation of trapezoidal fuzzy numbers with shape function. Ann. Fuzzy Math. Inform. 8 (2014), 89-112. | MR

[24] Ross, S.: Applied Probability Models with Optimization Applications. Holden Day, 1996. | MR

[25] Semmouri, A., Jourhmane, M., Belhallaj, Z.: Discounted Markov decision processes with fuzzy costs. Ann. Oper. Res. 295 (2020), 769-786. | DOI | MR

[26] Sennott, L.: Stochastic Dynamic Programming and Control of Queueing Systems. Systems. Wiley, New York 1999. | MR

[27] Syropoulos, A., Grammenos, T.: A Modern Introduction to Fuzzy Mathematics. Wiley, New Jersey 2020.

[28] Wang, J., Ma, X., Xu, Z., Zhan, J.: Three-way multi-attribute decision making under hesitant fuzzy environments. Inform. Sci. 552 (2021), 328-351. | DOI | MR

[29] Zadeh, L.: Fuzzy sets. Inform. Control 8 (1965), 338-353. | DOI | MR | Zbl