Comparing Reinforcement Learning Algorithms for a Trip Building Task: a Multi-objective Approach Using Non-Local Information

Henrique U. Gobbi ; Guilherme Dytz dos Santos ; Ana L. C. Bazzan

Computer Science and Information Systems, Tome 21 (2024) no. 1

Résumé

Using reinforcement learning (RL) to support agents in making decisions that consider more than one objective poses challenges. We formulate the problem of multiple agents learning how to travel from A to B as a reinforcement learning task modeled as a stochastic game, in which we take into account: (i) more than one objective, (ii) non-stationarity, (iii) communication of local and non-local information among the various actors. We use and compare RL algorithms, both for the single objective (Q-learning), as well as for multiple objectives (Pareto Q-learning), with and without non-local communication. We evaluate these methods in a scenario in which hundreds of agents have to learn how to travel from their origins to their destinations, aiming at minimizing their travel times, as well as the carbon monoxide vehicles emit. Results show that the use of non-local communication reduces both travel time and emissions.

Détail

Keywords: reinforcement learning, multi-agent systems, multi-objective reinforcement learning, route choice