Geodesic
Multi-Agent Cooperation Q-Learning Algorithm Based on Constrained Markov Game
Computer Science and Information Systems, Tome 17 (2020) no. 2.

Voir la notice de l'article provenant de la source Computer Science and Information Systems website

Multi-Agent system has broad application in real world, whose security performance, however, is barely considered. Reinforcement learning is one of the most important methods to resolve Multi-Agent problems. At present, certain progress has been made in applying Multi-Agent reinforcement learning to robot system, man-machine match, and automatic, etc. However, in the above area, an agent may fall into unsafe states where the agent may find it difficult to bypass obstacles, to receive information from other agents and so on. Ensuring the safety of Multi-Agent system is of great importance in the above areas where an agent may fall into dangerous states that are irreversible, causing great damage. To solve the safety problem, in this paper we introduce a Multi-Agent Cooperation Q-Learning Algorithm based on Constrained Markov Game. In this method, safety constraints are added to the set of actions, and each agent, when interacting with the environment to search for optimal values, should be restricted by the safety rules, so as to obtain an optimal policy that satisfies the security requirements. Since traditional Multi-Agent reinforcement learning algorithm is no more suitable for the proposed model in this paper, a new solution is introduced for calculating the global optimum state-action function that satisfies the safety constraints. We take advantage of the Lagrange multiplier method to determine the optimal action that can be performed in the current state based on the premise of linearizing constraint functions, under conditions that the state-action function and the constraint function are both differentiable, which not only improves the efficiency and accuracy of the algorithm, but also guarantees to obtain the global optimal solution. The experiments verify the effectiveness of the algorithm.
Keywords: Markov game, Distributed perception, Multi-Agent cooperation, constrained Markov decision process 
@article{CSIS_2020_17_2_a14,
     author = {Yangyang Ge and Fei Zhu and Wei Huang and Peiyao Zhao and Quan Liu},
     title = {Multi-Agent {Cooperation} {Q-Learning} {Algorithm} {Based} on {Constrained} {Markov} {Game}},
     journal = {Computer Science and Information Systems},
     publisher = {mathdoc},
     volume = {17},
     number = {2},
     year = {2020},
     url = {http://geodesic.mathdoc.fr/item/CSIS_2020_17_2_a14/}
}
TY  - JOUR
AU  - Yangyang Ge
AU  - Fei Zhu
AU  - Wei Huang
AU  - Peiyao Zhao
AU  - Quan Liu
TI  - Multi-Agent Cooperation Q-Learning Algorithm Based on Constrained Markov Game
JO  - Computer Science and Information Systems
PY  - 2020
VL  - 17
IS  - 2
PB  - mathdoc
UR  - http://geodesic.mathdoc.fr/item/CSIS_2020_17_2_a14/
ID  - CSIS_2020_17_2_a14
ER  - 
%0 Journal Article
%A Yangyang Ge
%A Fei Zhu
%A Wei Huang
%A Peiyao Zhao
%A Quan Liu
%T Multi-Agent Cooperation Q-Learning Algorithm Based on Constrained Markov Game
%J Computer Science and Information Systems
%D 2020
%V 17
%N 2
%I mathdoc
%U http://geodesic.mathdoc.fr/item/CSIS_2020_17_2_a14/
%F CSIS_2020_17_2_a14
Yangyang Ge; Fei Zhu; Wei Huang; Peiyao Zhao; Quan Liu. Multi-Agent Cooperation Q-Learning Algorithm Based on Constrained Markov Game. Computer Science and Information Systems, Tome 17 (2020) no. 2. http://geodesic.mathdoc.fr/item/CSIS_2020_17_2_a14/