关键词: 对抗博弈, 集中式决策, 协同作战, 智能决策, 深度强化学习

Abstract:

To address the issue of poor decision-making consistency in multi-agent cooperative operations within complex dynamic adversarial environments, a centralized command decision-making method for multi-agent combat operations based on deep reinforcement learning. Firstly, to better control multiple agents in generating combat actions with coordinated intentions, the multi-agent cooperative combat process is modeled and formally described as a Markov decision process. A state space distinguishing between scalar and entity information, along with a discrete composite action space, is constructed. A reward reshaping method based on task objective decomposition and contribution allocation is proposed. A centralized operational decision-making agent policy network model is designed, incorporating multi-feature information encoding, incomplete information reasoning, and multi-head composite action decoding. Secondly, to improve computational resource utilization and sample efficiency while ensuring the stability of the network training process, an distributed sampling training framework based on Actor-Learner is proposed. The model is trained using the SARD-PPO algorithm with policy entropy. Finally, simulation experiments conducted on a high-fidelity simulation platform verify the feasibility and effectiveness of the technical approach. By deploying the decision-making agent model for forward simulation, replay analysis and summary of the adversarial process are achieved.

Key words: adversarial games, centralized decision-making, collaborative operations, intelligent decision-making, deep reinforcement learning