对抗场景下基于多智能体联盟更新的指控逻辑链优化问题研究

doi:10.12305/j.issn.1001-506X.2026.08.23

Abstract

Abstract:

In adversarial scenario, due to factors such as complex terrain, rapid environmental changes, and fragmented battlefield information, how to conduct dynamic cooperation through multi-agent coalition, and update and optimize all links of the logic chain in real time, is one of the core challenges to improve operational effectiveness. This paper models the problem of constructing a land warfare logic chain as a problem of constructing a multi-agent alliance. Aiming at the challenges of low coalition decision-making efficiency and unstable cooperation, a coalition update mechanism based on game theory and mechanism design is proposed. Combined with reinforcement learning technology, the dynamic game behavior among multi-agents is optimized, thereby improving the cooperation efficiency of the coalition in complex confrontation environments. The introduced adaptive mechanism design can dynamically adjust the decision-making rules of coalition members according to changes in battlefield situation, ensuring the flexibility and robustness of the coalition. Experimental results show that the proposed method significantly improves the operational effectiveness and collaboration performance of the coalition in various adversarial scenarios, verifying its effectiveness. It can provide theoretical support and technical guidance for the construction of agent coalition in adversarial scenarios.

Key words: game theory, mechanism design, logic chain construction, reinforcement learning

CLC Number:

TP18
E11

Jie ZHANG, Chao WANG, Yu LIU, Dong LI, Zhiqun CHEN, Tianqi LU. Research on command and control logic chain optimization based on multi-agent coalition updates in adversarial scenarios[J]. Systems Engineering and Electronics, 2026, 48(8): 2782-2789.

Figures/Tables 4

References 34

1	刘祥雨, 王刚, 郭相科, 等. 面向区域防空场景的杀伤链设计方法[J]. 系统工程与电子技术, 2025, 47 (5): 1582- 1599.
	LIU X Y, WANG G, GUO X K, et al. Design method of kill chain for regional air defense scenarios[J]. Systems Engineering and Electronics, 2025, 47 (5): 1582- 1599.
2	钱林方, 张龙, 佟明昊, 等. 面向信息化智能打击的现代火炮技术发展与展望[J]. 陆军工程大学学报, 2025, 4 (2): 1- 8.
	QIAN L F, ZHANG L, TONG M H, et al. Development and prospect of modern artillery technology for information-based intelligent strike[J]. Journal of Army Engineering University, 2025, 4 (2): 1- 8.
3	张昊, 邓大松. 未来无人反无人杀伤链展望分析[J]. 战术导弹技术, 2024, 1 (5): 160- 167.
	ZHANG H, DENG D S. Prospective analysis of future unmanned counter-unmanned kill chain[J]. Tactical Missile Technology, 2024, 1 (5): 160- 167.
4	JOHNSON B, GREEN J M, BURNS G, et al. Mapping artificial intelligence to the naval tactical kill chain[J]. Naval Engineers Journal, 2023, 135 (1): 155- 166.
5	王耀祖, 尚柏林, 宋笔锋, 等. 基于杀伤链的作战体系网络关键节点识别方法[J]. 系统工程与电子技术, 2023, 45 (3): 736- 744.
	WANG Y Z, SHANG B L, SONG B F, et al. Key node identification method of combat system network based on kill chain[J]. Systems Engineering and Electronics, 2023, 45 (3): 736- 744.
6	ZHANG Z L, TIAN S P, KONG L T. Application for over-the-air computation in command and control data link systems[J]. Command Information System and Technology, 2022, 13 (6): 55- 62.
7	XU C T, ZHOU F, CHEN H H, et al. Autonomous adaptive capability evaluation method for intelligent command and control system[J]. Command Information System and Technology, 2023, 14 (6): 29- 38.
8	ZHANG J L, HAN K, ZHANG P, et al. A survey on joint-operation application for unmanned swarm formations under a complex confrontation environment[J]. Journal of Systems Engineering and Electronics, 2023, 34 (6): 1432- 1446. doi: 10.23919/JSEE.2023.000162
9	徐成涛, 周芳, 陈洪辉, 等. 智能化指挥控制系统自主适变能力评估方法[J]. 指挥信息系统与技术, 2023, 14 (6): 29- 38.
	XU C T, ZHOU F, CHEN H H, et al. Assessment method for autonomous adaptive capability of intelligent command and control system[J]. Command Information System and Technology, 2023, 14 (6): 29- 38.
10	张福林, 孔凡照, 黄凯. 基于智能调度策略的火力规划技术[J]. 指挥信息系统与技术, 2024, 15 (4): 46- 49，62.
	ZHANG F L, KONG F Z, HUANG K. Firepower planning technology based on intelligent scheduling strategy[J]. Command Information System and Technology, 2024, 15 (4): 46- 49，62.
11	GAO X L, WANG R, LIU Z H, et al. A land-based war-gaming simulation method based on multi-agent proximal policy optimization[C]//Proc. of the IEEE 24th International Conference on Software Quality, Reliability, and Security Companion, 2024: 611–618.
12	BLACK K, JANNER M, DU Y, et al. Training diffusion models with reinforcement learning [EB/OL]. [2025-07-01]. https://arxiv.org/abs/2305.13301.
13	KORKMAZ E. Adversarial robust deep reinforcement learning requires redefining robustness[C]//Proc. of the AAAI Conference on Artificial Intelligence, 2023: 8369–8377.
14	CHEN Y J, ZHENG Z C, GONG X L. Marnet: backdoor attacks against cooperative multi-agent reinforcement learning[J]. IEEE Trans. on Dependable and Secure Computing, 2022, 20 (5): 4188- 4198.
15	XU Z W, BAI Y P, ZHANG B, et al. Haven: hierarchical cooperative multi-agent reinforcement learning with dual coordination mechanism[C]// Proc. of the AAAI Conference on Artificial Intelligence, 2023: 11735–11743.
16	WU Z F, YU C, CHEN C, et al. Models as agents: optimizing multi-step predictions of interactive local models in model-based multi-agent reinforcement learning[C]// Proc. of the AAAI Conference on Artificial Intelligence, 2023: 10435–10443.
17	BOGGESS K, KRAUS S, FENG L. Explainable multi-agent reinforcement learning for temporal queries[C]// Proc. of the 32nd International Joint Conference on Artificial Intelligence, 2023: 55–63.
18	CHEN L, LU K, RAJESWARAN A, et al. Decision transformer: Reinforcement learning via sequence modeling[J]. Advances in Neural Information Processing Systems, 2021, 34 (1): 15084- 15097.
19	CHRISTIANOS F, PAPOUDAKIS G, RAHMAN M A, et al. Scaling multi-agent reinforcement learning with selective parameter sharing[C]//Proc. of the International Conference on Machine Learning, 2021: 1989–1998.
20	LIU I J, JAIN U, YEH R A, et al. Cooperative exploration for multi-agent deep reinforcement learning[C]//Proc. of the International Conference on Machine Learning, 2021: 6826–6836.
21	文永明, 李博研, 张宁宁, 等. 基于深度强化学习的多智能体编队协同控制[J]. 指挥信息系统与技术, 2023, 14 (5): 75- 79.
	WEN Y M, LI B Y, ZHANG N N, et al. Multi-agent formation cooperative control based on deep reinforcement learning[J]. Command Information System and Technology, 2023, 14 (5): 75- 79.
22	唐志一, 蔡颖, 王会. 基于自适应加权算法的多传感器数据融合方法[J]. 指挥信息系统与技术, 2022, 13 (5): 66- 70.
	TANG Z Y, CAI Y, WANG H. Multi-sensor data fusion method based on adaptive weighted algorithm[J]. Command Information System and Technology, 2022, 13 (5): 66- 70.
23	KOLPACZKI P, BENGS V, MUSCHALIK M, et al. Approximating the shapley value without marginal contributions[C]//Proc. of the AAAI Conference on Artificial Intelligence, 2024: 13246–13255.
24	LIU L, HU Y J, GAO Y, et al. Value function transfer for deep multi-agent reinforcement learning based on N-step returns[C]//Proc. of the 28th International Joint Conference on Artificial Intelligence, 2019: 457–463.
25	JIN C, KRISHNAMURTHY A, SIMCHOWITZ M, et al. Reward-free exploration for reinforcement learning[C]//Proc. of the International Conference on Machine Learning, 2020: 4870–4879.
26	ZANETTE A, WAINWRIGHT M. Stabilizing Q-learning with linear architectures for provable efficient learning[C]//Proc. of the International Conference on Machine Learning, 2022: 25920–25954.
27	WANG J, ZHANG Y, KIM T K, et al. Shapley Q-value: a local reward approach to solve global reward games[C]//Proc. of the AAAI Conference on Artificial Intelligence, 2020: 7285–7292.
28	OLSEN L H B, GLAD I K, JULLUM M, et al. A comparative study of methods for estimating model-agnostic Shapley value explanations[J]. Data Mining and Knowledge Discovery, 2024, 38 (4): 1782- 1829. doi: 10.1007/s10618-024-01016-z
29	LITTMAN M L. Markov games as a framework for multi-agent reinforcement learning[C]//Proc. of the 11th International Conference on Machine Learning, 1994: 157–163.
30	VASANKARI L, SAASTAMOINEN K. Strategizing the shallows: leveraging multi-agent reinforcement learning for enhanced tactical decision-making in Littoral Naval Warfare[C]//Proc. of the International Conference on Artificial Intelligence Applications and Innovations, 2024: 129–141.
31	ZHANG K, KAKADE S, BASAR T, et al. Model-based multi-agent RL in zero-sum Markov games with near-optimal sample complexity[J]. Advances in Neural Information Processing Systems, 2020, 33 (1): 1166- 1178.
32	KURNIAWATI H. Partially observable markov decision processes and robotics[J]. Annual Review of Control, Robotics, and Autonomous Systems, 2022, 5 (1): 253- 277. doi: 10.1146/annurev-control-042920-092451
33	MOGHADDAM A R, KEBRIAEI H. Multiagent reinforcement learning for Nash equilibrium seeking in general-sum Markov games[J]. IEEE Trans. on Systems, Man, and Cybernetics: Systems, 2025, 55(1): 221–227.
34	DA-COSTA A R, PARIS D M, LUJAK M, et al. Dynamic and cooperative multi-agent task allocation: enhancing Nash equilibrium through learning[C]//Proc. of the IEEE International Conference on Systems, Man, and Cybernetics, 2024: 1860–1866.

参数	数值
智能体数量	50
作战区域/m	1000×1000
学习率α	0.1
折扣因子γ	0.9
训练轮数	1000
每次仿真任务轮数	200

算法	限制时间/s	迭代次数	平均目标函数值
本文算法	1	20	8.33
	2	50	7.118
	5	100	7.089
未改进的DQN算法	1	20	18.353
	2	50	10.108
	5	100	9.089

方法	任务完成率/%	资源利用率/%	协同效能（分数）	抗干扰鲁棒性（分数）
静态联盟策略	70.5	55	3.81	4.55
拍卖机制任务分配	65.66	60.8	5.68	5.69
强化学习协作模型	85	80.5	8.45	7.88
本文算法	98.1	92.7	10.88	9.65

[1]	Tianran YIN, He LUO, Yue SHI, Xiaodie QIANG, Guoqiang WANG. A satellite and UAV joint mission planning method based on a reinforcement hybrid genetic algorithm [J]. Systems Engineering and Electronics, 2026, 48(7): 2293-2306.
[2]	Jingli YANG, Ying WANG, Tianyu GAO, Xiaotong FANG. Dynamic reconfiguration method for shipboard electronic information systems for fault propagation risk suppression [J]. Systems Engineering and Electronics, 2026, 48(7): 2307-2318.
[3]	Qiang LI, Di ZHOU, Siyuan LI, Yurong LIN. Adaptive guidance for missile based on meta-learning and predictive control [J]. Systems Engineering and Electronics, 2026, 48(7): 2424-2433.
[4]	Zan MA, Yubin LIU, Jie BAI, Yong CHEN, Shuguang SUN. Intelligent avoidance airworthiness safety risk assessment of unmanned aerial vehicle based on hierarchical STPA-MC [J]. Systems Engineering and Electronics, 2026, 48(6): 2000-2013.
[5]	Zhenshuai JIA, Bing XIAO, Hanyu QIAN, Zheyu ZHANG. Spacecraft on-orbit observation maneuver decision-making method based on multi-policy learning [J]. Systems Engineering and Electronics, 2026, 48(5): 1590-1598.
[6]	Huaqing ZHANG, Xiaofei ZHANG, Mingrui HAO, Jixiang JIANG, Shan LI. Robust multi-agent cooperative confrontation policy ofﬂine reinforcement learning [J]. Systems Engineering and Electronics, 2026, 48(5): 1670-1681.
[7]	Shaohui ZHANG, Lulu LI, Yafei LI, Qingshun WU, Guanfeng LI, Mingliang XU. Research on the planning methods of ammunition support operations for carrier-based aircraft: a survey [J]. Systems Engineering and Electronics, 2026, 48(4): 1303-1321.
[8]	Zhigang JIN, Zepei LIU, Xiaodong WU. Survey of game theory applications in Internet of Things security and privacy [J]. Systems Engineering and Electronics, 2026, 48(3): 1072-1082.
[9]	Pengcheng YANG, Qingqing YANG, Yingying GAO, Zhiwei YANG, Kewei YANG, Bo AI. Path planning for maritime moving targets search based on reinforcement learning [J]. Systems Engineering and Electronics, 2026, 48(2): 515-523.
[10]	Jiahui FANG, Kebo LI, Yangang LIANG. Reachability analysis of space proximity behavior patterns based on optimal impulse [J]. Systems Engineering and Electronics, 2026, 48(2): 652-659.
[11]	Xu WANG, Guangbin CAI, Xiaoya YU, Ziqi YE, Bin SHAN. Attitude control of hypersonic vehicle based on dual-dynamic PPO algorithm [J]. Systems Engineering and Electronics, 2026, 48(2): 694-704.
[12]	Zhikang LIN, Jialei LIU, Jiazhi MA, Longfei SHI, Jinbao XU. Counter-anti-radiation method method using distributed radiation source blinking decoy [J]. Systems Engineering and Electronics, 2026, 48(1): 1-11.
[13]	Jinyan XUE, Yasheng ZHANG, Xuefeng TAO, Mingqi YANG, Shuailong ZHAO. Advances in orbital maneuver control for GEO spacecraft [J]. Systems Engineering and Electronics, 2026, 48(1): 290-300.
[14]	Chuanlong SONG, Qianwu ZHANG, Jian HE, Wenjun ZHOU, Hui WANG, Weiwei KONG, Wenbo TIAN. Edge computing task offloading method of satellite-ground collaborative based on MADDPG algorithm [J]. Systems Engineering and Electronics, 2026, 48(1): 350-360.
[15]	Peng YAO, Meiyu HAN, Dechuan WANG, Zhicheng GAO. Multiple unmanned surface vehicles pursuit method based on adversarial evolutionary reinforcement learning [J]. Systems Engineering and Electronics, 2025, 47(9): 2960-2970.

Research on command and control logic chain optimization based on multi-agent coalition updates in adversarial scenarios

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 4

References 34

Related Articles 15

Recommended Articles

Metrics

Comments