基于改进MCTS的多无人机多任务联合决策

doi:10.12305/j.issn.1001-506X.2026.02.17

摘要/Abstract

摘要：

在多无人机协同突防过程中，针对无人机需完成目标分配与探干侦动作选择的任务决策存在模型构建难，对模型求解方法复杂度高的问题，提出一种蒙特卡罗树搜索(Monte Carlo tree search, MCTS)的改进方法实现多无人机多任务联合决策。首先，考虑无人机与雷达对抗中的角度、距离等态势要素，以及当前无人机动作执行成功概率和雷达状态有效概率，构建多无人机目标分配与探干侦动作统一决策数学模型。其次，提出搜索次数自适应调整的改进MCTS算法对模型求解，实现大规模解空间在线快速寻优。仿真结果表明，改进算法使多雷达系统对无人机的威胁程度下降约16.8%，相比多臂赌博机算法效果提升约5.08%，决策时间约0.23 s，比传统MCTS缩短约45.7%，有助于提升无人机战场生存率。

关键词: 多无人机协同, 任务联合决策, 目标分配, 蒙特卡罗树搜索算法

Abstract:

In the process of multiple unmanned aerial vehicles （UAV） cooperative penetration, to address the challenges of model construction and the high complexity of solving target allocation and detection jamming and reconnaissance action selection tasks, an improved method of Monte Carlo tree search （MCTS） is proposed to realize joint multi-mission decision making for multiple UAVs. First, a unified mathematical decision model for multi-UAV target allocation and reconnaissance actions is constructed, considering the situational factors such as angle and distance in the confrontation between UAV and radars, as well as the current probability of successful execution of UAV actions and the effective probability of radar states. An improved MCTS algorithm with adaptive adjustment of search frequency is proposed for fast online optimization in large solution spaces. Simulation results show that the improved algorithm decreases the threat level of multi-radar system to UAVs by about 16.8%, improves the effect of the algorithm compared to the multi-armed gambling machine by about 5.08%, and the decision time is about 0.23 s, which is about 45.7% shorter than that of the traditional MCTS, thereby improving UAV battlefield survivability.

Key words: multi-UAV collaboration, task joint decision, target assignment, Monte Carlo tree search (MCTS)

中图分类号:

TP 18

魏建林, 林彦超, 唐慧龙, 张旺, 王伟. 基于改进MCTS的多无人机多任务联合决策[J]. 系统工程与电子技术, 2026, 48(2): 556-568.

Jianlin WEI, Yanchao LIN, Huilong TANG, Wang ZHANG, Wei WANG. Multi-UAV multi-mission joint decision making based on improved MCTS[J]. Systems Engineering and Electronics, 2026, 48(2): 556-568.

图/表 26

表1

图1

表2

表3

图2

图3

图4

表4

表5

表6

表7

表8

表9

图5

图6

图7

图8

图9

图10

表10

图11

图12

图13

图14

图15

图16

参考文献 32

1	刘钢, 汤俊, 刘陈, et al. 无人飞行器集群协同行为建模技术综述[J]. 系统工程与电子技术, 2021, 43 (8): 2221- 2231.
	LIU G, TANG J, LIU C, et al. Survey of cooperative behavior modeling technology for unma-nned aerial vehicles cluster[J]. Systems Engineering and Electronics, 2021, 43 (8): 2221- 2231.
2	王建峰, 贾高伟, 郭正, 等. 多无人机协同任务规划方法研究综述[J]. 系统工程与电子技术, 2024, 46 (10): 3437−3450.
	WANG J F, JIA G W, GUO Z, Research status and development of multi-UAV system mission planning [J]. Systems Engineering and Electronics, 2024, 46 (10): 3437−3450.
3	TAN Y F, ZHOU C, QIAN F. Cooperative task allocation method for multi-unmanned aerial vehicles based on the modified genetic algorithm[J]. IET Intelligent Transport Systems, 2024, 18 (6): 1164- 1173. doi: 10.1049/itr2.12495
4	GHAURI S A, SARFRAZ M, QAMAR R A, et al. A review of multi-UAV task allocation algorithms for a search and rescue scenario[J]. Journal of Sensor and Actuator Networks, 2024, 13 (5): 47. doi: 10.3390/jsan13050047
5	FAN B Y, BO Y M. Learning improvement heuristics for multi-unmanned aerial vehicle task allocation[J]. Drones, 2024, 8 (11): 636. doi: 10.3390/drones8110636
6	ZHANG J D, YANG Q M, SHI G Q, et al. UAV cooperative air combat maneuver decision based on multi-agent reinforcement learning[J]. Journal of Systems Engineering and Electronics, 2021, 32 (6): 1421- 1438. doi: 10.23919/JSEE.2021.000121
7	WANG J F, JIA G W, LIN J C, et al. Cooperative task allocation for heterogeneous multi-UAV using multi-objective optimization algorithm[J]. Journal of Central South University, 2020, 27 (2): 432- 448. doi: 10.1007/s11771-020-4307-0
8	WANG G, LV X, CUI L Z, et al. The methods of task pre-allocation and reallocation for multi-UAV cooperative reconnaissance mission[J]. IET Collaborative Intelligent Manufacturing, 2023, 5 (4)
9	熊韫文, 魏才盛, 许丹, 等. 基于改进蒙特卡洛树搜索的无人机目标分配与突防决策方法 [J]. 中南大学学报(自然科学版), 2023, 54(8): 3132−3144.
	XIONG Y W, WEI C S, XU D, et al. Multi-UVA target allocation and penetration decision based on an improved Monte Carlo tree search method [J]. Journal of Central South University (Science and Technology), 2023, 54(8): 3132−3144.
10	文永明, 石晓荣, 黄雪梅, 等. 一种无人机集群对抗多耦合任务智能决策方法 [J]. 宇航学报, 2021, 42(4): 504−512.
	WEN Y M, SHI X R, HUANG X M, et al. An intelligent decision-making method for multi-coupling tasks of UAV cluster countermeasure [J]. Journal of Astronautics, 2021, 42(4): 504−512.
11	MAO X, WU G H, FAN M F, et al. DL-DRL: A double-level deep reinforcement learning approach for large-scale task scheduling of multi-UAV[J]. IEEE Trans. on Automation Science and Engineering, 2024, 22, 1028- 1044.
12	CHEN W K, HAI X S, HU Y M, et al. Hierarchical decision-making framework for multi-UAV task assignment via enhanced pigeon-inspired optimization[J]. Guidance, Navigation and Control, 2023, 3 (4): 2350028.
13	LUONG P, GAGNON F, TRAN L N, et al. Deep reinforcement learning-based resource allocation in cooperative uav-assisted wireless networks[J]. IEEE Trans. on Wireless Communications, 2021, 20 (11): 7610- 7625. doi: 10.1109/TWC.2021.3086503
14	LIANG Z B, LI Q, FU G D. Multi-UAV collaborative search and attack mission decision-making in unknown environments[J]. Sensors, 2023, 23 (17) doi: 10.1109/JSEN.2023.3306185
15	邹子缘, 陈琪锋. 基于决策树搜索的空间飞行器集群对抗目标分配方法[J]. 航空学报, 2022, 43 (S1): 78- 88.
	ZOU Z Y, CHEN Q F. Decision tree-based target assignment for confrontation of multiple space vehicles[J]. Acta Aeronautica et Astronautica Sinica, 2022, 43 (S1): 78- 88.
16	郑红星, 郭继峰, 谢旭东, 等. 一种未知动态环境下异构无人机集群分布式联盟形成方法[J]. 宇航学报, 2022, 43 (2): 189- 197.
	ZHENG H X, GUO J F, XIE X D, et al. A distributed coalition formation method of heterogeneous UAV swarm in unknown dynamic environment[J]. Journal of Astronautics, 2022, 43 (2): 189- 197.
17	BROWNE C B, POWLEY E, WHITEHOUSE D, et al. A survey of Monte Carlo tree search methods[J]. IEEE Trans. on Computational Intelligence and AI in Games, 2012, 4 (1): 1- 43. doi: 10.1109/TCIAIG.2012.2186810
18	HOFFMAN M, SONG E, BRUNDAGE M P, et al. Online improvement of condition-based maintenance policy via Monte Carlo tree search[J]. IEEE Trans. on Automation Science and Engineering, 2022, 19 (3): 2540- 2551. doi: 10.1109/TASE.2021.3088603
19	LIU Z C, HAO M R. Heterogeneous unmanned aerial vehicles cooperative search approach for complex environments[J]. Engineering Applications of Artificial Intelligence, 2024, 138, 109384. doi: 10.1016/j.engappai.2024.109384
20	CHEN T Y, MIAO Z H, LI W M, et al. A learning-based memetic algorithm for a cooperative task allocation problem of multiple unmanned aerial vehicles in smart agriculture[J]. Swarm and Evolutionary Computation, 2024, 91, 101694. doi: 10.1016/j.swevo.2024.101694
21	CHOUR K, PRADEEP P, MUNISHKIN A A, et al. Aerial vehicle routing and scheduling for UAS traffic management: a hybrid Monte Carlo tree search approach [C]// Proc. of the IEEE/AIAA 42nd Digital avionics Systems Conference, 2023.
22	LI Y Y, CHEN W Y, LIU S K, et al. Multi-UAV cooperative air combat target assignment method based on VNS-IBPSO in complex dynamic environment[J]. International Journal oF Aerospace Engineering, 2024 (1): 9980746.
23	张嘉辉, 蒙志君, 何家政, 等. 基于改进能量谷优化的多无人机空战目标分配[J]. 系统工程与电子技术, 2024, 46 (11): 3754- 3763.
	ZHANG J H, MENG Z J, HE J Z, et al. Multi-UAV air combat target allocation based on improved energy valley optimization[J]. Systems Engineering and Electronics, 2024, 46 (11): 3754- 3763.
24	XING H X, XING Q H, WANG K. A joint allocation method of multi-jammer cooperative jamming resources based on suppression effectiveness[J]. Mathematics, 2023, 11 (4): 1- 25.
25	LU D J, WANG X, WU X T, et al. Adaptive allocation strategy for cooperatively jamming netted radar system based on improved cuckoo search algorithm[J]. Defence Technology, 2023, 24, 285- 297. doi: 10.1016/j.dt.2022.04.013
26	ZHANG D L, SUN J, YI W, et al. Joint jamming beam and power scheduling for suppressing netted radar system [C]// Proc. of the IEEE Radar Conference, 2021.
27	李世忠, 王国宏, 吴巍, 等. 分布式干扰下组网雷达目标检测与跟踪技术[J]. 系统工程与电子技术, 2012, 34(4): 782−788.
	LI S Z, WANG G H, WU W, et al. Detect and tracking technique for radar network in the presence of distributed jamming. Systems Engineering and Electronics, 2012, 34(4): 782−788.
28	陆德江, 王星, 陈游, 等. 联合多种资源协同干扰组网雷达系统的自适应调度方法[J]. 系统工程与电子技术, 2023, 45(9): 2744−2754.
	LU D J, WANG X, CHEN Y, et al. Adaptive scheduling method of joint multi-resource for cooperative interference of networked radar system, 2023, 45(9): 2744−2754.
29	WANG C, JIU B, PU W Q, et al. Anti-jamming equilibrium strategy learning of frequency agile radar based on Monte Carlo tree search [C]// Proc. of the IEEE International Radar Conference, 2023.
30	, et al. Model checking for probabilistic multiagent systems[J]. Journal of Computer Science and Technology, 2023, 38 (5): 1162- 1186. doi: 10.1007/s11390-022-1218-6
31	罗俊仁, 张万鹏, 陆丽娜, 等. 即时策略博弈在线对抗规划方法综述[J]. 计算机科学, 2022, 49(6): 287−296.
	LUO J R, ZHANG W P, LU L N, et al. Survey on online adversarial planning for real-time strategy game[J]. Computer Science, 2022, 49(6): 287−296.
32	CHENG X T, MAGHSUDI S. Distributed consensus algorithm for decision-making in multi-agent multi-armed bandit[J]. IEEE Trans. on Control of Network Systems, 2024, E107A(5): 799−805.

对抗双方	数量	动作
红方多无人机	${N_U}$	探测
		干扰
		侦察
蓝方多雷达	${N_{{\mathrm{tar}}}}$	搜索
		确认
		跟踪
		识别
		干扰

影响因素	状态有效概率
搜索	${P_{{\mathrm{b}}\_{\mathrm{s}}}}$
确认	${P_{{\mathrm{b}}\_{\mathrm{c}}}}$
跟踪	${P_{{\mathrm{b}}\_{\mathrm{t}}}}$
识别	${P_{{\mathrm{b}}\_{\mathrm{r}}}}$
干扰	${P_{{\mathrm{b}}\_{\mathrm{j}}}}$

决策动作	动作执行成功概率
探测	${P_{{\mathrm{r}}\_{\mathrm{d}}}}$
干扰	${P_{{\mathrm{r}}\_{\mathrm{j}}}}$
侦察	${P_{{\mathrm{r}}\_{\mathrm{r}}}}$

序号	类型	雷达状态	X轴位置/km	Y轴位置/km
1	1	2	−496.07	13.92
2	1	3	−497.57	32.41
3	1	3	−468.87	1.13
4	1	2	−469.67	20.31
5	1	−1	−469.56	40.44

序号	X轴位置/km	Y轴位置/km	Z轴位置/km	速度/(km/s)
1	−594.51	11.53	31.46	1.60
2	−594.44	21.61	31.56	1.60
3	−595.13	30.93	30.86	1.60