基于改进MCTS的多无人机多任务联合决策

doi:10.12305/j.issn.1001-506X.2026.02.17

Abstract

Abstract:

In the process of multiple unmanned aerial vehicles （UAV） cooperative penetration, to address the challenges of model construction and the high complexity of solving target allocation and detection jamming and reconnaissance action selection tasks, an improved method of Monte Carlo tree search （MCTS） is proposed to realize joint multi-mission decision making for multiple UAVs. First, a unified mathematical decision model for multi-UAV target allocation and reconnaissance actions is constructed, considering the situational factors such as angle and distance in the confrontation between UAV and radars, as well as the current probability of successful execution of UAV actions and the effective probability of radar states. An improved MCTS algorithm with adaptive adjustment of search frequency is proposed for fast online optimization in large solution spaces. Simulation results show that the improved algorithm decreases the threat level of multi-radar system to UAVs by about 16.8%, improves the effect of the algorithm compared to the multi-armed gambling machine by about 5.08%, and the decision time is about 0.23 s, which is about 45.7% shorter than that of the traditional MCTS, thereby improving UAV battlefield survivability.

Key words: multi-UAV collaboration, task joint decision, target assignment, Monte Carlo tree search (MCTS)

CLC Number:

TP 18

Jianlin WEI, Yanchao LIN, Huilong TANG, Wang ZHANG, Wei WANG. Multi-UAV multi-mission joint decision making based on improved MCTS[J]. Systems Engineering and Electronics, 2026, 48(2): 556-568.

Figures/Tables 26

Table 1

Fig.1

Table 2

Table 3

Fig.2

Fig.3

Fig.4

Table 4

Table 5

Table 6

Table 7

Table 8

Table 9

Fig.5

Fig.6

Fig.7

Fig.8

Fig.9

Fig.10

Table 10

Fig.11

Fig.12

Fig.13

Fig.14

Fig.15

Fig.16

References 32

1	刘钢, 汤俊, 刘陈, et al. 无人飞行器集群协同行为建模技术综述[J]. 系统工程与电子技术, 2021, 43 (8): 2221- 2231.
	LIU G, TANG J, LIU C, et al. Survey of cooperative behavior modeling technology for unma-nned aerial vehicles cluster[J]. Systems Engineering and Electronics, 2021, 43 (8): 2221- 2231.
2	王建峰, 贾高伟, 郭正, 等. 多无人机协同任务规划方法研究综述[J]. 系统工程与电子技术, 2024, 46 (10): 3437−3450.
	WANG J F, JIA G W, GUO Z, Research status and development of multi-UAV system mission planning [J]. Systems Engineering and Electronics, 2024, 46 (10): 3437−3450.
3	TAN Y F, ZHOU C, QIAN F. Cooperative task allocation method for multi-unmanned aerial vehicles based on the modified genetic algorithm[J]. IET Intelligent Transport Systems, 2024, 18 (6): 1164- 1173. doi: 10.1049/itr2.12495
4	GHAURI S A, SARFRAZ M, QAMAR R A, et al. A review of multi-UAV task allocation algorithms for a search and rescue scenario[J]. Journal of Sensor and Actuator Networks, 2024, 13 (5): 47. doi: 10.3390/jsan13050047
5	FAN B Y, BO Y M. Learning improvement heuristics for multi-unmanned aerial vehicle task allocation[J]. Drones, 2024, 8 (11): 636. doi: 10.3390/drones8110636
6	ZHANG J D, YANG Q M, SHI G Q, et al. UAV cooperative air combat maneuver decision based on multi-agent reinforcement learning[J]. Journal of Systems Engineering and Electronics, 2021, 32 (6): 1421- 1438. doi: 10.23919/JSEE.2021.000121
7	WANG J F, JIA G W, LIN J C, et al. Cooperative task allocation for heterogeneous multi-UAV using multi-objective optimization algorithm[J]. Journal of Central South University, 2020, 27 (2): 432- 448. doi: 10.1007/s11771-020-4307-0
8	WANG G, LV X, CUI L Z, et al. The methods of task pre-allocation and reallocation for multi-UAV cooperative reconnaissance mission[J]. IET Collaborative Intelligent Manufacturing, 2023, 5 (4)
9	熊韫文, 魏才盛, 许丹, 等. 基于改进蒙特卡洛树搜索的无人机目标分配与突防决策方法 [J]. 中南大学学报(自然科学版), 2023, 54(8): 3132−3144.
	XIONG Y W, WEI C S, XU D, et al. Multi-UVA target allocation and penetration decision based on an improved Monte Carlo tree search method [J]. Journal of Central South University (Science and Technology), 2023, 54(8): 3132−3144.
10	文永明, 石晓荣, 黄雪梅, 等. 一种无人机集群对抗多耦合任务智能决策方法 [J]. 宇航学报, 2021, 42(4): 504−512.
	WEN Y M, SHI X R, HUANG X M, et al. An intelligent decision-making method for multi-coupling tasks of UAV cluster countermeasure [J]. Journal of Astronautics, 2021, 42(4): 504−512.
11	MAO X, WU G H, FAN M F, et al. DL-DRL: A double-level deep reinforcement learning approach for large-scale task scheduling of multi-UAV[J]. IEEE Trans. on Automation Science and Engineering, 2024, 22, 1028- 1044.
12	CHEN W K, HAI X S, HU Y M, et al. Hierarchical decision-making framework for multi-UAV task assignment via enhanced pigeon-inspired optimization[J]. Guidance, Navigation and Control, 2023, 3 (4): 2350028.
13	LUONG P, GAGNON F, TRAN L N, et al. Deep reinforcement learning-based resource allocation in cooperative uav-assisted wireless networks[J]. IEEE Trans. on Wireless Communications, 2021, 20 (11): 7610- 7625. doi: 10.1109/TWC.2021.3086503
14	LIANG Z B, LI Q, FU G D. Multi-UAV collaborative search and attack mission decision-making in unknown environments[J]. Sensors, 2023, 23 (17) doi: 10.1109/JSEN.2023.3306185
15	邹子缘, 陈琪锋. 基于决策树搜索的空间飞行器集群对抗目标分配方法[J]. 航空学报, 2022, 43 (S1): 78- 88.
	ZOU Z Y, CHEN Q F. Decision tree-based target assignment for confrontation of multiple space vehicles[J]. Acta Aeronautica et Astronautica Sinica, 2022, 43 (S1): 78- 88.
16	郑红星, 郭继峰, 谢旭东, 等. 一种未知动态环境下异构无人机集群分布式联盟形成方法[J]. 宇航学报, 2022, 43 (2): 189- 197.
	ZHENG H X, GUO J F, XIE X D, et al. A distributed coalition formation method of heterogeneous UAV swarm in unknown dynamic environment[J]. Journal of Astronautics, 2022, 43 (2): 189- 197.
17	BROWNE C B, POWLEY E, WHITEHOUSE D, et al. A survey of Monte Carlo tree search methods[J]. IEEE Trans. on Computational Intelligence and AI in Games, 2012, 4 (1): 1- 43. doi: 10.1109/TCIAIG.2012.2186810
18	HOFFMAN M, SONG E, BRUNDAGE M P, et al. Online improvement of condition-based maintenance policy via Monte Carlo tree search[J]. IEEE Trans. on Automation Science and Engineering, 2022, 19 (3): 2540- 2551. doi: 10.1109/TASE.2021.3088603
19	LIU Z C, HAO M R. Heterogeneous unmanned aerial vehicles cooperative search approach for complex environments[J]. Engineering Applications of Artificial Intelligence, 2024, 138, 109384. doi: 10.1016/j.engappai.2024.109384
20	CHEN T Y, MIAO Z H, LI W M, et al. A learning-based memetic algorithm for a cooperative task allocation problem of multiple unmanned aerial vehicles in smart agriculture[J]. Swarm and Evolutionary Computation, 2024, 91, 101694. doi: 10.1016/j.swevo.2024.101694
21	CHOUR K, PRADEEP P, MUNISHKIN A A, et al. Aerial vehicle routing and scheduling for UAS traffic management: a hybrid Monte Carlo tree search approach [C]// Proc. of the IEEE/AIAA 42nd Digital avionics Systems Conference, 2023.
22	LI Y Y, CHEN W Y, LIU S K, et al. Multi-UAV cooperative air combat target assignment method based on VNS-IBPSO in complex dynamic environment[J]. International Journal oF Aerospace Engineering, 2024 (1): 9980746.
23	张嘉辉, 蒙志君, 何家政, 等. 基于改进能量谷优化的多无人机空战目标分配[J]. 系统工程与电子技术, 2024, 46 (11): 3754- 3763.
	ZHANG J H, MENG Z J, HE J Z, et al. Multi-UAV air combat target allocation based on improved energy valley optimization[J]. Systems Engineering and Electronics, 2024, 46 (11): 3754- 3763.
24	XING H X, XING Q H, WANG K. A joint allocation method of multi-jammer cooperative jamming resources based on suppression effectiveness[J]. Mathematics, 2023, 11 (4): 1- 25.
25	LU D J, WANG X, WU X T, et al. Adaptive allocation strategy for cooperatively jamming netted radar system based on improved cuckoo search algorithm[J]. Defence Technology, 2023, 24, 285- 297. doi: 10.1016/j.dt.2022.04.013
26	ZHANG D L, SUN J, YI W, et al. Joint jamming beam and power scheduling for suppressing netted radar system [C]// Proc. of the IEEE Radar Conference, 2021.
27	李世忠, 王国宏, 吴巍, 等. 分布式干扰下组网雷达目标检测与跟踪技术[J]. 系统工程与电子技术, 2012, 34(4): 782−788.
	LI S Z, WANG G H, WU W, et al. Detect and tracking technique for radar network in the presence of distributed jamming. Systems Engineering and Electronics, 2012, 34(4): 782−788.
28	陆德江, 王星, 陈游, 等. 联合多种资源协同干扰组网雷达系统的自适应调度方法[J]. 系统工程与电子技术, 2023, 45(9): 2744−2754.
	LU D J, WANG X, CHEN Y, et al. Adaptive scheduling method of joint multi-resource for cooperative interference of networked radar system, 2023, 45(9): 2744−2754.
29	WANG C, JIU B, PU W Q, et al. Anti-jamming equilibrium strategy learning of frequency agile radar based on Monte Carlo tree search [C]// Proc. of the IEEE International Radar Conference, 2023.
30	, et al. Model checking for probabilistic multiagent systems[J]. Journal of Computer Science and Technology, 2023, 38 (5): 1162- 1186. doi: 10.1007/s11390-022-1218-6
31	罗俊仁, 张万鹏, 陆丽娜, 等. 即时策略博弈在线对抗规划方法综述[J]. 计算机科学, 2022, 49(6): 287−296.
	LUO J R, ZHANG W P, LU L N, et al. Survey on online adversarial planning for real-time strategy game[J]. Computer Science, 2022, 49(6): 287−296.
32	CHENG X T, MAGHSUDI S. Distributed consensus algorithm for decision-making in multi-agent multi-armed bandit[J]. IEEE Trans. on Control of Network Systems, 2024, E107A(5): 799−805.

对抗双方	数量	动作
红方多无人机	${N_U}$	探测
		干扰
		侦察
蓝方多雷达	${N_{{\mathrm{tar}}}}$	搜索
		确认
		跟踪
		识别
		干扰

影响因素	状态有效概率
搜索	${P_{{\mathrm{b}}\_{\mathrm{s}}}}$
确认	${P_{{\mathrm{b}}\_{\mathrm{c}}}}$
跟踪	${P_{{\mathrm{b}}\_{\mathrm{t}}}}$
识别	${P_{{\mathrm{b}}\_{\mathrm{r}}}}$
干扰	${P_{{\mathrm{b}}\_{\mathrm{j}}}}$

决策动作	动作执行成功概率
探测	${P_{{\mathrm{r}}\_{\mathrm{d}}}}$
干扰	${P_{{\mathrm{r}}\_{\mathrm{j}}}}$
侦察	${P_{{\mathrm{r}}\_{\mathrm{r}}}}$

序号	类型	雷达状态	X轴位置/km	Y轴位置/km
1	1	2	−496.07	13.92
2	1	3	−497.57	32.41
3	1	3	−468.87	1.13
4	1	2	−469.67	20.31
5	1	−1	−469.56	40.44

序号	X轴位置/km	Y轴位置/km	Z轴位置/km	速度/(km/s)
1	−594.51	11.53	31.46	1.60
2	−594.44	21.61	31.56	1.60
3	−595.13	30.93	30.86	1.60

Multi-UAV multi-mission joint decision making based on improved MCTS

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 26

References 32

Related Articles 15

Recommended Articles

Metrics

Comments

参数	数值	参数	数值
发射功率${P_{r,i}}$/W	600×10⁶	虚警概率${P_{{{fa}}}}$	1e-6
发射天线增益${G_{r,i}}$/dB	16	工作波长${\lambda _r}$/m	0.2
天线波瓣宽度${\theta _{0.5}}$/（°）	2	系统损耗${L_{r,i}}$/dB	3
非相干累计脉冲数I	12	脉冲宽度$ \tau $/us	10

蓝方雷达整体威胁程度	红方无人机决策方法
蓝方雷达整体威胁程度	MCTS_AT算法	多臂赌博机算法
0	0.75	0.75
1	0.56	0.59

[1]	Zaiqiang HAO, Zhao YANG, Tianqi YANG. Multi-missile cooperative layered interception guidance scheme based on target assignment [J]. Systems Engineering and Electronics, 2025, 47(5): 1671-1679.
[2]	Mengjie LI, Xuening CHANG, Jianmai SHI, Chao CHEN, Jincai HUANG, Zhong LIU. Developments of weapon target assignment: models, algorithms, and applications [J]. Systems Engineering and Electronics, 2023, 45(4): 1049-1071.
[3]	Wenfei ZHAO, Xiaolei LIU, Cuiling MA, Kenan TENG. DWTA of air defense for strategic location on the sea based on multi-objective fuzzy programming [J]. Systems Engineering and Electronics, 2023, 45(3): 777-784.
[4]	Xuening CHANG, Jianmai SHI, Chao CHEN, Jincai HUANG. Multi-stage weapon target assignment method based on Hungarian simulated annealing algorithms [J]. Systems Engineering and Electronics, 2023, 45(11): 3516-3523.
[5]	Chen DONG, Yixian SHUAI, Jinpeng ZHOU, Peng LAI, Xianlei CHENG. Cooperative air defense task planning of networked multi-sensor-multi-weapon [J]. Systems Engineering and Electronics, 2022, 44(12): 3738-3746.
[6]	Wenhai WU, Xiaofeng GUO, Siyu ZHOU, Li GAO. Improved differential evolution algorithm for solving weapon-targetassignment problem [J]. Systems Engineering and Electronics, 2021, 43(4): 1012-1021.
[7]	Jiayi LIU, Gang WANG, Jie ZHANG, Chuang WANG, Xituan SONG. Target optimal assignment model based on improved AGD-distributed multi-Agent system [J]. Systems Engineering and Electronics, 2020, 42(4): 863-870.
[8]	WU Yong, LIN Wenxiang, JIAO Jingtao, ZHANG Jiandong. Algorithm of target assignment of isomorphic resources based on induction [J]. Systems Engineering and Electronics, 2019, 41(2): 342-348.
[9]	CHU Xiaogeng, MA Zhengwei, CHEN Xingjun. Look-ahead margin-greedy constructive algorithm for the multi-objective optimization of the weapon target assignment problem [J]. Systems Engineering and Electronics, 2019, 41(10): 2252-2259.
[10]	LIU Qingguo, LIU Xinxue, XIA Wei, GUO Huijun. Target assignment optimization of multiple on-orbit weapon platforms with multiple warheads [J]. Systems Engineering and Electronics, 2018, 40(5): 1050-1056.
[11]	XU Hao, XING Qinghua, WANG Wei. WTA for air and missile defense based on fuzzy multi-objective programming [J]. Systems Engineering and Electronics, 2018, 40(3): 563-570.
[12]	OUYANG Zhihong, XUE Lei, DING Feng. Jamming target assignment method of regional electronic air defense against electrooptical precision guided weapon#br# [J]. Systems Engineering and Electronics, 2018, 40(12): 2621-2628.
[13]	XIE Junjie, LUO Pengcheng, MU Fuling, WANG Jun, DING Shuai. Air combat target assignment in ABMS based on Q-learning algorithm [J]. Systems Engineering and Electronics, 2017, 39(3): 557-561.
[14]	WANG Yi, SUN Jin-biao, XIAO Ming-qing, LUO Ji-xun. Research of dynamic weapon-target assignment problem based on type-2 interval fuzzy K-nearest neighbors classifier [J]. Systems Engineering and Electronics, 2016, 38(6): 1314-1319.
[15]	CHEN Jun-wei, CHANG Tian-qing, ZHANG Lei, YANG Guo-zhen, MA Dian-zhe. Two-stage model of WTA oriented armored unit combat method [J]. Systems Engineering and Electronics, 2016, 38(6): 1326-1331.

序号	当前时刻无人机序号
1	1
2	2
3	3
4	1
5	2

参数	数值	参数	数值
距离测量误差最大值$ \Delta r $/km	0.10	速度测量误差最大值$\Delta v$/（km/s）	0.02
角度测量误差最大值$ \Delta \theta $/（°）	0.80	MCTS次数	500
$ {\zeta _2} $	0.55	$ {\alpha _i} $	1/3
探索因子c	$\sqrt 2 $	$ {\zeta _1} $	0.45