面向有人/无人机协同打击的智能决策方法研究

doi:10.12305/j.issn.1001-506X.2025.04.25

Abstract

Abstract:

The trend in unmanned aerial vehicle air combat is the coordination between manned aerial vehicles and unmanned aerial vehicles, with intelligent decision-making being crucial for achieving coordinated attack between manned aerial vehicles and unmanned aerial vehicles. High dynamic battlefield environment, asymmetric combat tasks and heterogeneous multi-source coordination system lead to poor autonomous capability and real-time performance of unmanned aerial vhicles, and difficult strategic training, which is the difficulty of manned aerial vehicles and unmanned aerial vehicles cooperative attack research. A typical style of manned aerial vehicles and unmanned aerial vehicles cooperative attack pattern is designed based on the loyal wingman scheme of cooperative maneuvering of manned aerial vehicles and unmanned aerial vehicles. A reinforcement learning method based on improved multi-agent twin delayed deep deterministic (MATD3) policy gradient algorithm is proposed. Firstly, the cooperative maneuvering decision-making training framework based on MATD3 policy gradient algorithm, curriculum learning (CL) and the pre-train (PT) strategy based on transfer learning are designed to solve the difficult problem of the cooperative attack strategy training of manned aerial vehicles and unmanned aerial vehicles. Secondly, the reward function and state space for unmanned aerial vehicles cooperative maneuvers are established to facilitate multi-aerial coordinated operations. Finally, a digital simulation and deduction platform is built based on the six-degree-freedom simulation model to verify that the trained attack strategy has efficient attack and survivability and can guide the practical application of manned aerial vehicles and unmanned aerial vehicles coordinated arrack operations.

Key words: manned aerial vehicles and unmanned aerial vehicles coordination, air combat maneuver decision, deep reinforcement learning, loyal wingman

CLC Number:

TP181

Wei XIONG, Dong ZHANG, Zhi REN, Shuheng YANG. Research on intelligent decision-making methods for coordinated attack by manned aerial vehicles and unmanned aerial vehicles[J]. Systems Engineering and Electronics, 2025, 47(4): 1285-1299.

Figures/Tables 31

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Table 1

Fig.7

Table 2

Table 3

Fig.8

Fig.9

Fig.10

Fig.11

Fig.12

Table 4

Fig.13

Fig.14

Fig.15

Fig.16

Fig.17

Fig.18

Fig.19

Fig.20

Fig.21

Fig.22

Fig.23

Fig.24

Fig.25

Fig.26

Fig.27

References 33

1	丁达理, 谢磊, 王渊. 有人机/无人机协同作战运用及对战争形态影响[J]. 无人系统技术, 2020, 3 (4): 1- 9.
	DING D L , XIE L , WANG Y . The application of manned/unmanned aerial vehicle cooperative combat and its influence on war form[J]. Unmanned Systems Technology, 2020, 3 (4): 1- 9.
2	United States Department of Defense. Unmanned systems integrated roadmap FY2013-2038[R]. Washington, DC: United States Department of Defense, 2013.
3	贾高伟, 侯中喜. 美军有/无人机协同作战研究现状与分析[J]. 国防科技, 2017, 38 (6): 57- 59.
	JIA G W , HOU Z X . The analysis and current situation about the United States military manned/unmanned aerial vehicle[J]. National Defense Science and Technology, 2017, 38 (6): 57- 59.
4	王新尧, 曹云峰, 孙厚俊, 等. 基于DoDAF的有人/无人机协同作战体系结构建模[J]. 系统工程与电子技术, 2020, 42 (10): 2265- 2274. doi: 10.3969/j.issn.1001-506X.2020.10.15
	WANG X Y , CAO Y F , SUN H J , et al. Modeling for cooperative combat system architecture of manned/unmanned aerial vehicle based on DoDAF[J]. Systems Engineering and Electronics, 2020, 42 (10): 2265- 2274. doi: 10.3969/j.issn.1001-506X.2020.10.15
5	张路, 邓静, 邵正途. 俄军有人机与无人机协同作战分析及启示[J]. 舰船电子对抗, 2022, 45 (3): 1- 6.
	ZHANG L , DENG J , SHAO Z T . Analysis and enlightenment of manned and unmanned aerial vehicles cooperative operation for Russian army[J]. Shipboard Electronic Countermeasure, 2022, 45 (3): 1- 6.
6	李樾, 韩维, 陈清阳, 等. 凸优化算法在有人/无人机协同系统航迹规划中的应用[J]. 宇航学报, 2020, 41 (3): 276- 286.
	LI Y , HAN W , CHEN Q Y , et al. Application of convex optimization algorithm in trajectory planning of manned/unmanned cooperative system[J]. Journal of Astronautics, 2020, 41 (3): 276- 286.
7	XING D J , ZHEN Z Y , GONG H J . Offense-defense confrontation decision making for dynamic UAV swarm versus UAV swarm[J]. Proceedings of the Institution of Mechanical Engineers, 2019, 233 (15): 5689- 5702. doi: 10.1177/0954410019853982
8	BAPNAR B , KOYUNCU E . Assessment of aerial combat game via optimization-based receding horizon control[J]. IEEE Access, 2020, 8, 35853- 35863. doi: 10.1109/ACCESS.2020.2974792
9	胡利平, 梁晓龙, 何吕龙, 等. 基于情景分析的航空集群决策规则库构建方法[J]. 航空学报, 2020, 41 (S1): 723737.
	HU L P , LIANG X L , HE L L , et al. Construction method of aviation swarm decision rule base based on scenario analysis[J]. Acta Aeronautica et Astronautica Sinica, 2020, 41 (S1): 723737.
10	YANG Q M, ZHU Y, ZHANG J D, et al. UAV air combat autonomous maneuver decision based on DDPG algorithm[C]//Proc. of the IEEE 15th International Conference on Control and Automation, 2019: 37-42.
11	MA Y Y , WANG G Q , HU X X , et al. Cooperative occupancy decision making of multi-UAV in beyond-visual-range air combat: a game theory approach[J]. IEEE Access, 2020, 8, 11624- 11634. doi: 10.1109/ACCESS.2019.2933022
12	XU J W , DENG Z H , SONG Q , et al. Multi-UAV counter-game model based on uncertain information[J]. Applied Mathe-matic and Computation, 2020, 366, 124684. doi: 10.1016/j.amc.2019.124684
13	YANG Q M , ZHANG J D , SHI G Q , et al. Maneuver decision of UAV in short-range air combat based on deep reinforcement learning[J]. IEEE Access, 2020, 8, 363- 378. doi: 10.1109/ACCESS.2019.2961426
14	TANG R Z, ZHOU Z M, ZHANG C L, et al. The applications of artificial intelligence in situation assessment and game countermeasure during unmanned air combat[C]//Proc. of the IEEE International Conference on Unmanned Systems, 2019: 909-913.
15	KANESHIGE J, KRISHNAKUMAR K. Artificial immune system approach for air combat maneuvering[C]//Proc. of the Intelligent Computing: Theory and Applications, 2008: 68-79.
16	ASLAN S , ERKIN T . A multi-population immune plasma al gorithm for path planning of unmanned combat aerial vehicle[J]. Advanced Engineering Informatics, 2023, 55 (C): 101829.
17	LI S Y , CHEN M , WANG Y H , et al. Air combat decision- making of multiple UCAVs based on constraint strategy games[J]. Defence Technology, 2022, 18 (3): 368- 383. doi: 10.1016/j.dt.2021.01.005
18	RUAN W Y , DUAN H B , DENG Y M . Autonomous maneuver decisions via transfer learning pigeon-inspired optimization for UCAVs in dogfight engagements[J]. IEEE-CAA Journal of Automatica Sinica, 2022, 9 (9): 1639- 1657. doi: 10.1109/JAS.2022.105803
19	LI S Y , CHEN M , WANG Y H , et al. A fast algorithm to solve large-scale matrix games based on dimensionality reduction and its application in multiple unmanned combat air vehicles attack-defense decision-making[J]. Information Sciences, 2022, 594, 305- 321. doi: 10.1016/j.ins.2022.02.025
20	GENG W X, KONG F E, MA D Q. Study on tactical decision of UAV medium range air combat[C]//Proc. of the 26th Chinese Control and Decision Conference, 2014: 135-139.
21	HE X M, ZU W, CHANG H X, et al. Autonomous maneuvering decision research of UAV based on experience knowledge representation[C]//Proc. of the 28th Chinese Control and Decision Conference, 2016: 161-166.
22	KAUFMANN E , BAUERSFELD L , LOQUERCIO A , et al. Champion-level drone racing using deep reinforcement learning[J]. Nature, 2023, 620 (7976): 982- 987. doi: 10.1038/s41586-023-06419-4
23	AKROUR R , TATEO D , PETERS J . Continuous action reinforcement learning from a mixture of interpretable experts[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2022, 44 (10): 6795- 6806. doi: 10.1109/TPAMI.2021.3103132
24	LI B , HUANG J Y , BAI S X , et al. Autonomous air combat decision-making of UAV based on parallel self-play reinforcement learning[J]. CAAI Trans.on Intelligence Technology, 2023, 8 (1): 64- 81. doi: 10.1049/cit2.12109
25	LI Y F , SHI J P , JIANG W , et al. Autonomous maneuver decision-making for a UCAV in short-range aerial combat based on an MS-DDQN algorithm[J]. Defence Technology, 2022, 8 (9): 1697- 1714.
26	TIAN Z K, CHEN R Z, LI L, et al. Decompose a task into generalizable subtasks in multi-agent reinforcement learning [C]//Proc. of the Advances in Neural Information Processing Systems, 2024: 78514-78532.
27	ZHAN G , ZHANG X M , LI Z C , et al. Multiple-UAV reinforcement learning algorithm based on improved PPO in ray framework[J]. Drones, 2022, 6 (7): 166. doi: 10.3390/drones6070166
28	谭目来, 丁达理, 谢磊, 等. 基于模糊专家系统与IDE算法的UCAV逃逸机动决策[J]. 系统工程与电子技术, 2022, 44 (6): 1984- 1993. doi: 10.12305/j.issn.1001-506X.2022.06.26
	TAN M L , DING D L , XIE L , et al. UCAV escape maneuvering decision based on fuzzy expert system and IDE algorithm[J]. Systems Engineering and Electronics, 2022, 44 (6): 1984- 1993. doi: 10.12305/j.issn.1001-506X.2022.06.26
29	FUJIMOTO S, HOOF H V, MEGER D. Addressing function approximation error in actor-critic methods[C]//Proc. of the 35th International Conference on Machine Learning, 2018: 1587-1596.
30	CHEN P C , LIU S C , WANG X Z , et al. Physics-shielded multi-agent deep reinforcement learning for safe active voltage control with photovoltaic/battery energy storage systems[J]. IEEE Trans.on Smart Grid, 2023, 14 (4): 2656- 2667. doi: 10.1109/TSG.2022.3228636
31	ZHAO T T , LI F , HE L J , et al. DRL-based secure aggregation and resource orchestration in MEC-enabled hierarchical federated learning[J]. IEEE Internet of Things Journal, 2023, 10 (20): 17865- 17880. doi: 10.1109/JIOT.2023.3277553
32	杨书恒, 张栋, 熊威, 等. 基于可解释性强化学习的空战机动决策方法[J]. 航空学报, 2024, 45 (18): 252- 269.
	YANG S H , ZHANG D , XIONG W , et al. A decision-making method for air combat maneuver based on explainable reinforcement learning[J]. Acta Aeronautica et Astronautica Sinica, 2024, 45 (18): 252- 269.
33	AUSTIN F , CARBONE G , FALCO M , et al. Game theory for automated maneuvering during air-to-air combat[J]. Journal of Guidance Control and Dynamics, 1990, 13 (6): 1143- 1149.

状态	定义	状态	定义
s₁	x_r₀/x_max	s₂	x_r₁/x_max
s₃	x_r₂/x_max	s₄	y_r₀/y_max
s₅	y_r₁/y_max	s₆	y_r₂/y_max
s₇	h_r₀/h_max	s₈	h_r₁/h_max
s₉	h_r₂/h_max	s₁₀	x_b/x_max
s₁₁	y_b/y_max	s₁₂	h_b/h_max
s₁₃	\|d_r₀b\|/d_max	s₁₄	\|d_r₁b\|/d_max
s₁₅	\|d_r₂b\|/d_max	s₁₆	\|d_r₀r₁\|/d_max
s₁₇	\|d_r₀r₂\|/d_max	s₁₈	θ_d₀/π
s₁₉	θ_d₁/π	s₂₀	θ_d₂/π
s₂₁	θd₀₁/π	s₂₂	θ_d02/π
s₂₃	d₀/π	s₂₄	d₁/π
s₂₅	d₂/π	s₂₆	d₀₁/π
s₂₇	d₀₂/π	s₂₈	\|v_r₀\|/v_max
s₂₉	\|v_r₁\|/v_max	s₃₀	\|v_r₂\|/v_max
s₃₁	\|v_b\|/v_max	s₃₂	θ_r₀/π
s₃₃	θ_r₁/π	s₃₄	θ_r₂/π
s₃₅	θ_b/π	s₃₆	r₀/π
s₃₇	r₁/π	s₃₈	r₂/π
s₃₉	b/π	s₄₀	n_xr₀/n_xmax
s₄₁	n_xr₁/n_xmax	s₄₂	n_xr₂/n_xmax
s₄₃	n_yr₀/n_ymax	s₄₄	n_yr₁/n_ymax
s₄₅	n_yr₂/n_ymax	s₄₆	γ_r₀/π
s₄₇	γ_r₁/π	s₄₈	γ_r₂/π
s₄₉	o₁	s₅₀	o₂

参数	范围	参数	范围
n_x	[－1, 5]	θ	[－π/3, π/3]
n_y	[－1.5, 2]	φ_M	π/6
n_z	[－3, 3]	d_M/km	[1, 100]
v/(m/s)	[150, 400]	d_R/km	[0, 50]

参数	范围	参数	范围
n_x	[－1, 5]	θ	[－π/3, π/3]
n_y	[－1.5, 2]	φ_M	π/6
n_z	[－3, 3]	d_M/km	[1, 100]
v/(m/s)	[150, 400]	d_R/km	[0, 60]

参数	范围	参数	范围
n_x	[－1, 5]	θ	[－π/3, π/3]
n_y	[－1.5, 0.5]	φ_M	π/3
n_z	[－3, 3]	d_M/km	^{[1, 10]}
v/(m/s)	^{[150, 300]}	d_R/km	[0, 20]

[1]	Peng MA, Rui JIANG, Bin WANG, Mengfei XU, Changbo HOU. Strategy reconstruction for resilience against intelligence jamming based on implicit opponent modeling [J]. Systems Engineering and Electronics, 2025, 47(4): 1355-1363.
[2]	Kaiqiang TANG, Huiqiao FU, Jiasheng LIU, Guizhou DENG, Chunlin CHEN. Hierarchical optimization research of constrained vehicle routing based on deep reinforcement learning [J]. Systems Engineering and Electronics, 2025, 47(3): 827-841.
[3]	Xiarong CHEN, Jichao LI, Gang CHEN, Peng LIU, Jiang JIANG. Portfolio of weapon system-of-systems based on heterogeneous information networks [J]. Systems Engineering and Electronics, 2025, 47(3): 855-861.
[4]	Tingyu ZHANG, Ying ZENG, Nan LI, Hongzhong HUANG. Spacecraft power-signal composite network optimization algorithm based on DRL [J]. Systems Engineering and Electronics, 2024, 46(9): 3060-3069.
[5]	Yuqi XIA, Yanyan HUANG, Qia CHEN. Path planning for unmanned vehicle reconnaissance based on deep Q-network [J]. Systems Engineering and Electronics, 2024, 46(9): 3070-3081.
[6]	Zhipeng YANG, Zihao CHEN, Chang ZENG, Song LIN, Jindi MAO, Kai ZHANG. Online route planning decision-making method of aircraft in complex environment [J]. Systems Engineering and Electronics, 2024, 46(9): 3166-3175.
[7]	Hongda GUO, Jingtao LOU, Youchun XU, Peng YE, Yongle LI, Jinsheng CHEN. Event-triggered communication of multiple unmanned ground vehicles collaborative based on MADDPG [J]. Systems Engineering and Electronics, 2024, 46(7): 2525-2533.
[8]	Mengyu ZHANG, Yajie DOU, Ziyi CHEN, Jiang JIANG, Kewei YANG, Bingfeng GE. Review of deep reinforcement learning and its applications in military field [J]. Systems Engineering and Electronics, 2024, 46(4): 1297-1308.
[9]	Yanling LI, Feizhou LUO, Zhilei GE. Robust observer-based deep reinforcement learning for attitude stabilization of vertical takeoff and landing vehicle [J]. Systems Engineering and Electronics, 2024, 46(3): 1038-1047.
[10]	Fengguo WU, Wei TAO, Hui LI, Jianwei ZHANG, Chengchen ZHENG. UAV intelligent avoidance decisions based on deep reinforcement learning algorithm [J]. Systems Engineering and Electronics, 2023, 45(6): 1702-1711.
[11]	Jin TANG, Yangang LIANG, Zhihui BAI, Kebo LI. Landing control algorithm of rotor UAV based on DQN [J]. Systems Engineering and Electronics, 2023, 45(5): 1451-1460.
[12]	Siqi TANG, Zhisong PAN, Guyu HU, Yang WU, Yunbo LI. Application of deep reinforcement learning in space information network——status quo and prospects [J]. Systems Engineering and Electronics, 2023, 45(3): 886-901.
[13]	Guan WANG, Haizhong RU, Dali ZHANG, Guangcheng MA, Hongwei XIA. Design of intelligent control system for flexible hypersonic vehicle [J]. Systems Engineering and Electronics, 2022, 44(7): 2276-2285.
[14]	Lingyu MENG, Bingli GUO, Wen YANG, Xinwei ZHANG, Zuoqing ZHAO, Shanguo HUANG. Network routing optimization approach based on deep reinforcement learning [J]. Systems Engineering and Electronics, 2022, 44(7): 2311-2318.
[15]	Qingqing YANG, Yingying GAO, Yu GUO, Boyuan XIA, Kewei YANG. Target search path planning for naval battle field based on deep reinforcement learning [J]. Systems Engineering and Electronics, 2022, 44(11): 3486-3495.

Research on intelligent decision-making methods for coordinated attack by manned aerial vehicles and unmanned aerial vehicles

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 31

References 33

Related Articles 15

Recommended Articles

Metrics

Comments