基于深度分层强化学习的防空作战武器目标分配算法

doi:10.12305/j.issn.1001-506X.2026.10.23

系统工程与电子技术 ›› 2026, Vol. 48 ›› Issue (10): 3531-3540.doi: 10.12305/j.issn.1001-506X.2026.10.23

• 系统工程 • 上一篇

基于深度分层强化学习的防空作战武器目标分配算法

曹波¹, 邢清华², 刘家义³, 李龙跃²

1. 空军工程大学研究生院，陕西西安 710051
2. 空军工程大学防空反导学院，陕西西安 710051
3. 国防大学联合作战学院，河北石家庄 050084

收稿日期:2025-07-14 出版日期:2026-10-25 发布日期:2026-09-30
通讯作者: 李龙跃
作者简介:曹　波（1998—），男，博士研究生，主要研究方向为深度学习与智能优化算法、态势认知
邢清华（1966—），女，教授，博士，主要研究方向为智能作战指挥决策优化理论与方法
刘家义（1995—），男，讲师，博士，主要研究方向为深度学习与智能优化算法
基金资助:
国家自然科学基金（72071209）资助课题

Air defense operations weapon target assignment algorithm based on deep hierarchical reinforcement learning

Bo Cao¹, Qinghua Xing², Jiayi Liu³, Longyue Li²

1. Graduate School，Air Force Engineering University，Xi’an 710051，China
2. Air Defense and Antimissile School，Air Force Engineering University，Xi’an 710051，China
3. Joint Operations College，National Defense University，Shijiazhuang 050084，China

Received:2025-07-14 Online:2026-10-25 Published:2026-09-30
Contact: Longyue Li

摘要/Abstract

摘要：

针对传统求解方法在大规模作战场景中面临维数灾难和实时性挑战等问题，提出一种基于深度分层强化学习的防空作战武器目标分配算法。首先构建以最小化蓝方预期生存威胁和红方弹药消耗为目标的动态分配模型，之后将目标分配任务分解为战略层任务匹配与战术层目标拦截，分别采用近端策略优化（proximal policy optimization, PPO）算法和融合注意力机制的多智能体PPO算法求解，并引入三阶段课程学习机制提升模型泛化能力。实验结果表明，所提方法在效费比和收敛效率上优于传统强化学习算法和启发式算法，为提升防空作战决策效率提供了有效支撑。

关键词: 防空作战目标分配, 深度分层强化学习, 近端策略优化算法, 课程学习

Abstract:

A deep hierarchical reinforcement learning based air defense weapon target assignment algorithm is proposed to address the challenges of dimensionality disaster and real-time performance faced by traditional solving methods in large-scale combat scenarios. Firstly, a dynamic assignment model is constructed with the goal of minimizing the expected survival threat of the blue side and the ammunition consumption of the red side. Then, the target assignment task is decomposed into strategic level task matching and tactical level target interception, which are solved using the proximal policy optimization （PPO） algorithm and the multi-agent PPO algorithm incorporating attention mechanism, respectively. A three-stage curriculum learning mechanism is introduced to improve the generalization ability of the model. The experimental results show that the proposed method is superior to traditional reinforcement learning algorithms and heuristic algorithms in terms of cost-effectiveness and convergence efficiency, providing effective support for improving the decision-making efficiency of air defense operations.

Key words: target assignment for air defense operations, deep hierarchical reinforcement learning (DHRL), proximal policy optimization (PPO) algorithm, curriculum learning

中图分类号:

E 917

曹波, 邢清华, 刘家义, 李龙跃. 基于深度分层强化学习的防空作战武器目标分配算法[J]. 系统工程与电子技术, 2026, 48(10): 3531-3540.

Bo Cao, Qinghua Xing, Jiayi Liu, Longyue Li. Air defense operations weapon target assignment algorithm based on deep hierarchical reinforcement learning[J]. Systems Engineering and Electronics, 2026, 48(10): 3531-3540.

图/表 10

图1

图2

图3

表1

表2

图4

图5

表3

表4

图6

参考文献 29

1	Cao B, Song Y F, Wang G. Technology for intelligent battlefield situation awareness[J]. Chinese Journal of Aeronautics, 2025, 38 (6): 103371. doi: 10.1016/j.cja.2024.103371
2	Li J R, Wu G H, Wang L. A comprehensive survey of weapon target assignment problem: model, algorithm, and application[J]. Engineering Applications of Artificial Intelligence, 2024, 137, 109212. doi: 10.1016/j.engappai.2024.109212
3	刘富樯, 周伦, 刘中阳, 等. 基于三支决策和遗传算法的动态武器目标分配[J]. 兵工学报, 2025, 46 (3): 257. doi: 10.12382/bgxb.2024.0281
4	赵国亮, 穆培培, 李泽文. 求解静态武器-目标分配问题的混合优化算法[C]//第13届中国指挥控制大会, 2025: 74.
5	王浩博, 刘建涛. 基于改进模拟退火算法的防空武器集群打击目标动态分配[J]. 计算机测量与控制, 2024, 32 (7): 196. doi: 10.16526/j.cnki.11-4762/tp.2024.07.029
6	Arslan C, Karasakal O, Kirca O. Naval air defense planning problem: a novel formulation and heuristics[J]. Naval Research Logistics, 2024, 71 (7): 895. doi: 10.1002/nav.22186
7	Bai Z Z, Zhou H Y, Shi J M, et al. A hybrid multi-objective evolutionary algorithm with high solving efficiency for UAV defense programming[J]. Swarm and Evolutionary Computation, 2024, 87, 101572. doi: 10.1016/j.swevo.2024.101572
8	张嘉辉, 蒙志君, 何家政, 等. 基于改进能量谷优化的多无人机空战目标分配[J]. 系统工程与电子技术, 2024, 46 (11): 3754. doi: 10.12305/j.issn.1001-506X.2024.11.17
9	Zhao Y, Liu J C, Jiang J, et al. Shuffled frog leaping algorithm with non-dominated sorting for dynamic weapon-target assignment[J]. Journal of Systems Engineering and Electronics, 2023, 34 (4): 1007. doi: 10.23919/JSEE.2023.000102
10	Chang X N, Shi J M, Luo Z H, et al. Adaptive large neighborhood search algorithm for multi-stage weapon target assignment problem[J]. Computers & Industrial Engineering, 2023, 181, 109303.
11	Tunga H, Kar S, Giri D, et al. Efficacy analysis of NSGAII and multi-objective particle swarm optimization （MOPSO） in agent based weapon target assignment （WTA） model[J]. International Journal of Information Technology, 2024, 16 (3): 1347. doi: 10.1007/s41870-023-01674-0
12	Li T D, Wang G, Fu Q, et al. An intelligent algorithm for solving weapon-target assignment problem: DDPG-DNPE algorithm[J]. Computers, Materials & Continua, 2023, 76(3): 3499.
13	Wang X C, Zhang Y, Wang G. Target assignment for multiple stages of weapons systems using a deep Q-learning network and a modified artificial bee colony method[J]. Computers and Electrical Engineering, 2024, 118, 109378. doi: 10.1016/j.compeleceng.2024.109378
14	林雕, 朱燕, 汤奋. 一种面向动态场景的要地武器目标分配方法[J]. 军事运筹与评估, 2024, 39 (3): 68. doi: 10.19949/j.ams.mora.20231107.01
15	李腾达, 王刚, 付强, 等. 基于深度强化学习的防空武器目标分配问题研究[J]. 军事运筹与评估, 2024, 39 (2): 63. doi: 10.19949/j.ams.mora.20230216.02
16	李乔易, 王正杰, 张小宁, 等. 基于深度确定性梯度学习的集群多目标分配方法[J]. 北京理工大学学报, 2024, 44 (10): 1051. doi: 10.15918/j.tbit1001-0645.2023.200
17	Li X Y, Wang T, Wang D H, et al. Intelligent decision-making algorithm for airborne phased array radar search tasks based on a hierarchical strategy framework[J]. Chinese Journal of Aeronautics, 2024, 37 (11): 398. doi: 10.1016/j.cja.2024.09.006
18	Liu J Y, Wang G, Fu Q, et al. Task assignment in ground-to-air confrontation based on multiagent deep reinforcement learning[J]. Defence Technology, 2023, 19 (1): 210.
19	Wang Y D, Liang Y, Wang Z F. Hierarchical reinforcement-learning-based joint allocation of jamming task and power for countering networked radar[J]. Expert Systems with Applications, 2025, 61 (2): 2149.
20	Geng M H, Pateria S, Subagdja B, et al. HiSOMA: a hierarchical multi-agent model integrating self-organizing neural networks with multi-agent deep reinforcement learning[J]. Expert Systems with Applications, 2024, 252, 124117. doi: 10.1016/j.eswa.2024.124117
21	Peng Z, Lu Z F, Mao X, et al. Multi-ship dynamic weapon-target assignment via cooperative distributional reinforcement learning with dynamic reward[J]. IEEE Trans. on Emerging Topics in Computational Intelligence, 2025, 9 (1): 1843.
22	闫世祥, 刘海军. 基于深度强化学习的传感器-武器-目标分配方法[J]. 现代防御技术, 2025, 53 (4): 10.
23	Wu Q Z, Liu K X, Chen L, et al. Hierarchical reinforcement learning for swarm confrontation with high uncertainty[J]. IEEE Trans. on Automation Science and Engineering, 2025, 22, 8630. doi: 10.1109/TASE.2024.3487219
24	Yin S H, Xiang Z R. A hyper-heuristic algorithm via proximal policy optimization for multi-objective truss problems[J]. Expert Systems with Applications, 2024, 256 (12): 124929.
25	简泽民. 基于MAPPO的防空部署和武器目标分配方法研究[D]. 贵阳: 贵州大学, 2025.
26	Chaudhry S, Sharma A. Dynamic data distribution-based curriculum learning[J]. Information Sciences, 2025, 702, 121924. doi: 10.1016/j.ins.2025.121924
27	West J, Maire F, Browne C, et al. Improved reinforcement learning with curriculum[J]. Expert Systems with Applications, 2020, 158, 113515. doi: 10.1016/j.eswa.2020.113515
28	孙世彬, 王庆领. 基于改进多智能体强化学习的大规模无人机集群博弈对抗[J]. 海军航空大学学报, 2025, 40 (4): 528. doi: 10.7682/j.issn.2097-1427.2025.04.003
29	王星. 面向无人机集群对抗的多智能体深度强化学习策略研究[D]. 西安: 西安工业大学, 2025.

超参数	数值
Actor网络学习率	0.0001
Critic网络学习率	0.0001
Actor网络优化器	Adam
Critic网络优化器	Adam
隐藏层单元数	128/256
Batch Size	64
截断参数	0.2
经验回放池大小	1000000
折扣系数	0.99

算法名称	含义
DHRL-DWTA	本文所提算法
PPO-MAPPO-Attention	在DHRL-DWTA模型基础上去掉CL机制
PPO-MAPPO	在DHRL-DWTA模型基础上去掉CL机制和注意力机制
MAPPO	将红方每个拦截单元看作一个智能体，通过MAPPO算法求解
MADDPG^[29]	将红方每个拦截单元看作一个智能体，通过MADDPG算法求解

编组序号	有人机数量	无人机数量	威胁值
1	3	5	[0.7, 0.7, 0.8, 0.9, 0.9, 0.9, 0.9, 0.9]
2	4	4	[0.7, 0.7, 0.8, 0.9, 0.9, 0.9, 0.9, 0.9]
3	4	8	[0.7, 0.7, 0.7, 0.7, 0.8, 0.8, 0.9, 0.9, 0.9, 0.9, 0.9, 0.9]
4	5	3	[0.7, 0.7, 0.7, 0.8, 0.8, 0.9, 0.9, 0.9]
5	4	4	[0.7, 0.7, 0.7, 0.8, 0.9, 0.9, 0.9, 0.9]
6	6	6	[0.7, 0.7, 0.7, 0.7, 0.8, 0.8, 0.9, 0.9, 0.9, 0.9, 0.9, 0.9]
7	5	7	[0.7, 0.7, 0.7, 0.7, 0.8, 0.9, 0.9, 0.9, 0.9, 0.9, 0.9, 0.9]
8	4	8	[0.7, 0.7, 0.7, 0.7, 0.8, 0.9, 0.9, 0.9, 0.9, 0.9, 0.9, 0.9]
9	2	6	[0.7, 0.7, 0.8, 0.9, 0.9, 0.9, 0.9, 0.9]
10	3	9	[0.7, 0.7, 0.7, 0.7, 0.8, 0.8, 0.9, 0.9, 0.9, 0.9, 0.9, 0.9]

拦截单元序号	成本	对有人机毁伤概率	对无人机毁伤概率	导弹数量
1	1.5	0.45	0.8	32
2	2	0.55	0.85	32
3	1.5	0.45	0.8	32
4	2	0.55	0.85	32
5	2	0.5	0.85	32
6	1	0.45	0.7	32
7	0.8	0.45	0.55	32
8	1.5	0.45	0.8	32
9	2	0.55	0.85	32
10	1	0.45	0.7	32

[1]	杨志鹏, 陈子浩, 曾长, 林松, 毛金娣, 张凯. 复杂环境下的飞行器在线航路规划决策方法[J]. 系统工程与电子技术, 2024, 46(9): 3166-3175.
[2]	秦湖程, 黄炎焱, 陈天德, 张寒. 基于PPO算法的集群多目标火力规划方法[J]. 系统工程与电子技术, 2024, 46(11): 3764-3773.

基于深度分层强化学习的防空作战武器目标分配算法

Air defense operations weapon target assignment algorithm based on deep hierarchical reinforcement learning

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献 29

相关文章 2

编辑推荐

Metrics

本文评价