深度强化学习驱动的多约束分数阶滑模制导律

doi:10.12305/j.issn.1001-506X.2026.09.29

摘要/Abstract

摘要：

针对高超声速拦截弹末端制导存在的攻击角、视场角等多约束难题，提出深度强化学习驱动的多约束分数阶滑模制导律。改进双延迟深度确定性策略梯度算法，引入多头注意力机制作为特征提取层，实现环境状态多尺度特征融合，利用长短期记忆网络增强时序特征理解，通过自适应探索与回合制训练优化框架提升算法收敛性与稳定性。基于弹目三维相对运动学模型，构建含攻击角约束的分数阶滑模基础制导律框架，通过改进强化学习算法动态优化制导参数，解决多约束下精确制导问题。仿真结果表明，所提制导律在满足多种角度约束条件下可显著提升制导速度与精度，验证了所提方法的正确性与有效性。

关键词: 制导律, 攻击角约束, 视场角约束, 分数阶滑模控制, 深度强化学习

Abstract:

A deep reinforcement learning-driven multi-constraint fractional order sliding mode guidance law is proposed to address the challenges of multi-constraint such as attack angle and field of view in the end guidance of hypersonic interception missiles. Dual delay depth deterministic strategy gradient algorithm is improved, multi-head attention mechanism is introduced as the feature extraction layer, multi-scale feature fusion of environmental states is achieved, long short-term memory network is utilized to enhance temporal feature understanding, and convergence and stability of the algorithm is improved through adaptive exploration and turn-based training optimization framework. Based on the three-dimensional relative kinematics model of the missile target, a fractional order sliding mode basic guidance law framework with attack angle constraints is constructed. By improving the reinforcement learning algorithm, the guidance parameters are dynamically optimized to solve the problem of precise guidance under multi-constraint. Simulation results show that the proposed guidance law can significantly improve the guidance speed and accuracy under various angle constraints. The correctness and effectiveness of the proposed method are verified.

Key words: guidance law, attack angle constraint, field of view constraint, fractional order sliding mode control, deep reinforcement learning

中图分类号:

V 488

张倬境, 盛永智, 龚孝龙. 深度强化学习驱动的多约束分数阶滑模制导律[J]. 系统工程与电子技术, 2026, 48(9): 3167-3178.

Zhuojing Zhang, Yongzhi Sheng, Xiaolong Gong. Deep reinforcement learning-driven multi-constraint fractional order sliding mode guidance law[J]. Systems Engineering and Electronics, 2026, 48(9): 3167-3178.

图/表 22

图1

图2

图3

图4

图5

表1

表2

图6

图7

表3

表4

图8

表5

图9

表6

图10

表7

图11

表8

图12

表9

图13

参考文献 35

1	Ryoo C K, Cho H J, Tank M J. Optimal guidance laws with terminal impact angle constraint[J]. Journal of Guidance, Control, and Dynamics, 2005, 28 (4): 724.
2	席杰, 杨军, 朱学平. 带落角和导引头视角约束的制导律设计[J]. 电光与控制, 2014, 21 (9): 26. doi: 10.3969/j.issn.1671-637X.2014.09.006
3	Han T, Zhang S X, Hu Q L, et al. Field-of-view reduced impact angle guidance law via range shaping approach[J]. IEEE Trans. on Aerospace and Electronic Systems, 2025, accepted.
4	熊少锋, 魏明英, 赵明元, 等. 逆轨拦截机动目标的三维最优制导律[J]. 宇航学报, 2020, 41 (1): 80.
5	吴盘龙, 王勇, 钟俊, 等. 拦截机动目标的多约束下协同制导律[J]. 中国惯性技术学报, 2023, 31 (11): 1142.
6	Xiong T H, Wang C Y, Zhang K, et al. Design of three-dimensional vector guidance law for attack time and angle under field of view angle constraints[J]. Aero Weaponry, 2024, 31 (4): 49.
7	王鹏, 陈万春, 陈中原. 视场约束下攻击角度及时间控制三维协同制导[J]. 战术导弹技术, 2022 (4): 30.
8	Kang H L, Wang P, Song S, et al. Prescribed-time three-dimensional guidance for maneuvering targets with field-of-view and approach angle constraints[J]. IEEE Trans. on Aerospace and Electronic Systems, 2025, 61 (3): 5996. doi: 10.1109/TAES.2024.3523884
9	Zhang L, Yi W J, Guan J. Nonsingular sliding mode guidance law for impact time control with field-of-view constraint[J]. IEEE Access, 2025, 13, 32004. doi: 10.1109/ACCESS.2025.3531042
10	Zhao B, Xiang T Y. Three-dimensional guidance law with terminal angle constraint based on multivariable super-twisting sliding mode[J]. Control and Decision, 2024, 39 (11): 3577.
11	Zhao Y, Sheng Y Z, Liu X D. Sliding modecontrol based guidance law with impact angle constraint[J]. Chinese Journal of Aeronautics, 2014, 27 (1): 145. doi: 10.1016/j.cja.2013.12.011
12	Hang Y, Dai K R, Li H J, et al. Three dimensional adaptive fixed-time cooperative guidance law with impact time and angle constraints.[J]. Aerospace Science and Technology, 2022, 123, 107450. doi: 10.1016/j.ast.2022.107450
13	Ming C, Wang X M, Sun R. A novel non-singular terminal sliding mode control based integrated missile guidance and control with impact angle constraint[J]. Aerospace Science and Technology, 2019, 94, 105368. doi: 10.1016/j.ast.2019.105368
14	唐骁, 叶继坤. 针对高速机动目标的分数阶滑模制导律[J]. 航空兵器, 2021, 28 (2): 21. doi: 10.12132/ISSN.1673-5048.2020.0183
15	Sheng Y Z, Zhang Z, Xia L. Fractional-order sliding mode control based guidance law with impact angle constraint[J]. Nonlinear Dynamics, 2021, 106 (1): 425. doi: 10.1007/s11071-021-06820-6
16	盛永智, 甘佳豪, 张成新. 弹道可调的落角约束分数阶滑模制导律设计[J]. 航空学报, 2023, 44 (7): 182.
17	Zhang Z Q, Yin Z J, Meng W L, et al. Sliding-mode guidance law with field-of-view constraint and impact time control for missiles[J]. Journal of Physics: Conference Series, 2025, 2955, 012029. doi: 10.1088/1742-6596/2955/1/012029
18	康冰冰, 姜涛, 曹建, 等. 基于强化学习的带落角约束的制导律研究[J]. 航空兵器, 2023, 30 (6): 44. doi: 10.12132/ISSN.1673-5048.2023.0062
19	苏杰, 王康, 曾鑫, 等. 基于深度强化学习的落角和视场角约束制导律[J]. 兵工学报, 2025, 46 (4): 257.
20	张青龙, 赵斌, 许新鹏. 被动探测视场角约束下的深度强化学习制导方法[J]. 宇航学报, 2024, 45 (8): 1281.
21	惠俊鹏, 汪韧, 郭继峰. 基于强化学习的禁飞区绕飞智能制导技术[J]. 航空学报, 2023, 44 (11): 240.
22	Deng T B, Huang H, Fang Y W, et al. Rein-forcement learning based missile terminal guidance of maneuvering targets with decoys[J]. Chinese Journal of Aeronautics, 2023, 36 (12): 309. doi: 10.1016/j.cja.2023.05.028
23	Gao Y, Zhou R, Chen J Y. Integrated entry guidance with no-fly zone constraint using reinforcement learning and predictor-corrector technique[J]. Proceedings of the Institution of Mechanical Engineers, Part G: Journal of Aerospace Engineering, 2024, 238 (7): 728. doi: 10.1177/09544100241236995
24	陈文雪, 高长生, 荆武兴. 拦截机动目标的信赖域策略优化制导算法[J]. 航空学报, 2023, 44 (11): 282.
25	邱潇颀, 高长生, 荆武兴. 拦截大气层内机动目标的深度强化学习制导律[J]. 宇航学报, 2022, 43 (5): 685.
26	Gaudet B, Furfaro R, Linares R. Reinforcement learning for angle-only intercept guidance of maneuvering targets[J]. Aerospace Science and Technology, 2020, 99, 105746. doi: 10.1016/j.ast.2020.105746
27	Lee S, Lee Y, Kim Y, et al. Impact angle control guidance considering seeker’s field-of-view limit based on reinforcement learning[J]. Journal of Guidance, Control, and Dynamics, 2023, 46 (11): 2168.
28	李博皓, 安旭曼, 杨晓飞, 等. 攻击角度约束下的分布式强化学习制导方法[J]. 宇航学报, 2022, 43 (8): 1061.
29	Gong X P, Chen W C, Chen Z Y. All-aspect attack guidance law for agile missiles based on deep reinforcement learning[J]. Aerospace Science and Technology, 2022, 127, 107677. doi: 10.1016/j.ast.2022.107677
30	Ai X L, Wang L L, Yu J Q, et al. Field-of-view constrained two-stage guidance law design for three-dimensional salvo attack of multiple missiles via an optimal control approach[J]. Aerospace Science and Technology, 2019, 85, 334. doi: 10.1016/j.ast.2018.11.052
31	Li Z B, Dong Q L, Zhang X Y, et al. Impact angle-constrained integrated guidance and control for supersonic skid-to-turn missiles using backstepping with global fast terminal sliding mode control[J]. Aerospace Science and Technology, 2022, 122, 107386. doi: 10.1016/j.ast.2022.107386
32	Trigeassou J C, Maamri N, Sabatier J, et al. A Lyapunov approach to the stability of fractional differential equations[J]. Signal Processing, 2010, 91 (3): 437.
33	Chang G N, Wang Y, Chen Y, et al. Guidance law design for ballistic forming with multiple constraints[J]. Journal of Projectiles, Rockets, Missiles and Guidance, 2015 (5): 33.
34	尤浩, 常新龙, 赵久奋, 等. 带角度约束的三维积分二阶滑模拦截制导律[J]. 宇航学报, 2024, 45 (4): 590.
35	张书森, 孟秀云, 丁晓. 一种带终端视线角约束的反舰导弹三维协同制导律[J]. 火力与指挥控制, 2023, 48 (6): 84.

网络层级	策略网络		价值网络
网络层级	单元数	激活函数	单元数	激活函数
注意力层	6	无	6	无
LSTM层	12	无	16	无
归一化层1	256	无	256	无
线性层1	256	Tanh	256	Tanh
线性层2	256	Tanh	256	Tanh
线性层3	4	Tanh	4	Tanh

初始参数	数值
导弹初始位置$ \left(x_{0}, y_{0}, z_{0}\right) $/km	（0，22，0）
目标初始位置$ \left(x_{0}, y_{0}, z_{0}\right) $/km	（30，30，5）
导弹初始速度大小$ v_{{\mathrm{m}} 0} $/（m·s⁻¹）	1100
导弹初始弹道倾角$ \theta_{\mathrm{m}} $/（°）	0
导弹驾驶仪自然震荡频率${\omega _n}$/（°）	6.663
导弹驾驶仪阻尼比${\omega _n}$	0.707
导弹初始弹道偏角$ \phi_{{\mathrm{m}}} $/（°/s）	0
目标初始速度大小$ v_{{\mathrm{m}} 0} $/（m·s⁻¹）	1500
目标初始弹道倾角$ \theta_{\mathrm{m}} $/（°）	−10
目标初始弹道偏角$ \varphi_{{\mathrm{m}}} $/（°）	10
视场角约束$ A_{{\mathrm{f o v lim}}} $/（°）	38
末端攻击角高低角约束/（°）	10
末端攻击角方位角约束/（°）	10

参数	标准值	最大误差	最小误差
目标速度/（m/s）	1500	+75（5%）	−75（5%）
目标纵向加速度/（m/s）	0²	+50	−50
目标侧向加速度/（m/s²）	0	+50	−50

算法	均值	方差
AF-TD3	0.41	0.03
ACSM	3.39	2.93
TSG	0.87	0.24
AACG	2.38	1.64

算法	均值	方差
F-TD3	13.35	0.0023
ACSM	13.38	0.0024
TSG	13.42	0.0024
AACG	13.55	0.0000