基于元学习与预测控制的导弹自适应制导

doi:10.12305/j.issn.1001-506X.2026.07.26

摘要/Abstract

摘要：

针对气动扰动、目标机动、执行器故障及导弹参数变化带来的复杂影响，提出一种基于深度强化学习（deep reinforcement learning, DRL）的自适应制导框架。首先，采用熵正则化DRL，利用深层神经网络（deep neural network, DNN）作为预测模型，结合遗忘机制增强元学习，动态调整经验回放权重以聚焦当前环境。之后，设计双模型预测架构，集成交叉熵方法模型预测控制（cross-entropy method model predictive control, CEM-MPC）和自适应熵的模型预测路径积分（model predictive path integral, MPPI）控制，监测成本标准差变化率，在波动超阈值时自动切换控制策略，以发挥两种控制器的收敛速度和探索能力优势。仿真表明，相比传统基于学习的非自适应方法，所提方法视线角速率稳定，拦截成功率分别提升47.6%和6.0%，验证了本方法的有效性。

关键词: 强化学习, 深度神经网络, 元学习, 模型预测控制, 导弹制导

Abstract:

To address the complex impacts of aerodynamic perturbations, target maneuvers, actuator failures, and missile parameter variations, this paper proposes an adaptive guidance framework based on deep reinforcement learning （DRL）. First, the framework employs entropy-regularized DRL, utilizing a deep neural network （DNN） as the predictive model. By integrating a forgetting mechanism into meta-learning, it dynamically adjusts experience replay weights to prioritize the current environment. Subsequently, a dual-model predictive control architecture is designed, combining cross-entropy method model predictive control （CEM-MPC） and adaptive entropy model predictive path integral （MPPI） control. This architecture monitors the rate of change in cost standard deviation and automatically switches control strategies when fluctuations exceed a threshold, thereby leveraging the convergence speed and exploration advantages of both controllers. Simulations demonstrate that compared to traditional learning-based non-adaptive methods, the proposed approach stabilizes line-of-sight angular rates and improves interception success rates by 47.6% and 6.0% respectively, validating its effectiveness.

Key words: reinforcement learning, deep neural network (DNN), meta-learning, model predictive control, missile guidance

中图分类号:

TJ 765.3

李强, 周荻, 李思远, 林玉荣. 基于元学习与预测控制的导弹自适应制导[J]. 系统工程与电子技术, 2026, 48(7): 2424-2433.

Qiang LI, Di ZHOU, Siyuan LI, Yurong LIN. Adaptive guidance for missile based on meta-learning and predictive control[J]. Systems Engineering and Electronics, 2026, 48(7): 2424-2433.

图/表 20

图1

图2

图3

图4

图5

表1

图6

图7

表2

表3

图8

图9

表4

图10

图11

图12

表5

表6

表7

图13

参考文献 30

1	詹译傲, 李思远, 周荻, 等. 基于协同估计的三维多导弹博弈制导律[J]. 宇航学报, 2024, 45 (9): 1467- 1480. doi: 10.3873/j.issn.1000-1328.2024.09.012
	ZHAN Y A, LI S Y, ZHOU D, et al. Three-dimensional multi-missile game guidance law based on cooperative estimation[J]. Journal of Astronautics, 2024, 45 (9): 1467- 1480. doi: 10.3873/j.issn.1000-1328.2024.09.012
2	KIM Y, KIM B S. Pitch autopilot design for agile missiles with uncertain aerodynamic coefficients[J]. IEEE Trans. on Aerospace and Electronic Systems, 2013, 49 (2): 907- 914. doi: 10.1109/TAES.2013.6494388
3	KUMAR S R, RAO S, GHOSE D. Sliding-mode guidance and control for all-aspect interceptors with terminal angle constraints[J]. Journal of Guidance, Control, and Dynamics, 2012, 35 (4): 1230- 1246.
4	CHO H, RYOO C K, TSOURDOS A, et al. Optimal impact angle control guidance law based on linearization about collision triangle[J]. Journal of Guidance, Control, and Dynamics, 2014, 37 (3): 958- 964.
5	CHO D, KIM H J, TAHK M J. Impact angle constrained sliding mode guidance against maneuvering target with unknown acceleration[J]. IEEE Trans. on Aerospace and Electronic Systems, 2015, 51 (2): 1310- 1323. doi: 10.1109/TAES.2015.140358
6	KIM H G, KIM H J. All-aspect guidance with impact angle constraint against unknown target maneuver[J]. IEEE Trans. on Aerospace and Electronic Systems, 2019, 55 (2): 830- 845. doi: 10.1109/TAES.2018.2865143
7	HE S M, LIN D F, WANG J. Integral global sliding mode guidance for impact angle control[J]. IEEE Trans. on Aerospace and Electronic Systems, 2019, 55 (4): 1843- 1849. doi: 10.1109/TAES.2018.2876588
8	禹志龙, 李颖晖, 裴彬彬, 等. 基于事件触发采样的无尾翼飞机自适应容错姿态控制[J]. 系统工程与电子技术, 2024, 46 (3): 1058- 1066.
	YU Z L, LI Y H, PEI B B, et al. Adaptive fault-tolerant attitude control of tailless aircraft based on event-triggered sampling[J]. Systems Engineering and Electronics, 2024, 46 (3): 1058- 1066.
9	DONG W, WANG C Y, WANG J N, et al. Varying-gain proportional navigation guidance for precise impact time control[J]. Journal of Guidance, Control, and Dynamics, 2023, 46 (3): 535- 552.
10	WANG Y N, WANG H, LIN D F, et al. Nonlinear modified bias proportional navigation guidance law against maneuvering targets[J]. Journal of the Franklin Institute, 2022, 359 (7): 2949- 2975. doi: 10.1016/j.jfranklin.2022.02.012
11	刘远贺, 黎克波, 何绍溟, 等. 基于最优误差动力学的变速导弹飞行路程控制制导律[J]. 航空学报, 2023, 44 (7): 326909.
	LIU Y H, LI K B, HE S M, et al. Flying range control guidance for varying-speed missiles based on optimal error dynamics[J]. Acta Aeronautica Astronautica Sinica, 2023, 44 (7): 326909.
12	BALAS G J. Flight control law design: an industry perspective[J]. European Journal of Control, 2003, 9 (23): 207- 226.
13	JIANG X, ZUO L, WANG S, et al. Robust predictive fault-tolerant control under actuator random failures[J]. Chemical Engineering Science, 2025, 287, 120381.
14	CANDON M, HALE E, BALAJEWICZ M, et al. Parameterization of nonlinear aeroelastic reduced order models via direct interpolation of Taylor partial derivatives[J]. Nonlinear Dynamics, 2024, 117 (3): 2145- 2162.
15	NARAYANAN G, JEONG J H, JOO Y H. Resilient sampled-data control for fractional-order PMVG-based WTS with actuator saturation and probabilistic faults using fuzzy Lyapunov function method[J]. Information Sciences, 2025, 621, 121294.
16	闫循良, 王宽, 张子剑, 等. 基于LSTM-DDPG的再入制导方法[J]. 系统工程与电子技术, 2025, 47 (1): 268- 279.
	YAN X L, WANG K, ZHANG Z J, et al. Reentry guidance method based on LSTM-DDPG[J]. Systems Engineering and Electronics, 2025, 47 (1): 268- 279.
17	肖柳骏, 李雅轩, 刘新福. 基于强化学习的高超声速滑翔飞行器自适应末制导[J]. 兵工学报, 2025, 46(2): 240222.
	XIAO L J, LI Y X, LIU X F. Adaptive terminal guidance for hypersonic gliding vehicles using reinforcement learning [J]. Acta Armamentarii, 2025, 46(2): 240222.
18	杨志鹏, 陈子浩, 曾长, 等. 复杂环境下的飞行器在线航路规划决策方法[J]. 系统工程与电子技术, 2024, 46 (9): 3166- 3175.
	YANG Z P, CHEN Z H, ZENG C, et al. Online route planning decision-making method of aircraft in complex environment[J]. Systems Engineering and Electronics, 2024, 46 (9): 3166- 3175.
19	PEI Z, ROJAS-AREVALO A M, DE HAAN F J, et al. Reinforcement learning for decision-making under deep uncertainty[J]. Journal of Environmental Management, 2024, 358, 120968.
20	ZHANG J J, MA X T, YAN J P, et al. Uncertainty-driven trajectory truncation for model-based offline reinforcement learning[J]. https://doi.org/10.48550/arXiv, 2304, 04660
21	LAWRENCE N P, FORBES M G, LOEWEN P D, et al. Deep reinforcement learning with shallow controllers: an experimental application to PID tuning[J]. Control Engineering Practice, 2022, 121, 105046. doi: 10.1016/j.conengprac.2021.105046
22	DALLY K, KAMPEN E J V. Soft actor-critic deep reinforcement learning for fault tolerant flight control[EB/OL]. [2025-04-30]. https://doi.org/10.48550/arXiv.2202.09262.
23	YONG H, SEO J, KIM J, et al. Suspension control strategies using switched soft actor-critic models for real roads[J]. IEEE Trans. on Industrial Electronics, 2023, 70 (1): 824- 832. doi: 10.1109/TIE.2022.3153805
24	ZHANG Y L, XIE J N, DU X H, et al. Biped robot terrain adaptability based on improved sacalgorithm[J]. Mechanisms and Machine Science, 2024, 148, 123- 135.
25	CHENG Y, SONG Y. Autonomous decision-making generation of UAV based on soft actor-critic algorithm[C]//Proc. of the Chinese Control Conference, 2020: 7350–7355.
26	JANSEN H, VAN KAMPEN E J. Longitudinal handling qualities evaluation for soft actor-critic deep reinforcement learning flight control[C]//Proc. of the AIAA SCITECH 2025 Forum, 2025.
27	HATTORI R, HEDRICK N G, JAIN A, et al. Meta-reinforcement learning via orbitofrontal cortex[J]. Nature Neuroscience, 2023, 26 (11): 2017- 2030. doi: 10.1038/s41593-023-01442-0
28	GAUDET B, DROZD K, FURFARO R. Adaptive approach phase guidance for a hypersonic glider via reinforcement meta learning[C]//Proc. of the AIAA SCITECH 2022 Forum, 2022.
29	LIANG C, WANG W H, LIU Z H, et al. Learning to guide: guidance law based on deep meta-learning and model predictive path integral control[J]. IEEE Access, 2019, 7, 47353- 47365. doi: 10.1109/ACCESS.2019.2909579
30	KUMAR S R, GHOSE D. Three-dimensional impact angle guidance with coupled engagement dynamics[J]. Proceedings of the Institution of Mechanical Engineers, Part G: Journal of Aerospace Engineering, 2017, 231 (4): 621- 641. doi: 10.1177/0954410016641442

损失	最小损失	最大损失	平均损失
取值	0.038 014 20	0.626 501 79	0.056 630 45

方法	响应时间
方法1	2.782
方法2	3.031
方法3	2.188
方法4	2.635

参数	值
导弹−目标距离/m	3 000
导弹仰角/rad	−0.21
导弹偏角/rad	−0.16
视线仰角/rad	−0.61
视线偏角/rad	0.60
目标仰角/rad	−0.14
目标偏角/rad	−0.23
导弹初始速度/（m/s）	1000
目标初始速度/（m/s）	300
目标y轴加速度/（m/s⁻²）	30sin（t）
目标z轴加速度/（m/s⁻²）	30sin（t）
执行器故障范围	（−0.3,0）
执行器故障发生时间/s	2.5
零升阻力波动范围/%	（−3,3）
诱导阻力波动范围/%	（−3,3）
大气密度波动范围/%	（−3,3）
推力持续时间/s	3.0

方法	网络	控制器
方法1	本文采用的带有熵正则化的网络	本文采用的双模策略控制器
方法2	本文采用的带有熵正则化的网络	本文改进的CEM-MPC
方法3	基于学习的在线预测方法	未改进的MPPI控制

方法	拦截耗时/s	脱靶量/m	末端速度/（m/s）
方法1	4.633 6	0.000 56	818.19
方法2	4.619 1	0.080 55	820.26
方法3	4.626 6	0.004 69	819.13