基于分层STPA-MC的无人机智能避让适航安全风险评估

doi:10.12305/j.issn.1001-506X.2026.06.21

Abstract

Abstract:

Aiming at the safety risks caused by the application of deep reinforcement learning （DRL） in the unmanned aerial vehicle avoidance system, combined systems theory, control theory, safety analysis and simulation, a safety risk identification and assessment method for intelligent avoidance based on hierarchical systems-theoretic process analysis-Monte Carlo （STPA-MC） is proposed. Firstly, in view of the problem of insufficient capture of safety requirements of complex intelligent systems, based on STPA, the intelligent obstacle avoidance system architecture and DRL model construction are decoupled. In addition to analyzing failure risks and protection at the system level, the failure cause scenarios are also evaluated based on the DRL model development process, and the safety hazard impact traceability between levels is established. Secondly, in view of the quantification of key safety indicators, the MC method is used to evaluate the compliance of the soft actor critic intelligent algorithm with the airworthiness standards, analyze the safety impact of different factors, and derive quantitative requirements. Experiments show that in order to meet the airworthiness requirements, the perception distance needs to be greater than 445 m, and the standard deviation of the ranging error should be less than 0.43 m. The research results can provide theoretical support for the formulation of airworthiness standards for unmanned aerial vehicle intelligent avoidance.

Key words: hierarchical systems-theoretic process analysis （STPA）, intelligent avoidance, deep reinforcement learning （DRL）, airworthiness safety, Monte Carlo （MC）

CLC Number:

V 244.12

Zan MA, Yubin LIU, Jie BAI, Yong CHEN, Shuguang SUN. Intelligent avoidance airworthiness safety risk assessment of unmanned aerial vehicle based on hierarchical STPA-MC[J]. Systems Engineering and Electronics, 2026, 48(6): 2000-2013.

Figures/Tables 24

Fig.1

Fig.2

Fig.3

Table 1

Table 2

Fig.4

Table 3

Table 4

Fig.5

Fig.6

Table 5

Table 6

Deep causal scenarios"

研制生命周期	深层致因场景	缓解措施
训练环境搭建	D1（奖励函数存在稀疏性或超出合理范围）[S5]	通过课程学习从简单场景训练过度到复杂场景或进行奖励分解将避让导航节能等任务分配给子策略网络，通过顶层仲裁器整合或进行奖励重塑
	D2（传感器数据对齐错误（如相机与LIDAR坐标系未标定导致空间感知冲突））[S5]	通过硬件同步与离线标定（手眼标定算法）确保数据一致性
	D3（观测信息维度不足（如忽略障碍物加速度，自身电池容量）导致DRL策略短视）[S5]	设置最小安全观测集，强制包含动力系统状态（电压电机温度）环境动态信息（障碍物运动学参数）等
	D4（动作空间未考虑执行器物理极限（如电机最大转速、舵机角度范围））[S5]	在DRL输出层引入饱和函数，限制指令范围
	D5（离散动作划分过粗（如转向角仅设5档））[S5]	对连续动作与离散动作分层设计，避免维度灾难
	D6（仿真环境未模拟真实物理特性（如空气阻力、传感器噪声））[S5,S6]	采用支持高精度空气动力学和传感器噪声模型的仿真平台，并在测试时对比仿真飞行与真实飞行表现的差异，量化差距
	D7（训练环境空域密度过低导致缺少冲突样本）[S5,S7]	增加对训练样本质量评估，提升训练样本质量
模型训练	D8（神经网络架构复杂度与任务不匹配）[S5,S6,S7]	进行神经架构搜索，寻找最优网络架构或进行网络架构复杂度−安全性闭环研制
	D9（策略网络采用全连接结构忽略时空关联特征）[S5]	采用卷积神经网络−LSTM混合架构处理时序传感器数据
	D10（值网络估计存在偏差）[S5]	修改目标函数
	D11（强化学习算法类型与任务不匹配）[S5]	根据任务特性选择算法并进行多轮对比实验比较算法性能
	D12（超参数设置与任务不匹配）[S5,S7]	采用遗传算法等进行超参数寻优
	D13（存在探索利用困境）[S5，S7]	引入贪心探索或置信上限等探索策略
	D14（样本利用率低）[S5]	引入优先经验回放机制等提高样本利用效率
模型评估	D15（模型验证和确认评估计划设置不合理（如极端场景下避让失败率未被检测，测试用例未覆盖长尾场景，未评估模型在边缘设备上的推理延迟））[S5]	进行对抗测试生成，使用生成对抗网络生成极端场景（如极端光照，障碍物畸形）扩充测试集
	D16（评估时测试用例执行不完整）[S5,S7]	建立评估执行清单，关键测试场景留存备案
	D17（反馈时将模型缺陷归因错误）[S5,S6,S7]	引入标准化的缺陷归因流程并提供致因相关性分析
	D18（反馈可操作性不足）[S5]	确保反馈结构化，明确给出反馈建议的实施步骤和要求

Table 6

Fig.7

Fig.8

Table 7

Table 8

Fig.9

Fig.10

Fig.11

Fig.12

Fig.13

Fig.14

Table 9

Table 10

References 31

1	国家市场监督管理总局, 国家标准化管理委员会. 民用无人驾驶航空器系统安全要求(GB 42590-2023)[S]. 北京: 中国标准化出版社, 2023.
	State Administration for Market Regulation, National Standardization Administration. Safety requirements for civil unmanned aircraft system(GB 42590-2023)[S]. Beijing: China Standard Publishing House, 2023.
2	RTCA DO-365C. Minimum operational performance standard for detect and avoid systems[S]. Washington, D. C: Radio Technical Commission for Aeronautics, 2020.
3	LONGO G, STROHMEIER M, RUSSO E, et al. On a collision course: unveiling wireless attacks to the aircraft traffic collision avoidance system [C]//Proc. of the 33th USE-NIX Security Symposium, 2024.
4	CANDAN F, BEKE A, MAHFOUR M, et al. A real-time fuzzy interacting multiple model velocity obstacle avoidance approach for unmanned aerial vehicles[J]. Journal of Intelligent & Robotic Systems, 2024, 110, 61.
5	WANG X H, CHENG M Y, ZHANG S, et al. Multi UAV coopertive obstacle avoidance of 3D vector field histogram plusand dynamic window approach[J]. Drones, 2023, 7 (8): 504. doi: 10.3390/drones7080504
6	EASA. Roadmap AI. A human-centric approach to AI in aviation[EB/OL].[2025-02-18]. WWW.adfingo.com/easa-ai-roadmap-v10.pdf.
7	CHEN H, HUANG D, WANG C G, et al. Collision-free path planning for multiple drones based on safe reinforcement learning[J]. Drones, 2024, 8 (9): 481. doi: 10.3390/drones8090481
8	吴冯国, 陶伟, 李辉, 等. 基于深度强化学习算法的无人机智能规避决策[J]. 系统工程与电子技术, 2023, 45 (6): 1702- 1711.
	WU F G, TAO W, LI H, et al. UAV intelligent avoidance decisions based on deepreinforcement learning algorithm[J]. Systems Engineering and Electronics, 2023, 45 (6): 1702- 1711.
9	RAMEZANI M, HBIBI H, VOOS H, et al. UAV path planning employing MPC-reinforcement learning method considering collision avoidance[C]//Proc. of the IEEE 8th International Conference on Unmanned Aircraft Systems, 2023.
10	ZHOU J L, ZHANG H H, HUA M Z, et al. P-DRL: a framework for multi-UAVs dynamic formation control under operational uncertainty and unknown environment[J]. Drones, 2024, 8 (9): 475. doi: 10.3390/drones8090475
11	JANSON V, AHLBRECHT A, DURAK U. Architectural challenges in developing an AI-based collision avoidance system[C]//Proc. of the IEEE/AIAA 42th Digital Avionics Systems Conference, 2023.
12	马赞, 白杰, 陈勇, 等. 基于条件高斯PAC-Bayes的机载CNN分类器安全性评估[J]. 航空学报, 2025, 46 (7): 330824.
	MA Z, BAI J, CHEN Y, et al. Safety assessment for airborne CNN classifier based on conditional gaussian PAC-Bayes[J]. Acta Aeronautica et Astronautica Sinica, 2025, 46 (7): 330824.
13	SALMON P M, STANTON N A, WALKER G H, et al. Handbook of systems thinking methods [M]. 1st ed. Boca Raton: CRC Press, 2022.
14	GO E, JEON H C, LEE J S, et al. Enhancing urban public safety through UAS integration: a comprehensive hazard analysis with the STAMP/STPA framework[J]. Applied Sciences, 2024, 14 (11): 4609. doi: 10.3390/app14114609
15	BANIK S, KIM J, HOVAKIMYAN N, et al. Integrating vision systems and STPA for robust landing and take-off in VTOL aircraft[C]//Proc. of the AIAA SciTech Forum, 2025.
16	张宏宏, 甘旭升, 孙静娟, 等. 基于STPA-TOPAZ的低空无人机冲突解脱安全性分析[J]. 航空学报, 2022, 43 (7): 325354.
	ZHANG H H, GAN X S, SUN J J, et al. Analysis of low altitude UAV conflict resolution safety based on STPA-TOPAZ[J]. Acta Aeronautica et Astronautica Sinica, 2022, 43 (7): 325354.
17	BERGER J, TIUSANEN R, KOTHALAWALA H, et al. Applying priority-informed STPA to a nuclear I&C system[C]//Proc. of the 29th International Conference on Emerging Technologies and Factory Automation, 2024.
18	BERRI I, ZENNIR Y, MECHHOUD E A, et al. Integrating functional block diagrams and systems-theoretic-process analysis: a case study of a nuclear power station[J]. International Journal of Safety & Security Engineering, 2024, 14 (2): 359- 372.
19	ZHANG J X, ZHANG S H, LIANG Z W, et al. A risk assessment method based on DEMATEL-STPA and its application in safety risk evaluation of hydrogen refueling stations[J]. International Journal of Hydrogen Energy, 2024, 50, 889- 902. doi: 10.1016/j.ijhydene.2023.07.250
20	WANG J J, QI M, LIU S Z, et al. Comprehensive safety assessment of a hydrogen liquefaction system based on an integrated system-theoretic process analysis and best worst method[J]. International Journal of Hydrogen Energy, 2024, 66, 479- 489. doi: 10.1016/j.ijhydene.2024.04.082
21	HOBBS K L, HEINER B, BUSSE L, et al. Systems theoretic process analysis of a run time assured neural network control system[C]//Proc. of the AIAA SciTech Forum, 2023.
22	ZHANG S J, TANG T, LIU J T. A hazard analysis approach for the SOTIF in intelligent railway driving assistance systems using STPA and complex network[J]. Applied Sciences, 2021, 11 (16): 7714. doi: 10.3390/app11167714
23	DGHAYM D, HOANG T S, TURNOCK S R, et al. An STPA-based formal composition framework for trustworthy autonomous maritime systems[J]. Safety Science, 2021, 136, 105139. doi: 10.1016/j.ssci.2020.105139
24	EASA. First usable guidance for Level 1 machine learning applications[R]. Cologne: European Union Aviation Safety Agency, 2021: 10−48.
25	HAARNOJA T, ZHOU A, ABBEEL P, et al. Soft-actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor[C]//Proc. of the 35th International conference on machine learning, 2018.
26	AC-21-AA-2022-40民用无人驾驶航空器系统适航审定分级分类和系统安全性分析指南[S]. 北京: 中国民用航空局航空器适航审定司, 2022.
	AC-21-AA-2022-40Guidelines for airworthiness certification classification and system safety analysis of civil unmanned aerial vehicle systems[S]. Beijing: Aircraft Airworthiness Certification Department of Civil Aviation Administration of China, 2022.
27	ASHMORE R, CALINESCU R, PATERSON C. Assuring the machine learning lifecycle: desiderata, methods, and challenges[J]. Assoc Computing Machinery Computing Surveys, 2021, 54 (5): 111.
28	SAE AIR 6994. Artificial intelligence in aeronautical systems: use cases[S]. Warrendale: Society of Automotive Engineers, 2021.
29	WEINER A, ALVAREZ L, OWEN M, et al. A quantitatively derived nmac analog for smaller unmanned aircraft systems based on unmitigated collision risk[J]. Preprints, 2020, 14, 22.
30	MISSURA M, BENNEWITZ M. Predictivecollision avoidance for the dynamic window approach[C]//Proc. of the 36th International Conference on Robotics and Automation, 2019.
31	张宏宏, 甘旭升, 李昂, 等. 基于速度障碍法的无人机避障与航迹恢复策略[J]. 系统工程与电子技术, 2020, 42 (8): 1759- 1767. doi: 10.3969/j.issn.1001-506X.2020.08.16
	ZHANG H H, GAN X S, LI A, et al. UAV obstacle avoidance and track recovery strategy based on velocity obstacle method[J]. Systems Engineering and Electronics, 2020, 42 (8): 1759- 1767. doi: 10.3969/j.issn.1001-506X.2020.08.16

损失分级	对飞机的影响	对乘员及地面人员的影响	对地面设施的影响	对地面机组的影响
A1（轻微的）	安全裕度或功能特性轻微降低	身体不适	功能特性或冗余能力短暂或局部下降	轻微增加工作负担
A2（重大的）	安全裕度或功能特性显著降低	身体痛苦可能受伤	核心功能出现持续性降级或部分失效	明显增加工作负担
A3（危险的）	安全裕度和功能能力大幅下降	可能导致人员受伤严重	关键功能长时间失效或严重偏离设计	极大增加工作负担影响执行任务的完整性和准确性
A4（灾难性的）	完全丧失控制在限制空域内坠毁	导致人员一人或多人遇难	完全丧失核心功能且无法恢复	机组完全丧失对无人机的操纵能力

危险	可能导致的损失级别
H1（飞机发生空中碰撞）	A4
H2（飞机发生空中相近碰撞）	A3
H3（飞机在超出限环境或状态中运行）	A3
H4（飞机进行过度激进的机动违反飞机或部件的结构限制）	A3
H5（本机运动超出乘员健康和舒适度的限制）	A2

控制动作	避让系统提供避让决策
提供导致危险的控制行为	UCA1：提供错误避让决策 [H1，H2，H3，H4，H5]
未提供控制行为	UCA2：未提供避让决策[H1，H2]
提供控制行为时间不当	UCA3：过早或过晚提供避让决策[H1，H2]
提供控制行为的持续时间不当	UCA4：连续震荡调整避让决策[H1，H2，H5]

控制结构组件	浅层致因场景
感知系统	S1（对入侵机距离等计算误差过大）
	S2（仿真环境与真实世界差异过大导致感知模型失效）
	S3（感知距离过近）
	S4（感知信息更新太慢）
基于DRL的避让系统	S5（模型决策水平低）
	S6（模型鲁棒性差）
	S7（模型泛化性差）
RTA系统	S8（RTA系统中的监视组件计算的错误）
	S9（超出备用系统决策能力上限）
	S10（RTA系统中的控制选择器切换故障）
飞控系统	S11（飞控设备故障）
飞控系统	S12（飞控算法存在缺陷）
动力系统	S13（电池电压下降）
	S14（电机老化推力不足）
	S15（螺旋桨损坏）
数据链路	S16（数据格式不匹配）
	S17（时钟不同步，存在信号延迟）
	S18（数据丢失）
机身传感器	S19（机身传感器发生错误）
机身传感器	S20（机身传感器状态更新太慢）

超参数	影响
学习率	过大导致训练不稳定，甚至错过最优解。过小导致训练过慢甚至陷入局部最优
折扣因子	接近1考虑更多的未来奖励，适用于长期任务。接近0考虑即时奖励，适用于短期任务或决策
批量大小	较大批量提供更稳定的梯度估计，但需要更多的计算资源。较小的批量可以更频繁更新模型参数，但可能导致较高方差
目标网络更新频率	更新频率较高可能导致不稳定，较低导致训练速度较慢

Intelligent avoidance airworthiness safety risk assessment of unmanned aerial vehicle based on hierarchical STPA-MC

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 24

References 31

Related Articles 9

Recommended Articles

Metrics

Comments

入侵飞机装备	NMAC风险比测量
应答机或自动相关监视广播 OUT	≤0.18
非合作	≤0.30

参数	取值
动作网络学习率	3e−4
动作网络输入维度	10
动作网络隐含层架构	[256,256]
动作网络激活函数	Relu,Tanh
动作网络输出维度	2
值网络学习率	3e−4
值网络输入维度	12
值网络隐含层架构	[256,256]
值网络激活函数	Relu
值网络输出维度	1
样本数	256
经验池大小	10e6
软更新系数	5e−3
折扣因子	0.995
训练回合数	4600
每轮最大步数	6000

适航标准中定义需求^[24]	本文方法捕获的需求
应验证训练过程中（学习算法、模型架构超参数等）关键要素的选择合理性	R1（在训练过程中，超参数、网络架构、强化学习算法选择要与任务相匹配）[D8，D9，D10，D11]
应对数据质量进行验证	R2（需提交训练样本分布的覆盖分析，特别是对临界安全区域、罕见场景是否有足够训练支撑）[D7]
用于训练的硬件平台和用于验证的硬件平台之间的差异对推理模型行为的影响应该被识别和评估	R3（仿真环境需模拟真实物理特性）[D6]
应定义指标来评估人工智能/ML 组件的性能和可靠性	R4（应对模型避让性能进行评估）[S5]
应对算法和训练模型的鲁棒性（稳定性）进行分析	R5（应对模型的鲁棒性和泛化性进行评估）[S6，S7]

[1]	Kangjie ZHENG, Xinyu ZHANG, Weisong WANG, Zhensheng LIU. Intelligent ship dynamic autonomous obstacle avoidance decision based on DQN and rule [J]. Systems Engineering and Electronics, 2025, 47(6): 1994-2001.
[2]	Kaiqiang TANG, Huiqiao FU, Jiasheng LIU, Guizhou DENG, Chunlin CHEN. Hierarchical optimization research of constrained vehicle routing based on deep reinforcement learning [J]. Systems Engineering and Electronics, 2025, 47(3): 827-841.
[3]	Tingyu ZHANG, Ying ZENG, Nan LI, Hongzhong HUANG. Spacecraft power-signal composite network optimization algorithm based on DRL [J]. Systems Engineering and Electronics, 2024, 46(9): 3060-3069.
[4]	Zhipeng YANG, Zihao CHEN, Chang ZENG, Song LIN, Jindi MAO, Kai ZHANG. Online route planning decision-making method of aircraft in complex environment [J]. Systems Engineering and Electronics, 2024, 46(9): 3166-3175.
[5]	Mengyu ZHANG, Yajie DOU, Ziyi CHEN, Jiang JIANG, Kewei YANG, Bingfeng GE. Review of deep reinforcement learning and its applications in military field [J]. Systems Engineering and Electronics, 2024, 46(4): 1297-1308.
[6]	Fengguo WU, Wei TAO, Hui LI, Jianwei ZHANG, Chengchen ZHENG. UAV intelligent avoidance decisions based on deep reinforcement learning algorithm [J]. Systems Engineering and Electronics, 2023, 45(6): 1702-1711.
[7]	Jin TANG, Yangang LIANG, Zhihui BAI, Kebo LI. Landing control algorithm of rotor UAV based on DQN [J]. Systems Engineering and Electronics, 2023, 45(5): 1451-1460.
[8]	XIE Hao, GUO Aihuang, SONG Chunlin, JIAO Runze. eNB selection for LTE-V using deep reinforcement learning [J]. Systems Engineering and Electronics, 2019, 41(7): 1652-1657.
[9]	WANG Zhuo-fu, DING Ji-yong, LIU Yuan, LIU Xun. Analysis of critical path and most critical activity in PERT networks based on Monte Carlo method [J]. Journal of Systems Engineering and Electronics, 2012, 34(8): 1646-1651.

本方法捕获的新需求
R6（系统需提供观测空间完备性分析报告证明所有关键动态参数已被建模，并通过仿真与实物对比验证其覆盖性）[D1]
R7（需提供奖励函数有效性量化（稀疏奖励占比监测、奖励分布偏移检测），并验证其梯度稳定性，确保其在状态空间边界仍能引导安全行为）[D2]
R8（需验证动作空间定义是否符合执行器动力学约束，加入物理可行性分析报告）[D3，D4]
R9（系统需内置传感器冲突检测算法，并在审定测试中模拟标定偏移，验证冲突报警率）[D5]
R10（提交策略收敛性与稳定性分析报告，包括探索策略的边界条件与可能的行为异常分析）[D12]
R11（需通过对抗性样本生成测试，验证策略在认为构建的极端场景中仍能满足避碰要求）[D14]