面向LVC训练的蓝方虚拟实体近距空战决策建模

doi:10.12305/j.issn.1001-506X.2021.06.18

摘要/Abstract

摘要：

真实-虚拟-构造为近距空战对抗训练提供了有力支撑。针对课题对蓝方虚拟实体的实际决策建模需求, 在对比分析深度强化学习与经典智能优化方法的基础上, 从优化理论的角度对神经网络的权值空间和结构空间进行定义, 提出基于智能优化的进化神经网络决策模型及其求解方法。首先，分析近距空战战术特点, 战机飞行运动模型, 实际决策建模需求。其次，分别设计战机关键飞行状态、动作空间、适应度函数, 实现蓝方端到端感知与决策。最后, 给出基于经典遗传神经网络的决策模型及求解示例。结果表明, 所提方法可实现蓝方战机通过对抗数据来学习对手作战特点的功能, 验证了模型及方法的有效性; 同时所提方法对目前智能优化及其改进算法, 以及不同结构神经网络具有通用性。

关键词: 真实-虚拟-构造, 近距空战, 决策模型, 神经网络, 智能优化, 智能蓝方

Abstract:

The live-virtual-constructive (LVC) provides a strong support for the close-range air combat confrontation training. Aiming at the actual decision modeling requirements of blue-side virtual entity, based on the comparative analysis of deep reinforcement learning and classical intelligent optimization methods, the weight space and structure space of neural network are defined from the perspective of optimization theory, and the evolutionary neural network decision model based on intelligent optimization and its solution method are proposed. Firstly, the tactical characteristics of close-range air combat, fighter flight motion model and actual decision modeling requirements are analyzed. Secondly, the key flight state, action space and fitness function are designed to realize the end-to-end perception and decision of the blue-side. Finally, the decision model based on classical genetic neural network and its solution example are given. The results show that the proposed method can realize the function of learning the combat characteristics of the opponent through the confrontation data, and verify the effectiveness of the model and method; at the same time, it has universality for the current intelligent optimization and its improved algorithm, as well as different structure neural networks.

Key words: live-virtual-constructive (LVC), close-range air combat, decision model, neural network, intelligent optimization, blue-side of intelligence

高昂, 董志明, 李亮, 段莉, 郭齐胜. 面向LVC训练的蓝方虚拟实体近距空战决策建模[J]. 系统工程与电子技术, 2021, 43(6): 1606-1617.

Ang GAO, Zhiming DONG, Liang LI, Li DUAN, Qisheng GUO. Decision modeling of close-range air combat for LVC training in blue-side virtual entity[J]. Systems Engineering and Electronics, 2021, 43(6): 1606-1617.

图/表 14

图1

图2

图3

图4

图5

图6

图7

图8

图9

图10

表1

图11

表2

表3

参考文献 33

1	BEST C , FLTLT B R . Science and technology enablers of live virtual constructive training in the air domain[J]. Air & Space Power Journal, 2018, 32 (4): 59- 73.
2	GREEN J , ZECKHAUSER R . Thunder versus lightning: a performance and cost analysis of the a-10 "warthog" versus the F-35 joint strike fighter[J]. Journal of Benefit-cost Analysis, 2019, 10 (3): 434- 468. doi: 10.1017/bca.2019.27
3	ARMON T, JAN J R, PIETER S. Rewarding air combat behavior in training simulations[C]//Proc.of the IEEE International Conference on Systems, Man, and Cybernetics, 2015: 1397-1402.
4	ARMON T, JAN J R, PIETER S. Transfer learning of air combat behavior[C]//Proc.of the IEEE 14th International Confe-rence on Machine Learning and Applications, 2015: 226-231.
5	RIJKEN R, TOUBMAN A. The future of autonomous air combat behavior[C]//Proc.of the IEEE International Conference on Systems, Man, and Cybernetics, 2016: 3089-3094.
6	BABAK T, FARZAD K, LINUS J L. Evaluating deep reinforcement learning for computer generated forces in ground combat simulation[C]//Proc.of the IEEE International Conference on Systems, Man, and Cybernetics, 2017: 3433-3438.
7	潘耀宗, 张健, 杨海涛, 等. 战机自主作战机动双网络智能决策方法[J]. 哈尔滨工业大学学报, 2019, 51 (11): 144- 151. doi: 10.11918/j.issn.0367-6234.201811083
	PAN Y Z , ZHANG J , YANG H T , et al. Dual network intelligent decision method for fighter autonomous combat maneuver[J]. Journal of Harbin Institute of Technology, 2019, 51 (11): 144- 151. doi: 10.11918/j.issn.0367-6234.201811083
8	ZHANG X B , LIU G Q , YANG C J , et al. Research on air confrontation maneuver decision-making method based on reinforcement learning[J]. Electronics, 2018, 7 (11): 279. doi: 10.3390/electronics7110279
9	邵彦昊, 朱荣刚, 贺建良, 等. 基于深度学习的不可逃逸区内的规避决策研究[J]. 电光与控制, 2019, 26 (11): 60- 64. doi: 10.3969/j.issn.1671-637X.2019.11.013
	SHAO Y H , ZHU R G , HE J L , et al. Evasive decision-making in inescapable areas based on deep learning[J]. Electronics Optics & Control, 2019, 26 (11): 60- 64. doi: 10.3969/j.issn.1671-637X.2019.11.013
10	周凯, 魏瑞轩, 张启瑞, 等. 基于经验移植的自主空战对抗学习方法[J]. 航空学报, 2020, 40 (X): 724285.
	ZHOU K , WEI R X , ZHANG Q R , et al. Learning method based on experience transfer for autonomous air combat[J]. Acta Aeronautica et Astronautica Sinica, 2020, 40 (X): 724285.
11	何金, 丁勇, 高振龙. 基于Double Deep Q Network的无人机隐蔽接敌策略[J]. 电光与控制, 2020, 27 (7): 52- 57. doi: 10.3969/j.issn.1671-637X.2020.07.010
	HE J , DING Y , GAO Z L . A stealthy engagement maneuvering strategy of UAV based on Double Deep Q Network[J]. Electronics Optics & Control, 2020, 27 (7): 52- 57. doi: 10.3969/j.issn.1671-637X.2020.07.010
12	ZHOU K , WEI R X , ZHANG Q R , et al. Learning system for air combat decision inspired by cognitive mechanisms of the brain[J]. IEEE Access, 2020, 8, 8129- 8144. doi: 10.1109/ACCESS.2020.2964031
13	DEWANGGA W, SASONGKO R A. Air combat maneuver guidance system for UCAV Using neutral-advantage-disadvantage-mutual disadvantage approach[C]//Proc.of the IEEE 6th Asian Conference on Defence Technology, 2019: 155-161.
14	YANG Q M , ZHANG J D , SHI G Q , et al. Maneuver decision of UAV in short-range air combat based on deep reinforcement learning[J]. IEEE Access, 2019, 8, 363- 378.
15	XU X M, YANG R N, ZHANG T, et al. Trajectory prediction of target aircraft in air combat based on GA-OIF-Elman neural network[C]//Proc.of the IEEE International Confe-rence on Artificial Intelligence and Computer Applications, 2019: 108-113.
16	丁达理, 魏政磊, 唐上钦, 等. 自适应预测权重的空战鲁棒机动决策方法[J]. 系统工程与电子技术, 2020, 42 (10): 2275- 2284. doi: 10.3969/j.issn.1001-506X.2020.10.16
	DING D L , WEI Z L , TANG S Q , et al. Robust maneuvering decision-making method for air combat using adaptive prediction weight[J]. Systems Engineering and Electronics, 2020, 42 (10): 2275- 2284. doi: 10.3969/j.issn.1001-506X.2020.10.16
17	杜海文, 崔明朗, 韩瑾, 等. 无人作战飞机近距空战机动决策[C]//首届兵器工程大会, 2017: 902-906.
	DU H W, CUI M L, HAN J, et al. Close air combat maneuver decision for UAVs[C]//Proc.of the 1st Weapons Engineering Conference, 2017: 902-906.
18	张宏鹏, 黄长强, 轩永波, 等. 基于深度神经网络的无人作战飞机自主空战机动决策[J]. 兵工学报, 2020, 41 (8): 1613- 1622. doi: 10.3969/j.issn.1000-1093.2020.08.016
	ZHANG H P , HUANG C Q , XUAN Y B , et al. Maneuver decision of autonomous air combat of unmanned combat aerial vehicle based on deep neural network[J]. Acta Armamentarii, 2020, 41 (8): 1613- 1622. doi: 10.3969/j.issn.1000-1093.2020.08.016
19	马文, 李辉, 王壮, 等. 基于深度随机博弈的近距空战机动决策[J]. 系统工程与电子技术, 2021, 43 (2): 443- 451.
	MA W , LI H , WANG Z , et al. Close air combat maneuver decision based on deep stochastic game[J]. Systems Engineering and Electronics, 2021, 43 (2): 443- 451.
20	余敏建, 嵇慧明, 韩其松, 等. 基于合作协同进化的多机空战目标分配[J]. 系统工程与电子技术, 2020, 42 (6): 1290- 1300.
	YU M J , JI H M , HAN Q S , et al. Multi-aircraft air combat target allocation based on cooperative co-evolutionary[J]. Systems Engineering and Electronics, 2020, 42 (6): 1290- 1300.
21	LI Z W , CHANG Y Z , KOU Y X , et al. Approach to WTA in air combat using IAFSA-IHS algorithm[J]. Journal of Systems Engineering and Electronics, 2018, 29 (3): 519- 529. doi: 10.21629/JSEE.2018.03.09
22	DINIZ J C M , DA R F , DA S E P , et al. Optimization of DPM-QAM transmitter using cooperative coevolutionary genetic algorithm[J]. Journal of Lightwave Technology, 2018, 36 (12): 2450- 2462. doi: 10.1109/JLT.2018.2815347
23	CAO Y , WEI W Y , BAI Y , et al. Multi-base multi-UAV cooperative reconnaissance path planning with genetic algorithm[J]. Cluster Computing, 2019, 22, 5175- 5184. doi: 10.1007/s10586-017-1132-9
24	奚之飞, 徐安, 寇英信, 等. 多机协同空战机动决策流程[J]. 系统工程与电子技术, 2020, 42 (2): 381- 389.
	XI Z F , XU A , KOU Y X , et al. Decision process of multi-aircraft cooperative air combat maneuver[J]. Systems Engineering and Electronics, 2020, 42 (2): 381- 389.
25	YANG Z , ZHOU D Y , PIAO H Y , et al. Evasive maneuver strategy for UCAV in beyond-visual-range air combat based on hierarchical multi-objective evolutionary algorithm[J]. IEEE Access, 2020, 8, 46605- 46623. doi: 10.1109/ACCESS.2020.2978883
26	李世豪, 丁勇, 高振龙. 基于直觉模糊博弈的无人机空战机动决策[J]. 系统工程与电子技术, 2019, 41 (5): 1063- 1070.
	LI S H , DING Y , GAO Z L . UAV air combat maneuvering decision based on intuitionistic fuzzy game theory[J]. Systems Engineering and Electronics, 2019, 41 (5): 1063- 1070.
27	RAMÍREZ L N , Z· BIKOWSKI R . Effectiveness of autonomous decision making for unmanned combat aerial vehicles in dogfight engagements[J]. Journal of Guidance, Control, and Dynamics, 2018, 41 (4): 1021- 1024. doi: 10.2514/1.G002937
28	黄长强, 赵克新, 韩邦杰, 等. 一种近似动态规划的无人机机动决策方法[J]. 电子与信息学报, 2018, 40 (10): 2447- 2452.
	HUANG C Q , ZHAO K X , HAN B J , et al. Maneuvering decision-making method of UAV based on approximate dynamic programming[J]. Journal of Electronics & Information Technology, 2018, 40 (10): 2447- 2452.
29	HUANG C Q , DONG K S , HUANG H Q , et al. Autonomous air combat maneuver decision using bayesian inference and moving horizon optimization[J]. Journal of Systems Engineering and Electronics, 2018, 29 (1): 86- 97. doi: 10.21629/JSEE.2018.01.09
30	CHEN Y F, SUN X P, LIU D J, et al. Optimal guidance method for ucav in close free air combat[C]//Proc.of the IEEE International Conferences on Ubiquitous Computing & Communications and Data Science and Computational Intelligence and Smart Computing, Networking and Services, 2019: 356-360.
31	左家亮, 张滢, 杨任农, 等. 中距协同空战决策过程二次聚类重构与评估[J]. 系统工程与电子技术, 2020, 42 (1): 108- 117.
	ZUO J L , ZHANG Y , YANG R N , et al. Reconstruction and evaluation of medium-range cooperation air combat decision-making process with two phase clustering[J]. Systems Engineering and Electronics, 2020, 42 (1): 108- 117.
32	陈斌, 王江, 王阳. 战斗机嵌入式训练系统中的智能虚拟陪练[J]. 航空学报, 2020, 41 (6): 523467.
	CHEN B , WANG J , WANG Y . Intelligent virtual training partner in embedded training system of fighter[J]. Acta Aeronautica et Astronautica Sinica, 2020, 41 (6): 523467.
33	WANG Z , LI H , WU H L , LI H , et al. Improving maneuver strategy in air combat by alternate freeze games with a deep reinforcement learning algorithm[J]. Mathematical Problems in Engineering, 2020, (1): 7180639.

均值	场次
均值	1	2	3	4	5	6	7	8	9	10
0.68	0.72	0.69	0.60	0.66	0.60	0.66	0.68	0.67	0.81	0.72

序列号	序列号
序列号	1	2	3	4	5	6	7	8	9	10
1	1	0	0.37	0	0.50	0	0.46	0.25	0.29	0.25
2	0	1	0.50	0.34	0.08	0.32	0	0	0.11	0.35
3	0.37	0.50	1	0.10	0.22	0.07	0.22	0	0.19	0.28
4	0	0.34	0.1	1	0.07	0.64	0	0	0.31	0.25
5	0.50	0.08	0.22	0.07	1	0.04	0.43	0.52	0.38	0.04
6	0	0.32	0.07	0.64	0.04	1	0	0	0.41	0.16
7	0.46	0	0.22	0	0.43	0	1	0.40	0.04	0
8	0.25	0	0	0	0.52	0	0.40	1	0	0
9	0.29	0.11	0.19	0.31	0.38	0.41	0.04	0	1	0.05
10	0.25	0.35	0.28	0.25	0.04	0.16	0	0	0.05	1

序列号	序列号
序列号	1	2	3	4	5	6	7	8	9	10
1	1	0	0.47	0	0	0	0.46	0.11	0	0.56
2	0	1	0.04	0.34	0.61	0.29	0.13	0.55	0.29	0.23
3	0.47	0.04	1	0	0.28	0	0.59	0.23	0	0.53
4	0	0.34	0	1	0.16	0.50	0	0.13	0.14	0.01
5	0	0.61	0.28	0.16	1	0.05	0.16	0.83	0.25	0.16
6	0	0.29	0	0.50	0.05	1	0	0.17	0.58	0
7	0.46	0.13	0.59	0	0.16	0	1	0.19	0	0.25
8	0.11	0.55	0.23	0.13	0.83	0.17	0.19	1	0.19	0.10
9	0	0.29	0	0.14	0.25	0.58	0	0.19	1	0
10	0.56	0.23	0.53	0.01	0.16	0.16	0.25	0.10	0	1

[1]	宋爽, 张悦, 张琳娜, 岑翼刚, 李浥东. 基于深度学习的轻量化目标检测算法[J]. 系统工程与电子技术, 2022, 44(9): 2716-2725.
[2]	聂倩, 杨丽花, 呼博, 任露露. 基扩展模型下基于LSTM神经网络的时变信道预测方法[J]. 系统工程与电子技术, 2022, 44(9): 2971-2977.
[3]	王健, 何自豪, 刘洁, 杨珂. 基于梯度域导向滤波器和改进PCNN的图像融合算法[J]. 系统工程与电子技术, 2022, 44(8): 2381-2392.
[4]	王彩云, 吴钇达, 王佳宁, 马璐, 赵焕玥. 基于改进的CNN和数据增强的SAR目标识别[J]. 系统工程与电子技术, 2022, 44(8): 2483-2487.
[5]	王冠, 茹海忠, 张大力, 马广程, 夏红伟. 弹性高超声速飞行器智能控制系统设计[J]. 系统工程与电子技术, 2022, 44(7): 2276-2285.
[6]	樊成, 王布宏, 田继伟. 基于多任务学习图卷积模型的航空网络节点分类[J]. 系统工程与电子技术, 2022, 44(7): 2341-2349.
[7]	金国栋, 薛远亮, 谭力宁, 许剑锟. 基于孪生神经网络的目标跟踪算法进展研究[J]. 系统工程与电子技术, 2022, 44(6): 1805-1822.
[8]	张佳唯, 钱凤臣, 杨俊强, 赵骞, 张峥嵘. 弹性光网络中路由与频谱分配算法综述[J]. 系统工程与电子技术, 2022, 44(6): 2001-2010.
[9]	韦娟, 杨皇卫, 宁方立. 基于NMF与CNN联合优化的声学场景分类[J]. 系统工程与电子技术, 2022, 44(5): 1433-1438.
[10]	陈冬, 句彦伟. 基于语义分割实现的SAR图像舰船目标检测[J]. 系统工程与电子技术, 2022, 44(4): 1195-1201.
[11]	张普, 薛惠锋, 高山, 左轩. 具有混合执行器故障的多智能体分布式有限时间自适应协同容错控制[J]. 系统工程与电子技术, 2022, 44(4): 1220-1229.
[12]	胥涯杰, 鲜勇, 李邦杰, 任乐亮, 李少朋, 郭玮林. 基于神经网络的高超声速飞行器惯导系统精度提高方法[J]. 系统工程与电子技术, 2022, 44(4): 1301-1309.
[13]	孙剑炜, 王超, 施庆展, 任文博, 尧泽昆, 袁乃昌. 基于遗传算法的相位调制波形智能优化[J]. 系统工程与电子技术, 2022, 44(3): 722-729.
[14]	方伟, 王玉, 闫文君, 林冲. 基于神经网络的符号化飞行动作识别[J]. 系统工程与电子技术, 2022, 44(3): 737-745.
[15]	孙晶明, 虞盛康, 孙俊. 基于深度学习的HRRP识别姿态敏感性分析[J]. 系统工程与电子技术, 2022, 44(3): 802-807.