面向LVC训练的蓝方虚拟实体近距空战决策建模

doi:10.12305/j.issn.1001-506X.2021.06.18

Abstract

Abstract:

The live-virtual-constructive (LVC) provides a strong support for the close-range air combat confrontation training. Aiming at the actual decision modeling requirements of blue-side virtual entity, based on the comparative analysis of deep reinforcement learning and classical intelligent optimization methods, the weight space and structure space of neural network are defined from the perspective of optimization theory, and the evolutionary neural network decision model based on intelligent optimization and its solution method are proposed. Firstly, the tactical characteristics of close-range air combat, fighter flight motion model and actual decision modeling requirements are analyzed. Secondly, the key flight state, action space and fitness function are designed to realize the end-to-end perception and decision of the blue-side. Finally, the decision model based on classical genetic neural network and its solution example are given. The results show that the proposed method can realize the function of learning the combat characteristics of the opponent through the confrontation data, and verify the effectiveness of the model and method; at the same time, it has universality for the current intelligent optimization and its improved algorithm, as well as different structure neural networks.

Key words: live-virtual-constructive (LVC), close-range air combat, decision model, neural network, intelligent optimization, blue-side of intelligence

Ang GAO, Zhiming DONG, Liang LI, Li DUAN, Qisheng GUO. Decision modeling of close-range air combat for LVC training in blue-side virtual entity[J]. Systems Engineering and Electronics, 2021, 43(6): 1606-1617.

Figures/Tables 14

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Fig.9

Fig.10

Table 1

Fig.11

Table 2

Table 3

References 33

1	BEST C , FLTLT B R . Science and technology enablers of live virtual constructive training in the air domain[J]. Air & Space Power Journal, 2018, 32 (4): 59- 73.
2	GREEN J , ZECKHAUSER R . Thunder versus lightning: a performance and cost analysis of the a-10 "warthog" versus the F-35 joint strike fighter[J]. Journal of Benefit-cost Analysis, 2019, 10 (3): 434- 468. doi: 10.1017/bca.2019.27
3	ARMON T, JAN J R, PIETER S. Rewarding air combat behavior in training simulations[C]//Proc.of the IEEE International Conference on Systems, Man, and Cybernetics, 2015: 1397-1402.
4	ARMON T, JAN J R, PIETER S. Transfer learning of air combat behavior[C]//Proc.of the IEEE 14th International Confe-rence on Machine Learning and Applications, 2015: 226-231.
5	RIJKEN R, TOUBMAN A. The future of autonomous air combat behavior[C]//Proc.of the IEEE International Conference on Systems, Man, and Cybernetics, 2016: 3089-3094.
6	BABAK T, FARZAD K, LINUS J L. Evaluating deep reinforcement learning for computer generated forces in ground combat simulation[C]//Proc.of the IEEE International Conference on Systems, Man, and Cybernetics, 2017: 3433-3438.
7	潘耀宗, 张健, 杨海涛, 等. 战机自主作战机动双网络智能决策方法[J]. 哈尔滨工业大学学报, 2019, 51 (11): 144- 151. doi: 10.11918/j.issn.0367-6234.201811083
	PAN Y Z , ZHANG J , YANG H T , et al. Dual network intelligent decision method for fighter autonomous combat maneuver[J]. Journal of Harbin Institute of Technology, 2019, 51 (11): 144- 151. doi: 10.11918/j.issn.0367-6234.201811083
8	ZHANG X B , LIU G Q , YANG C J , et al. Research on air confrontation maneuver decision-making method based on reinforcement learning[J]. Electronics, 2018, 7 (11): 279. doi: 10.3390/electronics7110279
9	邵彦昊, 朱荣刚, 贺建良, 等. 基于深度学习的不可逃逸区内的规避决策研究[J]. 电光与控制, 2019, 26 (11): 60- 64. doi: 10.3969/j.issn.1671-637X.2019.11.013
	SHAO Y H , ZHU R G , HE J L , et al. Evasive decision-making in inescapable areas based on deep learning[J]. Electronics Optics & Control, 2019, 26 (11): 60- 64. doi: 10.3969/j.issn.1671-637X.2019.11.013
10	周凯, 魏瑞轩, 张启瑞, 等. 基于经验移植的自主空战对抗学习方法[J]. 航空学报, 2020, 40 (X): 724285.
	ZHOU K , WEI R X , ZHANG Q R , et al. Learning method based on experience transfer for autonomous air combat[J]. Acta Aeronautica et Astronautica Sinica, 2020, 40 (X): 724285.
11	何金, 丁勇, 高振龙. 基于Double Deep Q Network的无人机隐蔽接敌策略[J]. 电光与控制, 2020, 27 (7): 52- 57. doi: 10.3969/j.issn.1671-637X.2020.07.010
	HE J , DING Y , GAO Z L . A stealthy engagement maneuvering strategy of UAV based on Double Deep Q Network[J]. Electronics Optics & Control, 2020, 27 (7): 52- 57. doi: 10.3969/j.issn.1671-637X.2020.07.010
12	ZHOU K , WEI R X , ZHANG Q R , et al. Learning system for air combat decision inspired by cognitive mechanisms of the brain[J]. IEEE Access, 2020, 8, 8129- 8144. doi: 10.1109/ACCESS.2020.2964031
13	DEWANGGA W, SASONGKO R A. Air combat maneuver guidance system for UCAV Using neutral-advantage-disadvantage-mutual disadvantage approach[C]//Proc.of the IEEE 6th Asian Conference on Defence Technology, 2019: 155-161.
14	YANG Q M , ZHANG J D , SHI G Q , et al. Maneuver decision of UAV in short-range air combat based on deep reinforcement learning[J]. IEEE Access, 2019, 8, 363- 378.
15	XU X M, YANG R N, ZHANG T, et al. Trajectory prediction of target aircraft in air combat based on GA-OIF-Elman neural network[C]//Proc.of the IEEE International Confe-rence on Artificial Intelligence and Computer Applications, 2019: 108-113.
16	丁达理, 魏政磊, 唐上钦, 等. 自适应预测权重的空战鲁棒机动决策方法[J]. 系统工程与电子技术, 2020, 42 (10): 2275- 2284. doi: 10.3969/j.issn.1001-506X.2020.10.16
	DING D L , WEI Z L , TANG S Q , et al. Robust maneuvering decision-making method for air combat using adaptive prediction weight[J]. Systems Engineering and Electronics, 2020, 42 (10): 2275- 2284. doi: 10.3969/j.issn.1001-506X.2020.10.16
17	杜海文, 崔明朗, 韩瑾, 等. 无人作战飞机近距空战机动决策[C]//首届兵器工程大会, 2017: 902-906.
	DU H W, CUI M L, HAN J, et al. Close air combat maneuver decision for UAVs[C]//Proc.of the 1st Weapons Engineering Conference, 2017: 902-906.
18	张宏鹏, 黄长强, 轩永波, 等. 基于深度神经网络的无人作战飞机自主空战机动决策[J]. 兵工学报, 2020, 41 (8): 1613- 1622. doi: 10.3969/j.issn.1000-1093.2020.08.016
	ZHANG H P , HUANG C Q , XUAN Y B , et al. Maneuver decision of autonomous air combat of unmanned combat aerial vehicle based on deep neural network[J]. Acta Armamentarii, 2020, 41 (8): 1613- 1622. doi: 10.3969/j.issn.1000-1093.2020.08.016
19	马文, 李辉, 王壮, 等. 基于深度随机博弈的近距空战机动决策[J]. 系统工程与电子技术, 2021, 43 (2): 443- 451.
	MA W , LI H , WANG Z , et al. Close air combat maneuver decision based on deep stochastic game[J]. Systems Engineering and Electronics, 2021, 43 (2): 443- 451.
20	余敏建, 嵇慧明, 韩其松, 等. 基于合作协同进化的多机空战目标分配[J]. 系统工程与电子技术, 2020, 42 (6): 1290- 1300.
	YU M J , JI H M , HAN Q S , et al. Multi-aircraft air combat target allocation based on cooperative co-evolutionary[J]. Systems Engineering and Electronics, 2020, 42 (6): 1290- 1300.
21	LI Z W , CHANG Y Z , KOU Y X , et al. Approach to WTA in air combat using IAFSA-IHS algorithm[J]. Journal of Systems Engineering and Electronics, 2018, 29 (3): 519- 529. doi: 10.21629/JSEE.2018.03.09
22	DINIZ J C M , DA R F , DA S E P , et al. Optimization of DPM-QAM transmitter using cooperative coevolutionary genetic algorithm[J]. Journal of Lightwave Technology, 2018, 36 (12): 2450- 2462. doi: 10.1109/JLT.2018.2815347
23	CAO Y , WEI W Y , BAI Y , et al. Multi-base multi-UAV cooperative reconnaissance path planning with genetic algorithm[J]. Cluster Computing, 2019, 22, 5175- 5184. doi: 10.1007/s10586-017-1132-9
24	奚之飞, 徐安, 寇英信, 等. 多机协同空战机动决策流程[J]. 系统工程与电子技术, 2020, 42 (2): 381- 389.
	XI Z F , XU A , KOU Y X , et al. Decision process of multi-aircraft cooperative air combat maneuver[J]. Systems Engineering and Electronics, 2020, 42 (2): 381- 389.
25	YANG Z , ZHOU D Y , PIAO H Y , et al. Evasive maneuver strategy for UCAV in beyond-visual-range air combat based on hierarchical multi-objective evolutionary algorithm[J]. IEEE Access, 2020, 8, 46605- 46623. doi: 10.1109/ACCESS.2020.2978883
26	李世豪, 丁勇, 高振龙. 基于直觉模糊博弈的无人机空战机动决策[J]. 系统工程与电子技术, 2019, 41 (5): 1063- 1070.
	LI S H , DING Y , GAO Z L . UAV air combat maneuvering decision based on intuitionistic fuzzy game theory[J]. Systems Engineering and Electronics, 2019, 41 (5): 1063- 1070.
27	RAMÍREZ L N , Z· BIKOWSKI R . Effectiveness of autonomous decision making for unmanned combat aerial vehicles in dogfight engagements[J]. Journal of Guidance, Control, and Dynamics, 2018, 41 (4): 1021- 1024. doi: 10.2514/1.G002937
28	黄长强, 赵克新, 韩邦杰, 等. 一种近似动态规划的无人机机动决策方法[J]. 电子与信息学报, 2018, 40 (10): 2447- 2452.
	HUANG C Q , ZHAO K X , HAN B J , et al. Maneuvering decision-making method of UAV based on approximate dynamic programming[J]. Journal of Electronics & Information Technology, 2018, 40 (10): 2447- 2452.
29	HUANG C Q , DONG K S , HUANG H Q , et al. Autonomous air combat maneuver decision using bayesian inference and moving horizon optimization[J]. Journal of Systems Engineering and Electronics, 2018, 29 (1): 86- 97. doi: 10.21629/JSEE.2018.01.09
30	CHEN Y F, SUN X P, LIU D J, et al. Optimal guidance method for ucav in close free air combat[C]//Proc.of the IEEE International Conferences on Ubiquitous Computing & Communications and Data Science and Computational Intelligence and Smart Computing, Networking and Services, 2019: 356-360.
31	左家亮, 张滢, 杨任农, 等. 中距协同空战决策过程二次聚类重构与评估[J]. 系统工程与电子技术, 2020, 42 (1): 108- 117.
	ZUO J L , ZHANG Y , YANG R N , et al. Reconstruction and evaluation of medium-range cooperation air combat decision-making process with two phase clustering[J]. Systems Engineering and Electronics, 2020, 42 (1): 108- 117.
32	陈斌, 王江, 王阳. 战斗机嵌入式训练系统中的智能虚拟陪练[J]. 航空学报, 2020, 41 (6): 523467.
	CHEN B , WANG J , WANG Y . Intelligent virtual training partner in embedded training system of fighter[J]. Acta Aeronautica et Astronautica Sinica, 2020, 41 (6): 523467.
33	WANG Z , LI H , WU H L , LI H , et al. Improving maneuver strategy in air combat by alternate freeze games with a deep reinforcement learning algorithm[J]. Mathematical Problems in Engineering, 2020, (1): 7180639.

均值	场次
均值	1	2	3	4	5	6	7	8	9	10
0.68	0.72	0.69	0.60	0.66	0.60	0.66	0.68	0.67	0.81	0.72

序列号	序列号
序列号	1	2	3	4	5	6	7	8	9	10
1	1	0	0.37	0	0.50	0	0.46	0.25	0.29	0.25
2	0	1	0.50	0.34	0.08	0.32	0	0	0.11	0.35
3	0.37	0.50	1	0.10	0.22	0.07	0.22	0	0.19	0.28
4	0	0.34	0.1	1	0.07	0.64	0	0	0.31	0.25
5	0.50	0.08	0.22	0.07	1	0.04	0.43	0.52	0.38	0.04
6	0	0.32	0.07	0.64	0.04	1	0	0	0.41	0.16
7	0.46	0	0.22	0	0.43	0	1	0.40	0.04	0
8	0.25	0	0	0	0.52	0	0.40	1	0	0
9	0.29	0.11	0.19	0.31	0.38	0.41	0.04	0	1	0.05
10	0.25	0.35	0.28	0.25	0.04	0.16	0	0	0.05	1

序列号	序列号
序列号	1	2	3	4	5	6	7	8	9	10
1	1	0	0.47	0	0	0	0.46	0.11	0	0.56
2	0	1	0.04	0.34	0.61	0.29	0.13	0.55	0.29	0.23
3	0.47	0.04	1	0	0.28	0	0.59	0.23	0	0.53
4	0	0.34	0	1	0.16	0.50	0	0.13	0.14	0.01
5	0	0.61	0.28	0.16	1	0.05	0.16	0.83	0.25	0.16
6	0	0.29	0	0.50	0.05	1	0	0.17	0.58	0
7	0.46	0.13	0.59	0	0.16	0	1	0.19	0	0.25
8	0.11	0.55	0.23	0.13	0.83	0.17	0.19	1	0.19	0.10
9	0	0.29	0	0.14	0.25	0.58	0	0.19	1	0
10	0.56	0.23	0.53	0.01	0.16	0.16	0.25	0.10	0	1

[1]	Shuang SONG, Yue ZHANG, Linna ZHANG, Yigang CEN, Yidong LI. Lightweight target detection algorithm based on deep learning [J]. Systems Engineering and Electronics, 2022, 44(9): 2716-2725.
[2]	Qian NIE, Lihua YANG, Bo HU, Lulu REN. Time-varying channel prediction method based on LSTM neural networks under basis expansion model [J]. Systems Engineering and Electronics, 2022, 44(9): 2971-2977.
[3]	Jian WANG, Zihao HE, Jie LIU, Ke YANG. Image fusion algorithm based on gradient domain guided filtering and improved PCNN [J]. Systems Engineering and Electronics, 2022, 44(8): 2381-2392.
[4]	Caiyun WANG, Yida WU, Jianing WANG, Lu MA, Huanyue ZHAO. SAR image target recognition based on combinatorial optimization convolutional neural network [J]. Systems Engineering and Electronics, 2022, 44(8): 2483-2487.
[5]	Guan WANG, Haizhong RU, Dali ZHANG, Guangcheng MA, Hongwei XIA. Design of intelligent control system for flexible hypersonic vehicle [J]. Systems Engineering and Electronics, 2022, 44(7): 2276-2285.
[6]	Cheng FAN, Buhong WANG, Jiwei TIAN. Node classification of airline network based on the graph convolution network model with multi-task learning [J]. Systems Engineering and Electronics, 2022, 44(7): 2341-2349.
[7]	Guodong JIN, Yuanliang XUE, Lining TAN, Jiankun XU. Advances in object tracking algorithm based on siamese network [J]. Systems Engineering and Electronics, 2022, 44(6): 1805-1822.
[8]	Jiawei ZHANG, Fengchen QIAN, Junqiang YANG, Qian ZHAO, Zhengrong ZHANG. Survey on routing and spectrum allocation algorithm in elastic optical networks [J]. Systems Engineering and Electronics, 2022, 44(6): 2001-2010.
[9]	Juan WEI, Huangwei YANG, Fangli NING. Acoustic scene classification based on joint optimization of NMF and CNN [J]. Systems Engineering and Electronics, 2022, 44(5): 1433-1438.
[10]	Dong CHEN, Yanwei JU. Ship object detection SAR images based on semantic segmentation [J]. Systems Engineering and Electronics, 2022, 44(4): 1195-1201.
[11]	Pu ZHANG, Huifeng XUE, Shan GAO, Xuan ZUO. Distributed finite-time adaptive cooperative fault-tolerant control for multi-agent systems with integrated actuators faults [J]. Systems Engineering and Electronics, 2022, 44(4): 1220-1229.
[12]	Yajie XU, Yong XIAN, Bangjie LI, Leliang REN, Shaopeng LI, Weilin GUO. Method for improving the precision of hypersonic vehicle inertial navigation system based on neural network [J]. Systems Engineering and Electronics, 2022, 44(4): 1301-1309.
[13]	Jianwei SUN, Chao WANG, Qingzhan SHI, Wenbo REN, Zekun YAO, Naichang YUAN. Intelligent optimization of phase-modulation waveform based on genetic algorithm [J]. Systems Engineering and Electronics, 2022, 44(3): 722-729.
[14]	Wei FANG, Yu WANG, Wenjun YAN, Chong LIN. Symbolized flight action recognition based on neural network [J]. Systems Engineering and Electronics, 2022, 44(3): 737-745.
[15]	Jingming SUN, Shengkang YU, Jun SUN. Pose sensitivity analysis of HRRP recognition based on deep learning [J]. Systems Engineering and Electronics, 2022, 44(3): 802-807.

Decision modeling of close-range air combat for LVC training in blue-side virtual entity

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 14

References 33

Related Articles 15

Recommended Articles

Metrics

Comments