基于A3C的多功能雷达认知干扰决策方法

doi:10.12305/j.issn.1001-506X.2023.01.11

Abstract

Abstract:

In the field of multifunctional radar countermeasures, the current cognitive jamming decision-making method based on reinforcement learning theory is difficult to meet the high real-time requirements of radar countermeasures. Therefore, we apply asynchronous advantage actor-critic (A3C) to the field of cognitive jamming decision-making, design the cognitive jamming decision-making overall framework including the jammer model, the environment model (the target multifunctional radar) and its interaction mechanism, and develop a decision-making process, the jammer model uses asynchronous multi-threaded way to interact with the environment model. Simulation results show that on the basis of expanding the radar task transformation relationship table, compared with the cognitive jamming decision-making series methods based on deep Q network (DQN), theproposedmethod can significantly improve time efficiency, decreases the average decision-making time by more than 70%, and has obvious advantages in decision-making degree of accuracy, it shows that theproposed method can provide more powerful technical support for multifunctional radar countermeasures decision-making.

Key words: jamming decision-making, asynchronous advantage, actor-critic, time efficiency, decision-making degree of accuracy

CLC Number:

TN974

Weiqi ZOU, Chaoyang NIU, Wei LIU, Ouyang GAO, Haobo ZHANG. Cognitive jamming decision-making method against multifunctional radar based on A3C[J]. Systems Engineering and Electronics, 2023, 45(1): 86-92.

Figures/Tables 8

Fig.1

Fig.2

Fig.3

Table 1

Table 2

Table 3

Fig.4

Fig.5

References 15

1	FARINA A, TIMMONERI L. Live data test of electronic counter counter measures (ECCM) on a multifunctional prototype radar[C]// Proc. of the IEEE Metrology for Aerospace, 2016.
2	KOWALSKI E, CONWAY D, MORRIS A, et al. Multifunction phased array radar advanced technology demonstrator (MPAR ATD) nearfield testing and fielding[C]//Proc. of the IEEE Radar Conference, 2019.
3	周脉成. 基于博弈论的雷达干扰决策技术研究[D]. 西安: 西安电子科技大学, 2014.
	ZHOU M C. Research on radar jamming decision technology based on game theory[D]. Xi'an: Xidian University, 2014.
4	孙宏伟, 童宁宁, 孙富君. 基于D-S证据理论的电子干扰模式选择[J]. 弹箭与制导学报, 2003, 23 (2): 218- 220.
	SUN H W , TONG N N , SUN F J . Jamming design selection based on D-S theory[J]. Journal of Projectiles, Rockets, Missiles and Guidance, 2003, 23 (2): 218- 220.
5	李云杰, 朱云鹏, 高梅国. 基于Q-学习算法的认知雷达对抗过程设计[J]. 北京理工大学学报, 2015, 35 (11): 1194- 1199.
	LI Y J , ZHU Y P , GAO M G . Design of cognitive radar jamming based on Q-learning algorithm[J]. Transactions of Beijing Institute of Technology, 2015, 35 (11): 1194- 1199.
6	MOUSAVI S S, SCHUKAT M, HOWLEY E. Deep reinforcement learning: an overview[C]//Proc. of the SAI Intelligent Systems Conference, 2016: 426-440.
7	邢强, 贾鑫, 朱卫纲. 基于Q-学习的智能雷达对抗[J]. 系统工程与电子技术, 2018, 40 (5): 1031- 1035.
	XING Q , JIA X , ZHU W G . Intelligent radar countermeasure based on Q-learning[J]. Systems Engineering and Electronics, 2018, 40 (5): 1031- 1035.
8	张柏开, 朱卫纲. 基于Q-Learning的多功能雷达认知干扰决策方法[J]. 电讯技术, 2020, 60 (2): 129- 136.
	ZHANG B K , ZHU W G . A cognitive jamming decision method for multi-functional radar based on Q-learning[J]. Telecommunication Engineering, 2020, 60 (2): 129- 136.
9	张柏开, 朱卫纲. 对多功能雷达的DQN认知干扰决策方法[J]. 系统工程与电子技术, 2020, 42 (4): 819- 825.
	ZHANG B K , ZHU W G . DQN based decision-making method of cognitive jamming against multifunction radar[J]. Systems Engineering and Electronics, 2020, 42 (4): 819- 825.
10	MNIH V, BADIA A P, MIRZA M, et al. Asynchronous methods for deep reinforcement learning[C]//Proc. of the International Conference on Machine Learning, 2016: 1928-1937.
11	HUSSEIN A , ELYAN E , GABER M , et al. Deep imitation learning for 3D navigation tasks[J]. Neural Computing and Applications, 2018, 29 (7): 389- 404.
12	GENDERS W , RAZAVI S . Evaluating reinforcement learning state representations for adaptive traffic signal control[J]. Procedia Computer Science, 2018, 130, 26- 33.
13	KONDA V R , TSITSIKLIS J N . Onactor-critic algorithms[J]. SIAM Journal on Control and Optimization, 2003, 42 (4): 1143- 1166.
14	马爽. 多功能雷达电子情报信号处理关键技术研究[D]. 长沙: 国防科技大学, 2013.
	MA S. Research on ELINT signal processing key technologies for multifunction radar[D]. Changsha: National University of Defense Technology, 2013.
15	张柏开, 朱卫纲. MFR认知干扰决策体系构建及关键技术[J]. 系统工程与电子技术, 2020, 42 (9): 1969- 1975.
	ZHANG B K , ZHU W G . Construction and key technologies of cognitive jamming decision-making system against MFR[J]. Systems Engineering and Electronics, 2020, 42 (9): 1969- 1975.

功能层	任务层
功能层	1	2	3	4
T₁	T₁^A	-	-	-
T₂	T₂^A	T₂^B	-	-
T₃	T₃^A	T₃^B	T₃^C	-
				

雷达任务状态	干扰样式
雷达任务状态	1	2	3	4	5	6	7	8	9	10	11	12
T₁^A	-	T₂^B	-	T₂^A	-	T₁^A	-	-	-	T₁^A	-	-
T₂^A	T₁^A	-	-	-	T₂^A	T₃^A	-	T₃^B	-	-	-	-
T₂^B	T₁^A	-	-	-	T₂^B	-	T₃^C	-	T₃^A	-	-	-
T₃^A~T₁₃^C												
T₁₄^A	-	T₁₅^A	-	-	T₁₃^B	-	-	T₁₄^A	-	-	-	-
T₁₄^B	-	-	-	T₁₃^C	-	T₁₄^B	-	-	T₁₃^B	-	-	T₁₅^A
T₁₅^A	-	-	-	-	-	-	-	-	-	-	-	-

认知干扰决策方法	平均每回合所用时间/ms
DQN	160.579 5
Double DQN	77.591 8
Prioritized Replay DQN	127.440 2
A3C	2.654 2

[1]	Chen LI, Yanyan HUANG, Yongliang ZHANG, Tiande CHEN. Multi-agent decision-making method based on Actor-Critic framework and its application in wargame [J]. Systems Engineering and Electronics, 2021, 43(3): 755-762.
[2]	Bokai ZHANG, Weigang ZHU. Construction and key technologies of cognitive jamming decision-making system against MFR [J]. Systems Engineering and Electronics, 2020, 42(9): 1969-1975.
[3]	Bokai ZHANG, Weigang ZHU. DQN based decision-making method of cognitive jamming against multifunctional radar [J]. Systems Engineering and Electronics, 2020, 42(4): 819-825.

Cognitive jamming decision-making method against multifunctional radar based on A3C

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 8

References 15

Related Articles 3

Recommended Articles

Metrics

Comments