基于马尔可夫的多功能雷达认知干扰决策建模研究

doi:10.12305/j.issn.1001-506X.2022.08.13

系统工程与电子技术 ›› 2022, Vol. 44 ›› Issue (8): 2488-2497.doi: 10.12305/j.issn.1001-506X.2022.08.13

基于马尔可夫的多功能雷达认知干扰决策建模研究

朱霸坤^1,^2,*, 朱卫纲², 李伟³, 杨莹³, 高天昊³

1. 航天工程大学电子光学工程系, 北京 101416
2. 电子信息系统复杂电磁环境效应国家重点实验室, 河南洛阳 471032
3. 航天工程大学研究生院, 北京 101416

收稿日期:2021-06-01 出版日期:2022-08-01 发布日期:2022-08-24
通讯作者: 朱霸坤
作者简介:朱霸坤 (1997—), 男, 硕士研究生, 主要研究方向为认知电子战、雷达对抗|朱卫纲 (1973—), 女, 教授, 博士, 主要研究方向为现代信号处理、空间信息对抗、认知电子战|李伟 (1994—), 男, 硕士研究生, 主要研究方向为雷达辐射源识别|杨莹 (1997—), 女, 硕士研究生, 主要研究方向为雷达信号处理|高天昊 (1997—), 男, 硕士研究生, 主要研究方向为雷达辐射源识别
基金资助:
CEMEE国家重点实验室项目(CEMEE2020Z0203B)

Research on decision-making modeling of cognitive jamming for multi-functional radar based on Markov

Bakun ZHU^1,^2,*, Weigang ZHU², Wei LI³, Ying YANG³, Tianhao GAO³

1. Department of Electronic and Optical Engineering, Space Engineering University, Beijing 101416, China
2. State Key Laboratory of Complex Electromagnetic Environment Effects on Electronics and Information System, Luoyang 471032, China
3. Graduate School, Space Engineering University, Beijing 101416, China

Received:2021-06-01 Online:2022-08-01 Published:2022-08-24
Contact: Bakun ZHU

摘要/Abstract

摘要：

多功能雷达是现代电磁战场上不可或缺的重要装备, 针对多功能雷达的干扰一直是一个难题。本文在研究多功能雷达信号特点和雷达对抗过程的基础上, 提出了雷达状态联合表征的方法, 将多功能雷达的干扰决策问题建模为一个带收益的马尔可夫决策过程, 设计了认知干扰决策系统, 并通过基于Q-Learning的认知干扰决策算法求解该模型下的最佳干扰策略。通过仿真实验, 证明了基于Q-Learning的认知干扰决策算法能够在缺乏先验经验的情况下学习到最佳干扰策略, 具备“认知”的特性, 并且在不稳定的环境中也具有较强的适应性, 有效支撑了本文所提的干扰决策模型。

关键词: 雷达对抗, 马尔可夫决策过程, 雷达状态, 强化学习, Q-Learning

Abstract:

Multi-functional radar is an indispensable and important equipment in modern electromagnetic battlefield. The interference of multi-functional radar is always a difficult problem. In this paper, based on the study of the characteristics of multi-functional radar signal and the radar countermeasure process, the method of joint representation of radar state is proposed, and the interference problem of multi-functional radar is modeled as a Markov decision process with benefits. The cognitive interference decision system is designed. The interference strategy is solved by the cognitive interference decision algorithm based on Q-learning. Through the simulation experiment, it is proved that the cognitive interference decision algorithm based on Q-learning can learn the optimal interference strategy in the absence of prior experience, have the characteristic of "cognition", and have strong adaptability in the unstable environment, which effectively supports the interference decision model mentioned above.

Key words: radar confrontation, Markov decision process, radar state, reinforcement learning, Q-learning

中图分类号:

TN974

朱霸坤, 朱卫纲, 李伟, 杨莹, 高天昊. 基于马尔可夫的多功能雷达认知干扰决策建模研究[J]. 系统工程与电子技术, 2022, 44(8): 2488-2497.

Bakun ZHU, Weigang ZHU, Wei LI, Ying YANG, Tianhao GAO. Research on decision-making modeling of cognitive jamming for multi-functional radar based on Markov[J]. Systems Engineering and Electronics, 2022, 44(8): 2488-2497.

图/表 16

图1

图2

图3

图4

图5

图6

图7

图8

图9

图10

图11

图12

表1

图13

图14

表2

参考文献 32

1	王沙飞, 李岩, 徐迈, 等. 认知电子战原理与技术[M]. 北京: 国防工业出版社, 2018: 10- 11.
	WANG S F , LI Y , XU M , et al. Principle and technology of cognitive electronic warfare[M]. Beijing: National Defense Industry Press, 2018: 10- 11.
2	GONG L L, LUO J Q, WU S L. A radar emitter identification method based on pulse match template sequence[C]//Proc. of the IEEE 2nd International Conference on Signal Processing Systems, 2010, 3: 153-156.
3	周脉成. 基于博弈论的雷达干扰决策技术研究[D]. 西安: 西安电子科技大学, 2014: 29-32.
	ZHOU M C. Research on radar jamming decision technology based on game theory[D]. Xi'an: Xidian University, 2014: 29-32.
4	BACHMANN D J , EVANS R J , MORAN B . Game theoretic analysis of adaptive radar jamming[J]. IEEE Trans.on Aerospace and Electronic Systems, 2011, 47 (2): 1081- 1100. doi: 10.1109/TAES.2011.5751244
5	HE B, SU H T. Joint power allocation and beamforming between a multistatic radar and jammer based on game theory[C]// Proc. of the IEEE 11th International Conference on Communication Software and Networks, 2019: 337-341.
6	孙宏伟, 童宁宁, 孙富君. 基于D-S证据理论的电子干扰模式选择[J]. 弹箭与制导学报, 2003, (S2): 218- 220.
	SUN H W , TONG N N , SUN F J . Jamming design selection based on D-S theory[J]. Journal of Projectiles, Rockets, Missiles and Guidance, 2003, (S2): 218- 220.
7	TANG Z , GAO X G . Research on the self-defence electronic jamming decision-making based on the discrete dynamic Bayesian network[J]. Journal of Systems Engineering and Electronics, 2008, 19 (4): 702- 708. doi: 10.1016/S1004-4132(08)60142-5
8	SILVER D , SCHRITTWIESER J L , SIMONYAN K , et al. Mastering the game of go without human knowledge[J]. Nature, 2017, 550 (7676): 354- 359. doi: 10.1038/nature24270
9	YOO J , JANG D , KIM H J , et al. Hybrid reinforcement learning control for a micro quadrotor flight[J]. IEEE Control Systems Letters, 2020, 5 (2): 505- 510.
10	GUO X X , YAN W S , CUI R X . Reinforcement learning-based nearly optimal control for constrained-input partially unknown systems using differentiator[J]. IEEE Trans.on Neural Networks and Learning Systems, 2019, 31 (11): 4713- 4725.
11	ZHAO Z Y, WANG Q, LI X L. Deep reinforcement learning based lane detection and localization[EB/OL]. [2021-05-20]. https://www.sciencedirect.com/science/article/pii/S0925231220310833.
12	PARK H , SIM M K , CHOI D G . An intelligent financial portfolio trading strategy using deep Q-learning[J]. Expert Systems with Applications, 2020, 158, 113573. doi: 10.1016/j.eswa.2020.113573
13	MANDOW L , PÉREZ-DE-LA-CRUZ J L , RODRÍGUEZ-GAVILÁN A B , et al. Architectural planning with shape grammars and reinforcement learning: habitability and energy efficiency[J]. Engineering Applications of Artificial Intelligence, 2020, 96, 103909. doi: 10.1016/j.engappai.2020.103909
14	邢强, 贾鑫, 朱卫纲. 基于Q-学习的智能雷达对抗[J]. 系统工程与电子技术, 2018, 40 (5): 1031- 1035.
	XING Q , JIA X , ZHU W G . Intelligent radar countermeasure based on Q-learning[J]. Systems Engineering and Electronics, 2018, 40 (5): 1031- 1035.
15	李云杰, 朱云鹏, 高梅国. 基于Q-学习算法的认知雷达对抗过程设计[J]. 北京理工大学学报, 2015, 35 (11): 1194- 1199.
	LI Y J , ZHU Y P , GAO M G . Design of cognitive radar jamming basedon Q-learning algorithm[J]. Transactions of Beijing Institute of Technology, 2015, 35 (11): 1194- 1199.
16	QIANG X, ZHU W G, XIN J. Research on method of intelligent radar confrontation based on reinforcement learning[C]//Proc. of the IEEE 2nd International Conference on Computational Intelligence and Applications, 2017.
17	XING Q, ZHU W G, JIA X. Intelligent countermeasure design of radar working-modes unknown[C]//Proc. of the IEEE International Conference on Signal Processing, Communications and Computing, 2017.
18	张柏开, 朱卫纲. 对多功能雷达的DQN认知干扰决策方法[J]. 系统工程与电子技术, 2020, 42 (4): 819- 825.
	ZHNAG B K , ZHU W G . DQN based decision-making method of cognitive jamming against multifunctional radar[J]. Systems Engineering and Electronics, 2020, 42 (4): 819- 825.
19	CHARLISH A. Autonomous agents for multi-function radar resource management[D]. London: University College London, 2011.
20	张光义. 相控阵雷达技术[M]. 北京: 国防工业出版社, 2009: 30- 32.
	ZHANG G Y . Phased array radar technology[M]. Beijing: National Defense Industry Press, 2009: 30- 32.
21	VISNEVSKI N, KRISHNAMURTHY V, HAYKIN S, et al. Multi-function radar emitter modelling: a stochastic discrete event system approach[C]//Proc. of the IEEE Conference on Decision & Control, 2004.
22	VISNEVSKI N , KRISHNAMURTHY V , WANG A , et al. Syntactic modeling and signal processing of multifunction radars: a stochastic context-free grammar approach[J]. Proceedings of the IEEE, 2007, 95 (5): 1000- 1025. doi: 10.1109/JPROC.2007.893252
23	WANG A , KRISHNAMURTHY V . Signal interpretation of multifunction radars: modeling and statistical signal processing with stochastic context-free grammar[J]. IEEE Trans.on Signal Processing, 2008, 56 (3): 1106- 1109. doi: 10.1109/TSP.2007.908949
24	欧健. 多功能雷达行为辨识与预测技术研究[D]. 长沙: 国防科技大学, 2017: 20-30
	OU J. Research on behavior recognition and prediction techniques against multi-function radar[D]. Changsha: National University of Defence Technology, 2017: 20-30.
25	林石, 伍燕平, 肖进. 认知雷达对抗中的未知雷达状态识别方法分析[J]. 电子技术与软件工程, 2020, (13): 65- 66.
	LIN S , WU Y P , XIAO J . Analysis of unknown radar state recognition method in cognitive radar countermeasure[J]. Electronic Technology & Software Engineering, 2020, (13): 65- 66.
26	LIU Z M . Recognition of multifunction radars via hierarchically mining and exploiting pulse group patterns[J]. IEEE Trans.on Aerospace and Electronic Systems, 2020, 56 (6): 4659- 4672. doi: 10.1109/TAES.2020.2999163
27	杨秋, 顾杰, 魏平. 利用幅度重排的机载火控雷达工作模式识别方法[J]. 西安电子科技大学学报, 2021, 48 (2): 42- 48.
	YANG Q , GU J , WEI P . Recognition of operation model of airborne fire control radar using amplitude rearrangement[J]. Journal of Xidian University, 2021, 48 (2): 42- 48.
28	王勇军. 一种改进的事件驱动的MFR雷达字提取方法[J]. 现代雷达, 2019, 41 (3): 17- 20.17-20, 26
	WANG Y J . Novel approach of radar word extraction for MFRs based on event-drive method[J]. Modern Radar, 2019, 41 (3): 17- 20.17-20, 26
29	刘海军, 樊昀, 李悦, 等. 多功能雷达建模中的雷达字提取技术研究[J]. 国防科技大学学报, 2010, 32 (2): 91- 96. doi: 10.3969/j.issn.1001-2486.2010.02.017
	LIU H J , FAN J , LI R , et al. Research on extracting of radar words in modeling of multi-function radar[J]. Journal of National University of Defense Technology, 2010, 32 (2): 91- 96. doi: 10.3969/j.issn.1001-2486.2010.02.017
30	MIRANDA S , BAKER C , WOODBRIDGE K , et al. Know- ledge-based resource management for multifunction radar: a look at scheduling and task prioritization[J]. IEEE Signal Processing Magazine, 2006, 23 (1): 66- 76. doi: 10.1109/MSP.2006.1593338
31	MIRANDA S L C , BAKER C J , WOODBRIDGE K , et al. Comparison of scheduling algorithms for multifunction radar[J]. IET Radar, Sonar & Navigation, 2007, 1 (6): 414- 424.
32	张友益, 徐才宏. 雷达对抗及反对抗作战能力评估与验证[M]. 北京: 国防工业出版社, 2019: 10- 11.
	ZHANG Y Y , XU C H . Evaluation and verification of radar countermeasure and countermeasure[M]. Beijing: National Defense Industry Press, 2019: 10- 11.

参数	取值	名称
α	0.01	学习率
γ	0.95	折扣因子
ε	0.1	探索率

μ值	0.3	0.5	0.7	0.9	1
算法收敛时的步数	16.8	10.4	7.2	5.9	5.2

[1]	王冠, 茹海忠, 张大力, 马广程, 夏红伟. 弹性高超声速飞行器智能控制系统设计[J]. 系统工程与电子技术, 2022, 44(7): 2276-2285.
[2]	孟泠宇, 郭秉礼, 杨雯, 张欣伟, 赵柞青, 黄善国. 基于深度强化学习的网络路由优化方法[J]. 系统工程与电子技术, 2022, 44(7): 2311-2318.
[3]	郭冬子, 黄荣, 许河川, 孙立伟, 崔乃刚. 再入飞行器深度确定性策略梯度制导方法研究[J]. 系统工程与电子技术, 2022, 44(6): 1942-1949.
[4]	韩明仁, 王玉峰. 基于强化学习的全电推进卫星变轨优化方法[J]. 系统工程与电子技术, 2022, 44(5): 1652-1661.
[5]	谭诗翰, 金凤林, 顿聪颖. 面向用户需求的空天地一体化车载网络任务分配策略[J]. 系统工程与电子技术, 2022, 44(5): 1717-1727.
[6]	何立, 沈亮, 李辉, 王壮, 唐文泉. 强化学习中的策略重用: 研究进展[J]. 系统工程与电子技术, 2022, 44(3): 884-899.
[7]	朱霸坤, 朱卫纲, 李伟, 杨莹, 高天昊. 基于先验知识的多功能雷达智能干扰决策方法[J]. 系统工程与电子技术, 2022, 44(12): 3685-3695.
[8]	杨清清, 高盈盈, 郭玙, 夏博远, 杨克巍. 基于深度强化学习的海战场目标搜寻路径规划[J]. 系统工程与电子技术, 2022, 44(11): 3486-3495.
[9]	曾斌, 张鸿强, 李厚朴. 针对无人潜航器的反潜策略研究[J]. 系统工程与电子技术, 2022, 44(10): 3174-3181.
[10]	万齐天, 卢宝刚, 赵雅心, 温求遒. 基于深度强化学习的驾驶仪参数快速整定方法[J]. 系统工程与电子技术, 2022, 44(10): 3190-3199.
[11]	曾斌, 王睿, 李厚朴, 樊旭. 基于强化学习的战时保障力量调度策略研究[J]. 系统工程与电子技术, 2022, 44(1): 199-208.
[12]	江志炜, 黄洋, 吴启晖. 基于核函数强化学习的抗干扰频点分配[J]. 系统工程与电子技术, 2021, 43(6): 1547-1556.
[13]	刘家义, 岳韶华, 王刚, 姚小强, 张杰. 复杂任务下的多智能体协同进化算法[J]. 系统工程与电子技术, 2021, 43(4): 991-1002.
[14]	闫安, 陈章, 董朝阳, 何康辉. 基于模糊强化学习的双轮机器人姿态平衡控制[J]. 系统工程与电子技术, 2021, 43(4): 1036-1043.
[15]	李琛, 黄炎焱, 张永亮, 陈天德. Actor-Critic框架下的多智能体决策方法及其在兵棋上的应用[J]. 系统工程与电子技术, 2021, 43(3): 755-762.

基于马尔可夫的多功能雷达认知干扰决策建模研究

Research on decision-making modeling of cognitive jamming for multi-functional radar based on Markov

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 16

参考文献 32

相关文章 15

编辑推荐

Metrics

本文评价