基于对角掩蔽自注意力的空中目标意图特征插补方法

doi:10.12305/j.issn.1001-506X.2025.08.05

摘要/Abstract

摘要：

针对空中目标意图特征数据缺失问题，提出一种基于对角掩蔽自注意力机制的非自回归缺失值插补方法。该方法以Transformer Encoder为骨架网络，对角掩蔽自注意力确保网络模型更加关注不同时间步之间的时间依赖性和属性相关性，获得更有益的表征；以最小化合并插补损失及重建损失的复合损失函数为学习目标，使得网络模型在准确预测缺失值的同时收敛于观察值的分布。使用仿真系统中同一区域下、包含6种意图类型的特征数据，构造不同缺失率下的数据集对方法进行测试，结果表明：在设定的缺失值比例下，与基于门控循环神经网络的深度学习插补方法相比，该方法的插补偏差降低了19.8%~37.9%。下游意图识别结果显示，经过本文提出方法插补后的数据在同一分类器中表现更好。

关键词: 意图识别, 空中目标, 缺失值插补, 多变量时间序列, 对角掩蔽自注意力

Abstract:

A non-autoregressive imputation method based on diagonal masking self-attention mechanism is proposed to address the issue of missing values in features for air target intention recognition. The method utilizes the Transformer Encoder as its backbone. Diagonal masking self-attention ensures that the network model pays more attention to temporal dependencies and attribute correlations between different time steps, resulting in more useful representations. The learning objective is defined as minimizing a composite loss function that combines imputation loss and reconstruction loss. This objective enables the network model to accurately predict missing values while simultaneously converging towards the distribution of the observed values. The method is tested using feature data from the same region in the simulated system. The data cover six types of intentions, and datasets are constructed with different missing rates. The results indicate that the method reduces imputation bias by 19.8% to 37.9% compared to the deep learning imputation methods based on gated recurrent neural networks for a set percentage of missing values. The results of downstream intention recognition indicate that the data imputed after applying the proposed method performs better in the same classifier.

Key words: intention recognition, air target, missing value imputation, multivariate time series, diagonal masking self-attention

中图分类号:

TP 181

宋子豪, 周焰, 程伟, 黎慧, 张晨浩. 基于对角掩蔽自注意力的空中目标意图特征插补方法[J]. 系统工程与电子技术, 2025, 47(8): 2463-2474.

Zihao SONG, Yan ZHOU, Wei CHENG, Hui LI, Chenhao ZHANG. Imputation method based on diagonal masking self-attention for air target intention recognition features[J]. Systems Engineering and Electronics, 2025, 47(8): 2463-2474.

图/表 28

图1

图2

图3

图4

图5

图6

表1

表2

表3

表4

表5

表6

表7

图7

表8

表9

表10

表11

表12

表13

表14

表15

表16

表17

表18

表19

表20

表21

参考文献 29

1	HAWTHORNE G, ELLIOTT P. Imputing cross-sectional missing data: comparison of common techniques[J]. Australian & New Zealand Journal of Psychiatry, 2005, 39 (7): 583- 590.
2	JADHAV A, PRAMOD D, RAMANATHAN K. Comparison of performance of data imputation methods for numeric dataset[J]. Applied Artificial Intelligence, 2019, 33 (10): 913- 933. doi: 10.1080/08839514.2019.1637138
3	JUNNINEN H, NISKA H, TUPPURAINEN K. Methods for imputation of missing values in air quality data sets[J]. Atmospheric Environment, 2004, 38 (18): 2895- 2907. doi: 10.1016/j.atmosenv.2004.02.026
4	SUN Y, SONG S X, WANG C, et al. Swapping repair for misplaced attribute values[C]//Proc. of the IEEE 36th International Conference on Data Engineering, 2020: 721−732.
5	BATISTA G, MONARD M C. A study of k-nearest neighbour as an imputation method[C]//Proc. of the 2nd International Conference on Hybrid Intelligent Systems , 2002: 251−260.
6	JIA X Y, DONG X Y, CHEN M, et al. Missing data imputation for traffic congestion data based on joint matrix factorization[J]. Knowledge-Based Systems, 2021, 225, 107114. doi: 10.1016/j.knosys.2021.107114
7	ZHANG P J, REN P, LIU Y, et al. Autoregressive matrix factorization for imputation and forecasting of spatiotemporal structural monitoring time series[J]. Mechanical Systems and Signal Processing, 2022, 169, 108718. doi: 10.1016/j.ymssp.2021.108718
8	JUNGER W L, PONCE DE LEON A. Imputation of missing data in time series for air pollutants[J]. Atmospheric Environment, 2015, 102, 96- 104. doi: 10.1016/j.atmosenv.2014.11.049
9	NELWAMONDO F V, MOHAMED S, MARWALA T. Missing data: a comparison of neural network and expectation maximization techniques[J]. Current Science, 2007, 93 (11): 1514- 1521.
10	SONG S X, ZHANG A Q, WANG J M, et al. SCREEN: stream data cleaning under speed constraints[C]//Proc. of the ACM SIGMOD International Conference on Management of Data, 2015: 827−841.
11	SONG S X, ZHANG A Q, CHEN L, et al. Enriching data imputation with extensive similarity neighbors[J]. Proceedings of the VLDB Endowment, 2015, 8 (11): 1286- 1297. doi: 10.14778/2809974.2809989
12	WEERAKODY P B, WONG K W, WANG G, et al. A review of irregular time series data handling with gated recurrent neural networks[J]. Neurocomputing, 2021, 441 (21): 161- 178. doi: 10.1016/j.neucom.2021.02.046
13	CAO W, WANG D, LI J, et al. BRITS: bidirectional recurrent imputation for time series[C]//Proc. of the 32nd International Conference on Neural Information Processing Systems, 2018: 6776−6786.
14	YOON J S, ZAME W R, SCHAAR M. Estimating missing data in temporal data streams using multi-directional recurrent neural networks[J]. IEEE Trans. on Biomedical Engineering, 2019, 66 (5): 1477- 1490. doi: 10.1109/TBME.2018.2874712
15	NI Q J, CAO X H. MBGAN: an improved generative adversarial network with multi-head self-attention and bidirectional RNN for time series imputation[J]. Engineering Applications of Artificial Intelligence, 2022, 115, 105232. doi: 10.1016/j.engappai.2022.105232
16	LUO Y H, CAI X R, ZHANG Y, et al. Multivariate time series imputation with generative adversarial networks[C]//Proc. of the 32nd International Conference on Neural Information Processing Systems, 2018: 1603 - 1614.
17	WU Z J, MA C, SHI X C, et al. BRNN-GAN: generative adversarial networks with bi-directional recurrent neural networks for multivariate time series imputation[C]//Proc. of the IEEE 27th International Conference on Parallel and Distributed Systems, 2021: 217−224.
18	SALIMANS T, GOODFELLOW I, ZAREMBA W, et al. Improved techniques for training GANs[C]//Proc. of the 30th International Conference on Neural Information Processing Systems, 2016: 2234 - 2242.
19	陈俊扬, 戴志江, 李雪亮, 等. 基于强化学习的多变量时序数据缺失值补全方法[J]. 中国科技论文, 2023, 18 (11): 1229- 1212. doi: 10.3969/j.issn.2095-2783.2023.11.007
	CHEN J Y, DAI Z J, LI X L, et al. Reinforcement learning based missing value completion method for multivariate time series data[J]. China Sciencepaper, 2023, 18 (11): 1229- 1212. doi: 10.3969/j.issn.2095-2783.2023.11.007
20	ALABADLA M, SIDI F, ISHAK I, et al. Systematic review of using machine learning in imputing missing values[J]. IEEE Access, 2022, 10, 44483- 44502. doi: 10.1109/ACCESS.2022.3160841
21	任利强, 贾舒宜, 王海鹏, 等. 基于深度学习的时间序列分类研究综述[EB/OL]. [2024-03-31]. http://kns.cnki.net/kcms/detail/11.4494.TN.20240109.0749.008.html.
	REN L Q, JIA S Y, WANG H P. A review of research on time series classification based on deep learning[EB/OL]. [2024-03-31]. http://kns.cnki.net/kcms/detail/11.4494.TN.20240109.0749.008.html.
22	石磊, 王毅, 成颖, 等. 自然语言处理中的注意力机制研究综述[J]. 数据分析与知识发现, 2020, 4 (5): 122- 124.
	SHI L, WANG Y, CHENG Y, et al. Review of attention mechanism in natural language processing[J]. Data Analysis and Knowledge Discovery, 2020, 4 (5): 122- 124.
23	吴嘉俊, 苏春, 张玉茹. 基于双重自注意力机制和长短时记忆网络的剩余寿命预测[J]. 系统工程与电子技术, 2024, 46 (6): 1986- 1994.
	WU J J, SU C, ZHANG Y R. Remaining useful life prediction based on double self-attention mechanism and long short-term memory network[J]. Systems Engineering and Electronics, 2024, 46 (6): 1986- 1994.
24	张春杰, 王冠博, 陈奇, 等. 基于纯自注意力机制的毫米波雷达手势识别[J]. 系统工程与电子技术, 2024, 46 (3): 859- 867.
	ZHANG C J, WANG G B, CHEN Q, et al. Gesture recognition based on millimeter-wave radar with pure self-attention mechanism[J]. Systems Engineering and Electronics, 2024, 46 (3): 859- 867.
25	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proc. of the 31st International Conference on Neural Information Processing Systems, 2017: 6000 - 6010.
26	BRITTAIN J, CENDON M, NIZZI J, et al. Data scientist’s analysis toolbox: comparison of Python, R, and SAS performance[J]. SMU Data Science Review, 2018, 1 (2): 7.
27	LIEW A W C, LAW N F, YAN H. Missing value imputation for gene expression data: computational techniques to recover missing data from available information[J]. Briefings in Bioinformatics, 2011, 12 (5): 498- 513. doi: 10.1093/bib/bbq080
28	TANG F, ISHWARAN H. Random forest missing data algorithms[J]. Statistical Analysis and Data Mining: the ASA Data Science Journal, 2017, 10 (6): 363- 377. doi: 10.1002/sam.11348
29	DU W J, COTE D, LIU Y. SAITS: self-attention-based imputation for time series[J]. Expert Systems with Applications, 2023, 219, 119619. doi: 10.1016/j.eswa.2023.119619

超参数	符号	数值
BS	BS	64
人为添加缺失值比例/%	RM_artificial	20
多头DMSA单元个数	N	1
FFN中隐藏层特征维度	d_hidden	512
模型隐含维度	d_model	64
多头DMSA头数	h	4
Q、K、V的维度	d_k, d_k, d_v	64, 64, 64
学习率	LR	0.005
迭代次数上限	N_epoch	90

中央处理器	Intel I5 10200 CPU
内存/GB	32
硬盘容量/GB	500
图形处理单元	NVIDIA GeForce RTX 2060
操作系统	Windows 10
深度学习框架	Pytorch 1.9.1
Cuda版本	11.1

方法	MAE	RMSE	MRE/%
Mean	0.1953	0.5169	25.65
KNN	0.1201	0.4347	15.79
RF	0.2044	0.6010	26.85
M-RNN	0.0364	0.2938	4.79
BRITS	0.0355	0.2864	4.67
Transformer	0.0333	0.1967	4.38
本文	0.0261	0.1958	3.43

方法	MAE	RMSE	MRE/%
Mean	0.1956	0.5287	25.69
KNN	0.1201	0.4397	15.78
RF	0.2036	0.5983	26.76
M-RNN	0.0472	0.2968	6.20
BRITS	0.0433	0.2813	5.69
Transformer	0.0424	0.2609	5.56
本文	0.0269	0.2017	3.54

方法	MAE	RMSE	MRE/%
Mean	0.1955	0.5259	25.68
KNN	0.1268	0.4543	16.65
RF	0.2039	0.5938	26.78
M-RNN	0.0635	0.2979	8.33
BRITS	0.0620	0.2819	8.14
Transformer	0.0531	0.3096	6.98
本文	0.0492	0.3081	6.46