基于对角掩蔽自注意力的空中目标意图特征插补方法

doi:10.12305/j.issn.1001-506X.2025.08.05

Abstract

Abstract:

A non-autoregressive imputation method based on diagonal masking self-attention mechanism is proposed to address the issue of missing values in features for air target intention recognition. The method utilizes the Transformer Encoder as its backbone. Diagonal masking self-attention ensures that the network model pays more attention to temporal dependencies and attribute correlations between different time steps, resulting in more useful representations. The learning objective is defined as minimizing a composite loss function that combines imputation loss and reconstruction loss. This objective enables the network model to accurately predict missing values while simultaneously converging towards the distribution of the observed values. The method is tested using feature data from the same region in the simulated system. The data cover six types of intentions, and datasets are constructed with different missing rates. The results indicate that the method reduces imputation bias by 19.8% to 37.9% compared to the deep learning imputation methods based on gated recurrent neural networks for a set percentage of missing values. The results of downstream intention recognition indicate that the data imputed after applying the proposed method performs better in the same classifier.

Key words: intention recognition, air target, missing value imputation, multivariate time series, diagonal masking self-attention

CLC Number:

TP 181

Zihao SONG, Yan ZHOU, Wei CHENG, Hui LI, Chenhao ZHANG. Imputation method based on diagonal masking self-attention for air target intention recognition features[J]. Systems Engineering and Electronics, 2025, 47(8): 2463-2474.

Figures/Tables 28

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Table 1

Table 2

Table 3

Table 4

Table 5

Table 6

Table 7

Fig.7

Table 8

Table 9

Table 10

Table 11

Table 12

Table 13

Table 14

Table 15

Table 16

Table 17

Table 18

Table 19

Table 20

Table 21

References 29

1	HAWTHORNE G, ELLIOTT P. Imputing cross-sectional missing data: comparison of common techniques[J]. Australian & New Zealand Journal of Psychiatry, 2005, 39 (7): 583- 590.
2	JADHAV A, PRAMOD D, RAMANATHAN K. Comparison of performance of data imputation methods for numeric dataset[J]. Applied Artificial Intelligence, 2019, 33 (10): 913- 933. doi: 10.1080/08839514.2019.1637138
3	JUNNINEN H, NISKA H, TUPPURAINEN K. Methods for imputation of missing values in air quality data sets[J]. Atmospheric Environment, 2004, 38 (18): 2895- 2907. doi: 10.1016/j.atmosenv.2004.02.026
4	SUN Y, SONG S X, WANG C, et al. Swapping repair for misplaced attribute values[C]//Proc. of the IEEE 36th International Conference on Data Engineering, 2020: 721−732.
5	BATISTA G, MONARD M C. A study of k-nearest neighbour as an imputation method[C]//Proc. of the 2nd International Conference on Hybrid Intelligent Systems , 2002: 251−260.
6	JIA X Y, DONG X Y, CHEN M, et al. Missing data imputation for traffic congestion data based on joint matrix factorization[J]. Knowledge-Based Systems, 2021, 225, 107114. doi: 10.1016/j.knosys.2021.107114
7	ZHANG P J, REN P, LIU Y, et al. Autoregressive matrix factorization for imputation and forecasting of spatiotemporal structural monitoring time series[J]. Mechanical Systems and Signal Processing, 2022, 169, 108718. doi: 10.1016/j.ymssp.2021.108718
8	JUNGER W L, PONCE DE LEON A. Imputation of missing data in time series for air pollutants[J]. Atmospheric Environment, 2015, 102, 96- 104. doi: 10.1016/j.atmosenv.2014.11.049
9	NELWAMONDO F V, MOHAMED S, MARWALA T. Missing data: a comparison of neural network and expectation maximization techniques[J]. Current Science, 2007, 93 (11): 1514- 1521.
10	SONG S X, ZHANG A Q, WANG J M, et al. SCREEN: stream data cleaning under speed constraints[C]//Proc. of the ACM SIGMOD International Conference on Management of Data, 2015: 827−841.
11	SONG S X, ZHANG A Q, CHEN L, et al. Enriching data imputation with extensive similarity neighbors[J]. Proceedings of the VLDB Endowment, 2015, 8 (11): 1286- 1297. doi: 10.14778/2809974.2809989
12	WEERAKODY P B, WONG K W, WANG G, et al. A review of irregular time series data handling with gated recurrent neural networks[J]. Neurocomputing, 2021, 441 (21): 161- 178. doi: 10.1016/j.neucom.2021.02.046
13	CAO W, WANG D, LI J, et al. BRITS: bidirectional recurrent imputation for time series[C]//Proc. of the 32nd International Conference on Neural Information Processing Systems, 2018: 6776−6786.
14	YOON J S, ZAME W R, SCHAAR M. Estimating missing data in temporal data streams using multi-directional recurrent neural networks[J]. IEEE Trans. on Biomedical Engineering, 2019, 66 (5): 1477- 1490. doi: 10.1109/TBME.2018.2874712
15	NI Q J, CAO X H. MBGAN: an improved generative adversarial network with multi-head self-attention and bidirectional RNN for time series imputation[J]. Engineering Applications of Artificial Intelligence, 2022, 115, 105232. doi: 10.1016/j.engappai.2022.105232
16	LUO Y H, CAI X R, ZHANG Y, et al. Multivariate time series imputation with generative adversarial networks[C]//Proc. of the 32nd International Conference on Neural Information Processing Systems, 2018: 1603 - 1614.
17	WU Z J, MA C, SHI X C, et al. BRNN-GAN: generative adversarial networks with bi-directional recurrent neural networks for multivariate time series imputation[C]//Proc. of the IEEE 27th International Conference on Parallel and Distributed Systems, 2021: 217−224.
18	SALIMANS T, GOODFELLOW I, ZAREMBA W, et al. Improved techniques for training GANs[C]//Proc. of the 30th International Conference on Neural Information Processing Systems, 2016: 2234 - 2242.
19	陈俊扬, 戴志江, 李雪亮, 等. 基于强化学习的多变量时序数据缺失值补全方法[J]. 中国科技论文, 2023, 18 (11): 1229- 1212. doi: 10.3969/j.issn.2095-2783.2023.11.007
	CHEN J Y, DAI Z J, LI X L, et al. Reinforcement learning based missing value completion method for multivariate time series data[J]. China Sciencepaper, 2023, 18 (11): 1229- 1212. doi: 10.3969/j.issn.2095-2783.2023.11.007
20	ALABADLA M, SIDI F, ISHAK I, et al. Systematic review of using machine learning in imputing missing values[J]. IEEE Access, 2022, 10, 44483- 44502. doi: 10.1109/ACCESS.2022.3160841
21	任利强, 贾舒宜, 王海鹏, 等. 基于深度学习的时间序列分类研究综述[EB/OL]. [2024-03-31]. http://kns.cnki.net/kcms/detail/11.4494.TN.20240109.0749.008.html.
	REN L Q, JIA S Y, WANG H P. A review of research on time series classification based on deep learning[EB/OL]. [2024-03-31]. http://kns.cnki.net/kcms/detail/11.4494.TN.20240109.0749.008.html.
22	石磊, 王毅, 成颖, 等. 自然语言处理中的注意力机制研究综述[J]. 数据分析与知识发现, 2020, 4 (5): 122- 124.
	SHI L, WANG Y, CHENG Y, et al. Review of attention mechanism in natural language processing[J]. Data Analysis and Knowledge Discovery, 2020, 4 (5): 122- 124.
23	吴嘉俊, 苏春, 张玉茹. 基于双重自注意力机制和长短时记忆网络的剩余寿命预测[J]. 系统工程与电子技术, 2024, 46 (6): 1986- 1994.
	WU J J, SU C, ZHANG Y R. Remaining useful life prediction based on double self-attention mechanism and long short-term memory network[J]. Systems Engineering and Electronics, 2024, 46 (6): 1986- 1994.
24	张春杰, 王冠博, 陈奇, 等. 基于纯自注意力机制的毫米波雷达手势识别[J]. 系统工程与电子技术, 2024, 46 (3): 859- 867.
	ZHANG C J, WANG G B, CHEN Q, et al. Gesture recognition based on millimeter-wave radar with pure self-attention mechanism[J]. Systems Engineering and Electronics, 2024, 46 (3): 859- 867.
25	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proc. of the 31st International Conference on Neural Information Processing Systems, 2017: 6000 - 6010.
26	BRITTAIN J, CENDON M, NIZZI J, et al. Data scientist’s analysis toolbox: comparison of Python, R, and SAS performance[J]. SMU Data Science Review, 2018, 1 (2): 7.
27	LIEW A W C, LAW N F, YAN H. Missing value imputation for gene expression data: computational techniques to recover missing data from available information[J]. Briefings in Bioinformatics, 2011, 12 (5): 498- 513. doi: 10.1093/bib/bbq080
28	TANG F, ISHWARAN H. Random forest missing data algorithms[J]. Statistical Analysis and Data Mining: the ASA Data Science Journal, 2017, 10 (6): 363- 377. doi: 10.1002/sam.11348
29	DU W J, COTE D, LIU Y. SAITS: self-attention-based imputation for time series[J]. Expert Systems with Applications, 2023, 219, 119619. doi: 10.1016/j.eswa.2023.119619

超参数	符号	数值
BS	BS	64
人为添加缺失值比例/%	RM_artificial	20
多头DMSA单元个数	N	1
FFN中隐藏层特征维度	d_hidden	512
模型隐含维度	d_model	64
多头DMSA头数	h	4
Q、K、V的维度	d_k, d_k, d_v	64, 64, 64
学习率	LR	0.005
迭代次数上限	N_epoch	90

中央处理器	Intel I5 10200 CPU
内存/GB	32
硬盘容量/GB	500
图形处理单元	NVIDIA GeForce RTX 2060
操作系统	Windows 10
深度学习框架	Pytorch 1.9.1
Cuda版本	11.1

方法	MAE	RMSE	MRE/%
Mean	0.1953	0.5169	25.65
KNN	0.1201	0.4347	15.79
RF	0.2044	0.6010	26.85
M-RNN	0.0364	0.2938	4.79
BRITS	0.0355	0.2864	4.67
Transformer	0.0333	0.1967	4.38
本文	0.0261	0.1958	3.43

方法	MAE	RMSE	MRE/%
Mean	0.1956	0.5287	25.69
KNN	0.1201	0.4397	15.78
RF	0.2036	0.5983	26.76
M-RNN	0.0472	0.2968	6.20
BRITS	0.0433	0.2813	5.69
Transformer	0.0424	0.2609	5.56
本文	0.0269	0.2017	3.54

方法	MAE	RMSE	MRE/%
Mean	0.1955	0.5259	25.68
KNN	0.1268	0.4543	16.65
RF	0.2039	0.5938	26.78
M-RNN	0.0635	0.2979	8.33
BRITS	0.0620	0.2819	8.14
Transformer	0.0531	0.3096	6.98
本文	0.0492	0.3081	6.46

Imputation method based on diagonal masking self-attention for air target intention recognition features

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 28

References 29

Related Articles 8

Recommended Articles

Metrics

Comments

模型	参数数量	训练时间/s
M-RNN	2 690 210	32.14
BRITS	2 168 144	64.57
Transformer	429 382	29.53
本文	269 336	2.75

插补方法	识别方法
插补方法	SVM	LR	CNN	RNN
Mean	0.8598	0.7355	0.9408	0.9582
KNN	0.8519	0.7312	0.9289	0.9576
RF	0.8410	0.7333	0.9146	0.9311
M-RNN	0.8854	0.7195	0.9539	0.9739
BRITS	0.8851	0.7201	0.9689	0.9796
Transformer	0.8835	0.7185	0.9652	0.9772
本文	0.9081	0.7193	0.9695	0.9828

学习目标	评价指标
学习目标	MAE	RMSE	MRE/%
IL	0.029 0	0.203 8	3.81
RL	0.482 9	0.726 9	63.45
IL+RL	0.026 1	0.195 8	3.43

α取值	评价指标
α取值	MAE	RMSE	MRE/%
1	0.026 1	0.195 8	3.43
0.5	0.028 9	0.204 4	3.81
2	0.026 5	0.176 0	3.49

注意力机制	评价指标
注意力机制	MAE	RMSE	MRE/%
vanilla SA	0.026 9	0.192 2	3.54
DMSA	0.026 1	0.195 8	3.43

[1]	Wei CAI, Xin WANG, Xinhao JIANG, Zhiyong YANG, Dong CHEN. Research on few shot target detection method based on decoupling [J]. Systems Engineering and Electronics, 2024, 46(9): 2941-2950.
[2]	Jianguo YIN, Wen SHENG, Wei JIANG. Radar air target recognition based on deep residual shrinkage network [J]. Systems Engineering and Electronics, 2024, 46(9): 3012-3018.
[3]	Shuaikang LIU, Wei CAO, Zhiqiang GUAN, Xueling YANG, Jinxin XU. Classification of radar air targets based on RMDLPP [J]. Systems Engineering and Electronics, 2024, 46(4): 1220-1228.
[4]	Yutang MA, Peng SUN, Jieyong ZHANG, Peng WANG, Yunfei YAN, Liang ZHAO. Air group intention recognition method under imbalance samples [J]. Systems Engineering and Electronics, 2022, 44(12): 3747-3755.
[5]	Yunke SUN, Zhigeng FANG, Ding CHEN. Multi-time threat assessment based on dynamic grey principal component analysis [J]. Systems Engineering and Electronics, 2021, 43(3): 740-746.
[6]	LI Zhengxin, ZHANG Fengming, WANG Ying, TAO Qian, LI Chao. Method of missing data imputation for multivariate time series [J]. Systems Engineering and Electronics, 2018, 40(1): 225-230.
[7]	CHEN Hao, REN Qing-long, HUA Yi, QIU Yu-ning. Fuzzy neural network based tactical intention recognition for sea targets [J]. Systems Engineering and Electronics, 2016, 38(8): 1847-1853.
[8]	DENG Hai-jun,YIN Quan-jun,HU Ji-wen,ZHA Ya-bing. Tactical intention recognition based on multi-entity Bayesian network [J]. Journal of Systems Engineering and Electronics, 2010, 32(11): 2374-2379.

方法	MAE	RMSE	MRE/%
Mean	0.1956	0.5287	25.69
KNN	0.1288	0.4601	16.91
RF	0.2024	0.5874	26.58
M-RNN	0.0727	0.3550	9.55
BRITS	0.0664	0.3109	8.72
Transformer	0.0612	0.3195	8.04
本文	0.0532	0.3084	6.98

插补方法	识别方法
插补方法	SVM	LR	CNN	RNN
Mean	0.6766	0.6728	0.9235	0.9373
KNN	0.6699	0.6644	0.9289	0.9320
RF	0.6515	0.6518	0.9098	0.9133
M-RNN	0.8751	0.7196	0.9519	0.9648
BRITS	0.8762	0.7168	0.9558	0.9647
Transformer	0.8786	0.7145	0.9649	0.9710
本文	0.9054	0.7152	0.9726	0.9791

学习目标	评价指标
学习目标	MAE	RMSE	MRE/%
IL	0.031 1	0.180 6	4.08
RL	0.485 3	0.742 1	63.78
IL+RL	0.026 9	0.201 7	3.54

学习目标	评价指标
学习目标	MAE	RMSE	MRE/%
IL	0.050 8	0.298 3	6.67
RL	0.534 1	0.785 1	70.13
IL+RL	0.049 2	0.308 1	6.46