基于双重自注意力机制和长短时记忆网络的剩余寿命预测

doi:10.12305/j.issn.1001-506X.2024.06.16

Abstract

Abstract:

Prediction of remaining useful life (RUL) is an important part of fault prognostic and health management. Traditional long short-term memory (LSTM) network cannot select the key features actively, and it is difficult to effectively extract the degradation information contained in big data. This paper proposes an RUL prediction approach based on an improved LSTM network, where the random forest (RF) algorithm is adopted to filter the input features in order to select key features actively. A double self-attention mechanism is used to complete the adaptive weight assignment from feature dimension and the time dimension. Thus, the proposed approach can focus on the key features and historical time during the learning process. By fusing the statistical features, the model can improve the accuracy of RUL prediction. To illustrate the effectiveness of the proposed method, a case study is conducted with a data set of aircraft engine. The results indicate that the proposed method can effectively improve the accuracy of RUL prediction with complicated data sets.

Key words: remaining useful life (RUL) prediction, random forest (RF), double self-attention mechanism, long short-term memory (LSTM) network, aircraft engine

CLC Number:

TH17

Jiajun WU, Chun SU, Yuru ZHANG. Remaining useful life prediction based on double self-attention mechanism and long short-term memory network[J]. Systems Engineering and Electronics, 2024, 46(6): 1986-1994.

Figures/Tables 16

Fig.1

Fig.2

Fig.3

Fig.4

Table 1

Fig.5

Fig.6

Table 2

Table 3

Fig.7

Table 4

Fig.8

Fig.9

Table 5

Fig.10

Fig.11

References 33

1	OMSHI E M , GRALL A . Replacement and imperfect repair of deteriorating system: study of a CBM policy and impact of repair efficiency[J]. Reliability Engineering & System Safety, 2021, 215, 107905.
2	LEI Y G , LI N P , GUO L , et al. Machinery health prognostics: a systematic review from data acquisition to RUL prediction[J]. Mechanical Systems and Signal Processing, 2018, 104, 799- 834. doi: 10.1016/j.ymssp.2017.11.016
3	NIETO P J G , GARCIA-GONZALO E , LASHERAS F S , et al. Hybrid PSO-SVM-based method for forecasting of the remaining useful life for aircraft engines and evaluation of its reliability[J]. Reliability Engineering & System Safety, 2015, 138, 219- 231.
4	SCHONLAU M , ZOU R Y . The random forest algorithm for statistical learning[J]. The Stata Journal, 2020, 20 (1): 3- 29. doi: 10.1177/1536867X20909688
5	LIN M Q , YOU Y Q , WANG W , et al. Battery health prognosis with gated recurrent unit neural networks and hidden Markov model considering uncertainty quantification[J]. Reliability Engineering & System Safety, 2023, 230, 108978.
6	XU Z Y , SALEH J H . Machine learning for reliability engineering and safety applications: review of current status and future opportunities[J]. Reliability Engineering & System Safety, 2021, 211, 107530.
7	LI X , DING Q , SUN J Q . Remaining useful life estimation in prognostics using deep convolution neural networks[J]. Reliability Engineering & System Safety, 2018, 172, 1- 11.
8	ZHU J , CHEN N , PENG W W . Estimation of bearing remaining useful life based on multiscale convolutional neural network[J]. IEEE Trans.on Industrial Electronics, 2019, 66 (4): 3208- 3216. doi: 10.1109/TIE.2018.2844856
9	SHANG Z W , ZHANG B R , LI W X , et al. Machine remaining life prediction based on multi-layer self-attention and temporal convolution network[J]. Complex & Intelligent Systems, 2022, 8 (2): 1409- 1424.
10	ZHANG J J , WANG P , YAN R Q , et al. Long short-term memory for machine remaining life prediction[J]. Journal of Manufacturing Systems, 2018, 48, 78- 86. doi: 10.1016/j.jmsy.2018.05.011
11	SU C , LI L , WEN Z J . Remaining useful life prediction via a variational autoencoder and a time-window-based sequence neural network[J]. Quality and Reliability Engineering International, 2020, 36 (5): 1639- 1656. doi: 10.1002/qre.2651
12	李京峰, 陈云翔, 项华春, 等. 基于LSTM-DBN的航空发动机剩余寿命预测[J]. 系统工程与电子技术, 2020, 42 (7): 1637- 1644.
	LI J F , CHEN Y X , XIANG H C , et al. Remaining useful life prediction for aero-engine based on the similarity of degradation characteristics[J]. Systems Engineering and Electronics, 2020, 42 (7): 1637- 1644.
13	CHEN Z H , WU M , ZHAO R , et al. Machine remaining useful life prediction via an attention-based deep learning approach[J]. IEEE Trans.on Industrial Electronics, 2021, 68 (3): 2521- 2531. doi: 10.1109/TIE.2020.2972443
14	ZHANG Y , XIN Y Q , LIU Z W , et al. Health status assessment and remaining useful life prediction of aero-engine based on BiGRU and MMoE[J]. Reliability Engineering & System Safety, 2022, 220, 108263.
15	闫啸家, 梁伟阁, 张钢, 等. 基于RCNN-ABiLSTM的机械设备剩余寿命预测方法[J]. 系统工程与电子技术, 2023, 45 (3): 931- 940.
	YAN X J , LIANG W G , ZHANG G , et al. Prediction method for mechanical equipment based on RCNN-ABiLSTM[J]. Systems Engineering and Electronics, 2023, 45 (3): 931- 940.
16	MA X L , TAO Z M , WANG Y H , et al. Long short-term memory neural for traffic speed prediction using remote microwave sensor data[J]. Transportation Research Part C: Emerging Technologies, 2015, 54, 187- 197. doi: 10.1016/j.trc.2015.03.014
17	WANG H , PENG M J , MIAO Z , et al. Remaining useful life prediction techniques for electric valves based on convolution auto encoder and long short term memory[J]. ISA Transactions, 2021, 108, 333- 342. doi: 10.1016/j.isatra.2020.08.031
18	GU L D , ZHENG R , ZHOU Y F , et al. Remaining useful life prediction using composite health index and hybrid LSTM-SVR model[J]. Quality and Reliability Engineering International, 2022, 38 (7): 3559- 3578. doi: 10.1002/qre.3151
19	DU W B , WANG Y L , QIAO Y . Recurrent spatial-temporal attention network for action recognition in videos[J]. IEEE Trans.on Image Processing, 2018, 27 (3): 1347- 1360. doi: 10.1109/TIP.2017.2778563
20	PAN T Y , CHEN J L , YE Z S , et al. A multi-head attention network with adaptive meta-transfer learning for RUL prediction of rocket engines[J]. Reliability Engineering & System Safety, 2022, 225, 108610.
21	LIU L , SONG X , ZHOU Z T . Aircraft engine remaining useful life estimation via a double attention-based data-driven architecture[J]. Reliability Engineering & System Safety, 2022, 221, 108330.
22	KHELIF R , CHEBEL-MORELLO B , MALINOWSKI S , et al. Direct remaining useful life estimation based on support vector regression[J]. IEEE Trans.on Industrial Electronics, 2017, 64 (3): 2276- 2285. doi: 10.1109/TIE.2016.2623260
23	汪千程, 苏春, 文泽军. 基于协整分析的风力机多工况监测与故障诊断[J]. 中国机械工程, 2022, 33 (13): 1596- 1603. doi: 10.3969/j.issn.1004-132X.2022.13.010
	WANG Q C , SU C , WEN Z J . Multi-condition monitoring and fault diagnosis of wind turbines based on cointegration analysis[J]. China Mechanical Engineering, 2022, 33 (13): 1596- 1603. doi: 10.3969/j.issn.1004-132X.2022.13.010
24	张紫嫣, 卢文科, 左锋, 等. 基于分段线性插值法的应变片式扭矩传感器的温度补偿[J]. 电子测量技术, 2022, 45 (8): 143- 147.
	ZHANG Z Y , LU W K , ZUO F , et al. Temperature compensation of strain gauge torque sensor based on piecewise linear interpolation method[J]. Electronic Measurement Technology, 2022, 45 (8): 143- 147.
25	CHENG Y J , ZENG J Y , WANG Z L , et al. A health state-related ensemble deep learning method for aircraft engine remaining useful life prediction[J]. Applied Soft Computing, 2023, 135, 110041. doi: 10.1016/j.asoc.2023.110041
26	GONG R , LI J X , WANG C L . Remaining useful life prediction based on multisensor fusion and attention TCN-BiGRU model[J]. IEEE Sensors Journal, 2022, 22 (21): 21101- 21110. doi: 10.1109/JSEN.2022.3208753
27	SAXENA A, GOEBEL K, SIMON D, et al. Damage propagation modeling for aircraft engine run-to-failure simulation[C]//Proc. of the International Conference on Prognostics and Health Management, 2008: 1-9.
28	LI H , ZHAO W , ZHANG Y X , et al. Remaining useful life prediction using multi-scale deep convolutional neural network[J]. Applied Soft Computing, 2020, 89, 106113. doi: 10.1016/j.asoc.2020.106113
29	ZHANG C , LIM P , QIN A K , et al. Multiobjective deep belief networks ensemble for remaining useful life estimation in prognostics[J]. IEEE Trans.on Neural Networks and Learning Systems, 2017, 28 (10): 2306- 2318. doi: 10.1109/TNNLS.2016.2582798
30	XIA M , ZHENG X , IMRAN M , et al. Data-driven prognosis method using hybrid deep recurrent neural network[J]. Applied Soft Computing, 2020, 93, 106351. doi: 10.1016/j.asoc.2020.106351
31	REMADNA I, TERRISSA S L, ZEMOURI R, et al. Leveraging the power of the combination of CNN and bi-directional LSTM networks for aircraft engine RUL estimation[C]//Proc. of the Prognostics and Health Management Conference, 2020: 116-121.
32	XIANG S , QIN Y , LUO J , et al. Multicellular LSTM-based deep learning model for aero-engine remaining useful life prediction[J]. Reliability Engineering & System Safety, 2021, 216, 107927.
33	LI Y F , CHEN Y F , SHAO H N , et al. A novel dual attention mechanism combined with knowledge for remaining useful life prediction based on gated recurrent units[J]. Reliability Engineering & System Safety, 2023, 239, 109514.

指标数量	子数据集
指标数量	FD001	FD002	FD003	FD004
工作条件数量	1	6	1	6
故障模式数量	1	1	2	2
训练集数量	100	260	100	249
测试集数量	100	259	100	248

子数据集	选取的传感器特征编号
FD001	11, 9, 4, 12, 7, 14, 15, 21, 2, 3, 20, 13, 8, 17, 6
FD002	13, 11, 15, 4, 14, 9, 8, 2, 12, 3, 7, 21, 20, 17, 6, 16
FD003	11, 9, 4, 12, 7, 14, 15, 21, 2, 3, 20, 13, 8, 17, 6
FD004	13, 11, 15, 4, 14, 9, 8, 2, 12, 3, 7, 21, 20, 17, 6, 16

参数	取值
决策树个数	200
随机失活率	0.2
批量大小	128
迭代轮次	50
LSTM隐藏层单元数	50
第1层全连接层单元数	50
第2层全连接层单元数	10
第3层全连接层单元数	10

方法	FD001				FD004
方法	RMSE	STD	Score	STD	RMSE	STD	Score	STD
LSTM	13.21	0.51	306.06	54.58	19.46	1.05	3 786.78	1 691.87
RF+LSTM	13.20	0.33	306.42	45.93	19.14	0.56	3 658.44	1 498.73
RF+Feats+LSTM	12.52	0.57	253.09	26.14	18.81	0.45	2 381.83	468.51
本文方法	12.357	0.31	269.28	22.76	18.27	0.44	1 793.52	171.05

方法	年份	FD001		FD002		FD003		FD004
方法	年份	RMSE	Score	RMSE	Score	RMSE	Score	RMSE	Score
MLP^[29]	2017	16.78	561	28.78	14 027	18.47	480	30.96	10 444
DCNN^[7]	2018	12.61	273	22.36	10 412	12.64	284	23.31	12 466
RNN^[7]	2018	13.44	339	24.03	14 245	13.36	316	24.02	13 931
DLSTM^[30]	2020	14.57	—	23.20	—	14.92	—	28.72	—
VAE+LSTM^[11]	2020	15.88	322	25.78	4 990	14.29	309	23.93	4 720
CNN+Bi-LSTM^[31]	2020	10.74	—	15.20	—	13.85	—	18.60	—
MCLSTM^[32]	2021	13.71	315	—	—	—	—	23.81	4 826
Attention+LSTM^[13]	2021	14.54	322	—	—	—	—	27.08	5 649
Multi-attention+TCN^[9]	2022	13.25	235	19.57	1 655	13.43	239	21.69	2 415
Double attention-based architecture^[21]	2022	12.25	198	17.08	1 575	13.39	290	19.86	1 741
Dual attention+GRU^[33]	2023	11.77	—	16.09	—	11.66	—	20.10	—
MMoE+BiGRU^[14]	2022	13.22	—	18.26	—	13.79	—	18.38	—
RCNN+ABi-LSTM^[15]	2023	12.98	258	19.16	2 980	13.24	246	22.29	3 795
本文方法	2023	12.35	269	15.13	924	13.24	487	18.27	1 794

Remaining useful life prediction based on double self-attention mechanism and long short-term memory network

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 16

References 33

Related Articles 12

Recommended Articles

Metrics

Comments

[1]	Shaobo CHEN, Jiamin YAN, Kuichen BU. Anti-intercept maneuver method of vehicle based on prediction of miss distance [J]. Systems Engineering and Electronics, 2023, 45(9): 2922-2930.
[2]	Shaochi CHENG, Guangrong YOU. Research on the method of simplifying evaluation index based on improved random forest algorithm [J]. Systems Engineering and Electronics, 2023, 45(7): 2108-2113.
[3]	Lican DAI, Xin LIU, Haiying ZHANG, Xiang DAI, Chenggang WANG. Flight target track prediction based on Kalman filter algorithm unfolding [J]. Systems Engineering and Electronics, 2023, 45(6): 1814-1820.
[4]	Xiaojia YAN, Weige LIANG, Gang ZHANG, Bo SHE, Fuqing TIAN. Prediction method for mechanical equipment based on RCNN-ABiLSTM [J]. Systems Engineering and Electronics, 2023, 45(3): 931-940.
[5]	Botao SONG, Guangliang XU. Missile trajectory prediction method based on LSTM and 1DCNN [J]. Systems Engineering and Electronics, 2023, 45(2): 504-512.
[6]	Ruiping JI, Chengyi ZHANG, Yan LIANG, Yuedong WANG. Trajectory prediction of boost-phase ballistic missile based on LSTM [J]. Systems Engineering and Electronics, 2022, 44(6): 1968-1976.
[7]	Zhaoguo HOU, Huawei WANG, Liang ZHOU, Qiang FU. Fault diagnosis of rotating machinery based on improved deep residual network [J]. Systems Engineering and Electronics, 2022, 44(6): 2051-2059.
[8]	Yifan ZHANG, Shuanghui ZHANG, Yongxiang LIU, Feng JING. Radar HRRP sequence target recognition method of attention mechanism based stacked LSTM network [J]. Systems Engineering and Electronics, 2021, 43(10): 2775-2781.
[9]	Xiaoyue HU, Kai KANG, Hua QIAN, Shunqing ZHANG. Link adaptation algorithm based on LSTM network for LEO satellite [J]. Systems Engineering and Electronics, 2021, 43(1): 237-243.
[10]	Jingfeng LI, Yunxiang CHEN, Huachun XIANG, Zhongyi CAI. Remaining useful life prediction for aircraft engine based on LSTM-DBN [J]. Systems Engineering and Electronics, 2020, 42(7): 1637-1644.
[11]	ZHANG Yan, WANG Cunsong, LU Ningyun, JIANG Bin. Remaining useful life prediction for aeroengine based on thesimilarity of degradation characteristics [J]. Systems Engineering and Electronics, 2019, 41(6): 1414-1421.
[12]	SUN Jian-zhong, ZUO Hong-fu. Bayesian method for system real-time performance reliability assessment [J]. Journal of Systems Engineering and Electronics, 2013, 35(8): 1790-1796.