基于深度特征融合的SAR图像与AIS信息关联方法

doi:10.12305/j.issn.1001-506X.2023.11.15

Abstract

Abstract:

Spaceborne synthetic aperture radar (SAR) and automatic identification system (AIS) can obtain the information of the detected target. And the association and fusion of the information obtained from the two sensors is beneficial to realize efficient maritime reconnaissance and surveillance. Due to the heterogeneity gas between different data, traditional methods mostly rely on artificial features to establish correlation between SAR images and AIS information, but these methods have disadvantages of poor accuracy and low efficiency. In this paper, the association method between SAR images and AIS information based on depth feature fusion is proposed. According to the characteristics of the two modal data, the corresponding feature learning network is designed respectively to obtain the unimodal feature representation, and the feature information of different modals is further fused to enhance the semantic correlation of cross-modal information. Then, the association learning between cross-modal features is carried out by the designed association learning objective function. Verification on the constructed dataset shows that the proposed method can achieve high correlation accuracy and strong adaptability, and the effectiveness of the proposed dataset and method is verified.

Key words: multi-source data association, deep learning, remote sensing image, cross-modal retrieval

CLC Number:

TP751.1

Haoran LI, Wei XIONG, Yaqi CUI. An association method between SAR images and AIS information based on depth feature fusion[J]. Systems Engineering and Electronics, 2023, 45(11): 3491-3497.

Figures/Tables 11

Fig.1

Fig.2

Fig.3

Table 1

Fig.4

Table 2

Fig.5

Table 3

Fig.6

Fig.7

Table 4

References 31

1	SOLDI G , GAGLIONE D , FORTI N , et al. Space-based global maritime surveillance. Part Ⅱ: artificial intelligence and data fusion techniques[J]. IEEE Aerospace and Electronic Systems Magazine, 2021, 36 (9): 30- 42. doi: 10.1109/MAES.2021.3070884
2	LEHNER S, BRUSCH S, FRITZ T. Ship surveillance by joint use of SAR and AIS[C]//Proc. of the IEEE Oceans Europe, 2009: 1-5.
3	BRUSCH S , LEHNER S , FRITZ T , et al. Ship surveillance with TerraSAR-X[J]. IEEE Trans.on Geoscience and Remote Sensing, 2010, 49 (3): 1092- 1103.
4	SANTAMARIA C , ALVAREZ M , GREIDANUS H , et al. Mass processing of Sentinel-1 images for maritime surveillance[J]. Remote Sensing, 2017, 9 (7): 678. doi: 10.3390/rs9070678
5	PELICH R , CHINI M , HOSTACHE R , et al. Large-scale automatic vessel monitoring based on dual-polarization Sentinel-1 and AIS data[J]. Remote Sensing, 2019, 11 (9): 1078. doi: 10.3390/rs11091078
6	UIBOUPIN R, RAUDSEPP U, SIPELGAS L. Detection of oil spills on SAR images, identification of polluters and forecast of the slicks trajectory[C]//Proc. of the IEEE/OES US/EU-Baltic International Symposium, 2008: 1-5.
7	LONGÉPÉ N , MOUCHE A A , GOACOLOU M , et al. Polluter identification with spaceborne radar imagery, AIS and forward drift modeling[J]. Marine Pollution Bulletin, 2015, 101 (2): 826- 833. doi: 10.1016/j.marpolbul.2015.08.006
8	GARELLO R, KERBAOL V. Oil pollution monitoring: An integrated approach[C]//Proc. of the IEEE Workshop on Environmental, Energy, and Structural Monitoring Systems, 2017: 1-6.
9	SONG J, KIM D. Identification of unclassified ships implementing AIS information and SAR image-based ship detection results[C]//Proc. of the IEEE International Geoscience and Remote Sensing Symposium, 2021: 3557-3560.
10	TOUZI R , VACHON P W . RCM polarimetric SAR for enhanced ship detection and classification[J]. Canadian Journal of Remote Sensing, 2015, 41 (5): 473- 484. doi: 10.1080/07038992.2015.1110010
11	VELOTTO D , BENTES C , TINGS B , et al. First comparison of Sentinel-1 and TerraSAR-X data in the framework of maritime targets detection: South Italy case[J]. IEEE Journal of Oceanic Engineering, 2016, 41 (4): 993- 1006. doi: 10.1109/JOE.2016.2520216
12	KUREKIN A A , LOVEDAY B R , CLEMENTS O , et al. Operational monitoring of illegal fishing in Ghana through exploitation of satellite earth observation and AIS data[J]. Remote Sensing, 2019, 11 (3): 293. doi: 10.3390/rs11030293
13	LONGÉPÉ N , HAJDUCH G , ARDIANTO R , et al. Completing fishing monitoring with spaceborne vessel detection system (VDS) and automatic identification system (AIS) to assess illegal fishing in Indonesia[J]. Marine Pollution Bulletin, 2018, 131, 33- 39. doi: 10.1016/j.marpolbul.2017.10.016
14	ROWLANDS G , BROWN J , SOULE B , et al. Satellite surveillance of fishing vessel activity in the ascension island exclusive economic zone and marine protected area[J]. Marine Policy, 2019, 101, 39- 50. doi: 10.1016/j.marpol.2018.11.006
15	International Telecommunication Union. Technical characteristics for an automatic identification system using time division multiple access in the VHF maritime mobile frequency band[EB/OL]. [2022-03-18]. https://www.itu.int/dms_pubrec/itu-r/rec/m/R-REC-M.1371-5-201402-I!!PDF-E.pdf
16	BAR-SHALOM Y , FORTMANN T E , CABLE P G . Tracking and data association[J]. Acoustical Society of America Journal, 1990, 87 (2): 918- 919. doi: 10.1121/1.398863
17	ACHIRI L, GUIDA R, IERVOLINO P. SAR and AIS fusion for maritime surveillance[C]//Proc. of the IEEE 4th International Forum on Research and Technology for Society and Industry (RTSI), 2018: 1-4.
18	LANG H T , WU S W , XU Y J . Ship classification in SAR images improved by AIS knowledge transfer[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 15 (3): 439- 443. doi: 10.1109/LGRS.2018.2792683
19	GRAZIANO M D , RENGA A , MOCCIA A . Integration of automatic identification system (AIS) data and single-channel synthetic aperture radar (SAR) images by SAR-based ship velocity estimation for maritime situational awareness[J]. Remote Sensing, 2019, 11 (19): 2196. doi: 10.3390/rs11192196
20	RODGER M , GUIDA R . Classification-aided SAR and AIS data fusion for space-based maritime surveillance[J]. Remote Sensing, 2020, 13 (1): 104. doi: 10.3390/rs13010104
21	WANG L W , LI Y , HUANG J , et al. Learning two-branch neural networks for image-text matching tasks[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2018, 41 (2): 394- 407.
22	YAO S S, NIU B N, LIU J Q. Enhancing sampling and counting method for audio retrieval with time-stretch resistance[C]// Proc. of the IEEE Fourth International Conference on Multimedia Big Data (BigMM), 2018: 1-5.
23	TIAN S , YIN X C , SU Y , et al. A unified framework for tracking based text detection and recognition from web videos[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2017, 40 (3): 542- 554.
24	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.
25	DENG Z, HU X, ZHU L, et al. R3net: recurrent residual refinement network for saliency detection[C]//Proc. of the 27th International Joint Conference on Artificial Intelligence, 2018: 684-690.
26	XU K, BA J, KIROS R, et al. Show, attend and tell: neural image caption generation with visual attention[C]//Proc. of the International Conference on Machine Learning, 2015: 2048-2057.
27	FAGHRI F, FLEET D J, KIROS J R, et al. Vse++: improving visual-semantic embeddings with hard negatives[J]. arXiv preprint arXiv: 1707. 05612, 2017.
28	ZHANG T W , ZHANG X L , LI J W , et al. SAR ship detection dataset (SSDD): official release and comprehensive data analysis[J]. Remote Sensing, 2021, 13 (18): 3690. doi: 10.3390/rs13183690
29	WANG K Y, YIN Q Y, WANG W, et al. A comprehensive survey on cross-modal retrieval[J]. arXiv preprint arXiv: 1607. 06215, 2016.
30	LEE K H, CHEN X, HUA G, et al. Stacked cross attention for image-text matching[C]//Proc. of the European Conference on Computer Vision (ECCV), 2018: 201-216.
31	YUAN Z Q , ZHANG W K , FU K , et al. Exploring a fine-grained multiscale method for cross-modal remote sensing image retrieval[J]. IEEE Trans.on Geoscience and Remote Sensing, 2021, 60, 1- 19.

网络层	输入尺寸	输出尺寸	网络参数
Layer1	256×256	128×128	7×7, 64, 2
Layer2	128×128	64×64	3×3 maxpool, 2 [3×3, 64 3×3, 64]×2
Layer3	64×64	32×32	[3×3, 128 3×3, 128]×2
Layer4	32×32	16×16	[3×3, 256 3×3, 256]×2
Layer5	16×16	8×8	[3×3, 512 3×3, 512]×2

网络模型	输入维度	输出维度	网络参数
GRU	300	512	6层
1D-CNN	300	512	2层

任务	模型	召回率(Recall)
任务	模型	R@1	R@5	R@10	R_m
(SAR-＞AIS)	A-MFMN	50.77	56.92	77.23	68.82 (AMFMN)
	SCAN	55.38	61.54	70.77	68.82 (AMFMN)
	本文	68.31	74.15	84.92	70.00 (SCAN)
(AIS-＞SAR)	AMFMN	53.54	85.54	89.54	70.00 (SCAN)
	SCAN	58.46	84.62	89.23	79.44 (本文)
	本文	67.69	89.85	91.69	79.44 (本文)

模型	(SAR-＞AIS)				(AIS-＞SAR)		R_m
模型	R@1	R@5	R@10	R@1	R@5	R@10	R_m
M1	32.92	36.92	53.54	32.00	77.85	86.46	53.28
M2	51.38	58.75	73.23	54.46	85.23	89.54	68.67
M3	57.85	65.23	79.08	60.62	89.85	92.62	74.21
M4	62.46	70.46	81.85	67.08	87.69	91.08	76.77
M5	56.62	63.69	79.38	60.92	85.54	89.23	72.56
M6	60.13	65.54	81.54	63.08	88.92	90.77	75.03
M7	64.62	71.38	83.38	66.46	87.69	90.15	77.28
M8	67.08	72.92	83.08	66.77	89.23	90.77	78.31
M	68.31	74.15	84.92	67.69	89.85	91.69	79.44

[1]	Meng WANG, Bing ZHU. Application of uncertainty modeling in 2D and 3D object detection [J]. Systems Engineering and Electronics, 2023, 45(8): 2370-2376.
[2]	Kai SHAO, Ziqun DU, Guangyu WANG. CSI feedback method for dynamically adjusting compression rate based on model pruning [J]. Systems Engineering and Electronics, 2023, 45(8): 2615-2622.
[3]	Tianshu CUI, Dong WANG, Zhen HUANG. Automatic modulation classification based on lightweight network for space cognitive communication [J]. Systems Engineering and Electronics, 2023, 45(7): 2220-2226.
[4]	Yu JIANG, Qi YUAN, Zhitao HU, Weiwei WU, Xin GU. Airport arrival and departure delay time prediction based on meteorological factors [J]. Systems Engineering and Electronics, 2023, 45(6): 1722-1731.
[5]	Qian CHENG, Jia LI, Juan DU. Ship target detection algorithm of optical remote sensing image based on YOLOv5 [J]. Systems Engineering and Electronics, 2023, 45(5): 1270-1276.
[6]	Yang CHEN, Canhui LIAO, Kun ZHANG, Jian LIU, Pengju WANG. A signal modulation indentification algorithm based on self-supervised contrast learning [J]. Systems Engineering and Electronics, 2023, 45(4): 1200-1206.
[7]	Ye ZHANG, Yi HOU, Kewei OUYANG, Shilin ZHOU. Survey of univariate sequence data classification methods [J]. Systems Engineering and Electronics, 2023, 45(2): 313-335.
[8]	Zhengtu SHAO, Dengrong XU, Wenli XU, Hanzhong WANG. Radar active jamming recognition based on LSTM and residual network [J]. Systems Engineering and Electronics, 2023, 45(2): 416-423.
[9]	Renfei CHEN, Yong PENG, Zhongwen LI. A novel detector for floating objects based on continual unsupervised domain adaptation strategy [J]. Systems Engineering and Electronics, 2023, 45(11): 3391-3401.
[10]	Chensong TAO, Siwei CHEN, Shunping XIAO. SAR image interrupted sampling repeater jamming detection based on deep learning models [J]. Systems Engineering and Electronics, 2023, 45(11): 3465-3473.
[11]	Shiyang HE, Ling WANG, Daiyin ZHU, Jun QIAN. Thunderstorm prediction method based on spatiotemporal memory decoupling RNN [J]. Systems Engineering and Electronics, 2023, 45(11): 3474-3480.
[12]	Zichao LIU, Jiang WANG, Shaoming HE. Time and angle control guidance law based on deep learning [J]. Systems Engineering and Electronics, 2023, 45(11): 3579-3587.
[13]	Ce JI, Bohan SONG, Rong GENG, Minjun LIANG. Deep learning based channel estimation for OFDM systems in fast time-varying channel [J]. Systems Engineering and Electronics, 2023, 45(11): 3649-3655.
[14]	Jianli DING, Qiqi ZHANG, Jing WANG, Weigang HUO. ADS-B anomaly detection method based on Transformer-VAE [J]. Systems Engineering and Electronics, 2023, 45(11): 3680-3689.
[15]	Ting SONG, Zezhao WU, Ai GAO, Jianping YUAN. CycleGAN-based data enhancement method for lunar surface images [J]. Systems Engineering and Electronics, 2023, 45(10): 3041-3048.

An association method between SAR images and AIS information based on depth feature fusion

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 31

Related Articles 15

Recommended Articles

Metrics

Comments