基于CSLS-CycleGAN的侧扫声纳水下目标图像样本扩增法

doi:10.12305/j.issn.1001-506X.2024.05.06

系统工程与电子技术 ›› 2024, Vol. 46 ›› Issue (5): 1514-1524.doi: 10.12305/j.issn.1001-506X.2024.05.06

• 电子技术 • 上一篇

基于CSLS-CycleGAN的侧扫声纳水下目标图像样本扩增法

汤寓麟¹, 王黎明¹^,*, 余德荧¹, 李厚朴¹, 刘敏², 张卫东³

1. 海军工程大学电气工程学院, 湖北武汉 430033
2. 中国人民解放军91001部队, 北京 100841
3. 中国人民解放军31016部队, 北京 100088

收稿日期:2023-03-25 出版日期:2024-04-30 发布日期:2024-04-30
通讯作者: 王黎明
作者简介:汤寓麟(1996—), 男, 博士研究生, 主要研究方向为智能感知与智能系统、水下目标检测和计算机视觉
王黎明(1978—), 男, 副教授, 博士, 主要研究方向为无人装备智能感知与自主控制
余德荧(1998—), 男, 博士研究生, 主要研究方向为北斗/GNSS精密定位算法、海洋应用
李厚朴(1985—), 男, 教授, 博士, 主要研究方向为大地测量数学分析
刘敏(1980—), 男, 高级工程师, 博士, 主要研究方向为大地测量海空重力测量与数据处理技术、海洋测绘及海洋地理信息应用
张卫东(1981—), 男, 工程师, 主要研究方向为地理信息系统
基金资助:
国家优秀青年科学基金(42122025);国家自然科学基金(41974005);国家自然科学基金(41971416);国家自然科学基金(42074074);湖北省杰出青年科学基金(2019CFA086)

CSLS-CycleGAN based side-scan sonar sample augmentation method for underwater target image

Yulin TANG¹, Liming WANG¹^,*, Deying YU¹, Houpu LI¹, Min LIU², Weidong ZHANG³

1. College of Electrical Engineering, Naval University of Engineering, Wuhan 430033, China
2. Unit 91001 of the PLA, Beijing 100841, China
3. Unit 31016 of the PLA, Beijing 100088, China

Received:2023-03-25 Online:2024-04-30 Published:2024-04-30
Contact: Liming WANG

摘要/Abstract

摘要：

针对侧扫声纳水下目标图像稀缺，获取难度大、成本高，导致基于深度学习的目标检测模型性能差的问题，结合光学域类目标数据集丰富的现状，提出一种基于通道和空间注意力(channel and spatial attention, CSA)模块、最小二乘生成对抗生成网络(least squares generative adversarial networks, LSGAN)及循环对抗生成网络(cycle generative adversarial networks, CycleGAN)的侧扫声纳水下目标图像样本扩增方法。首先，受CycleGAN的启发，设计基于循环一致性的单循环网络结构，保证模型的训练效率。然后，在生成器中融合CSA模块，减少信息弥散的同时增强跨纬度交互。最后，设计了基于LSGAN的损失函数，提高生成图像质量的同时提高训练稳定性。在船舶光学域数据集与侧扫声纳沉船数据集上进行实验，所提方法实现了光学-侧扫声纳样本间信息的高效、稳健转换以及大量侧扫声纳目标样本的扩增。同时，基于本文生成样本训练后的检测模型进行了水下目标检测，结果表明，使用本文样本扩增数据训练后的模型在少样本沉船目标检测的平均准确率达到了84.71%，证明了所提方法实现了零样本和小样本水下强代表性目标样本的高质量扩增，并为高性能水下目标检测模型构建提供了一种新的途径。

关键词: 样本扩增, 侧扫声纳, 循环生成对抗网络, 通道和空间注意力模块, 最小二乘生成对抗网络

Abstract:

In view of the scarcity, difficulty and high cost of side-scan sonar underwater target images, and the poor performance of deep-learning based target detection model, combined with the abundant target data set in optical domain, a sample augmentation method for underwater target side-scan sonar images based on channel and spatial attention(CSA) module and least squares generative adversarial networks(LSGAN) and cycle generative adversarial networks(CycleGAN) is propesed. Firstly, inspired by CycleGAN, a single cycle network structure based on cycle consistency is designed to ensure the training efficiency of the model. Then, the CSA module is integrated into the generator to reduce information dispersion while enhancing cross-latitude interaction. Finally, a loss function based on LSGAN is designed to improve the quality of the generated image while improving the training stability. Experiments are carried out on ship optical domain data set and side-scan sonar shipwreck data set. The results show that the proposed method achieves efficient and robust conversion of information between optical and side-scan sonar samples and augmentation of a large number of side-scan sonar target samples. At the same time, the underwater target detection is carried out based on the detection model generated after sample training in this paper. The results show that the average precision value of the model after training with sample augmentation data in this paper reachs 84.71% in detecting shipwreck targets with few samples, which proves that the method in this paper achieves high-quality amplification of highly representative underwater target samples with zero samples and small samples. It also provides a new way to construct high-performance underwater target detection model.

Key words: sample augmentation, side-scan sonar, cycle generative adversarial networks (CycleGAN), channel and spatial attention (CSA) module, least squares generative adversarial networks (LSGAN)

中图分类号:

P227

汤寓麟, 王黎明, 余德荧, 李厚朴, 刘敏, 张卫东. 基于CSLS-CycleGAN的侧扫声纳水下目标图像样本扩增法[J]. 系统工程与电子技术, 2024, 46(5): 1514-1524.

Yulin TANG, Liming WANG, Deying YU, Houpu LI, Min LIU, Weidong ZHANG. CSLS-CycleGAN based side-scan sonar sample augmentation method for underwater target image[J]. Systems Engineering and Electronics, 2024, 46(5): 1514-1524.

图/表 15

图1

图2

图3

图4

图5

图6

表1

图7

表2

图8

表3

表4

图9

表5

图10

参考文献 38

1	HENRIKSE L. Real-time underwater object detection based on an electrically scanned high-resolution sonar[C]//Proc. of the IEEE Symposium on Autonomous Underwater Vehicle Technology, 2002.
2	ARSHAD M R . Recent advancement in sensor technology for underwater applications[J]. Indian Journal of Marine Sciences, 2009, 38 (3): 267- 273.
3	GREENE A , RAHMAN A F , KLINE R , et al. Side scan sonar: a cost-efficient alternative method for measuring seagrass cover in shallow environments[J]. Estuarine, Coastal and Shelf Science, 2018, doi: 10.1016/j.ecss,2018.04.017
4	HURTOS N , PALOMERAS N , CARRERA A , et al. Autonomous detection, following and mapping of an underwater chain using sonar[J]. Ocean Engineering, 2017, 130 (1): 336- 350.
5	BUSCOMBE D . Shallow water benthic imaging and substrate characterization using recreational-grade side scan-sonar[J]. Environmental Modelling & Software, 2017, 89, 1- 18.
6	FLOWERS H J , HIGHTOWER J E . A novel approach to surveying sturgeon using side-scan sonar and occupancy modeling[J]. Marine & Coastal Fisheries, 2013, 5 (1): 211- 223.
7	JOHNSON S G , DEAETT M A . The application of automated recognition techniques to side-scan sonar imagery[J]. IEEE Journal of Oceanic Engineering: a Journal Devoted to the Application of Electrical and Electronics Engineering to the Oceanic Environment, 1994, 19 (1): 138- 144.
8	BURGUERA A , BONIN-FONT F . On-line multi-class segmentation of side-scan sonar imagery using an autonomous underwater vehicle[J]. Journal of Marine Science and Engineering, 2020, 8 (8): 557. doi: 10.3390/jmse8080557
9	CHEN E , GUO J . Real time map generation using sidescan sonar scanlines for unmanned underwater vehicles[J]. Ocean Engineering, 2014, 91, 252- 262. doi: 10.1016/j.oceaneng.2014.09.017
10	ZHU B Y , WANG X , CHU Z W , et al. Active learning for recognition of shipwreck target in side-scan sonar image[J]. Remote Sensing, 2019, 11 (3): 243. doi: 10.3390/rs11030243
11	阳凡林, 独知行, 吴自银, 等. 基于灰度直方图和几何特征的声纳图像目标识别[J]. 海洋通报, 2006, 25 (5): 64- 69.
	YANG F L , DU Z X , WU Z Y , et al. Object recognizing on sonar image based on histogram and geometric feature[J]. Marine Science Bulletin, 2006, 25 (5): 64- 69.
12	LANGNER F, KNAUER C, JANS W, et al. Side scan sonar image resolution and automatic object detection, classification and identification[C]//Proc. of the Oceans-Europe Conference, 2009.
13	ISAACS J C. Sonar automatic target recognition for underwater UXO remediation[C]//Proc. of the Computer Vision & Pattern Recognition Workshops, 2015.
14	YUAN F , XIAO F Q , ZHANG K H , et al. Noise reduction for sonar images by statistical analysis and fields of experts[J]. Journal of Visual Communication and Image Representation, 2021, 74, 102995. doi: 10.1016/j.jvcir.2020.102995
15	REED S , PETILLOT Y , BELL J . Automated approach to classification of mine-like objects in sidescan sonar using highlight and shadow information[J]. IEE Proceedings—Radar, Sonar and Navigation, 2004, 151 (1): 48- 56. doi: 10.1049/ip-rsn:20040117
16	KUMAR N , MITRA U , NARAYANAN S S . Robust object classification in underwater sidescan sonar images by using reliability-aware fusion of shadow features[J]. IEEE Journal of Oceanic Engineering, 2015, 40 (3): 592- 606. doi: 10.1109/JOE.2014.2344971
17	ZHU P P, ISAACS J, FU B, et al. Deep learning feature extraction for target recognition and classification in underwater sonar images[C]//Proc. of the IEEE 56th Annual Conference on Decision and Control, 2017.
18	NEUPANE D , SEOK J . A review on deep learning-based approaches for automatic sonar target recognition[J]. Electronics, 2020, 9 (11): 1972.
19	TOPPLE J M , FAWCETT J A . MiNet: efficient deep learning automatic target recognition for small autonomous vehicles[J]. IEEE Geoscience and Remote Sensing Letters, 2021, 18 (6): 1014- 1018.
20	HUO G Y , YANG S X , LI Q , et al. A robust and fast method for sidescan sonar image segmentation using nonlocal despeckling and active contour model[J]. IEEE Trans. on Cybernetics, 2016, 47 (4): 855- 872.
21	FELDENS P , DARR A , FELDENS A , et al. Detection of boulders in side scan sonar mosaics by a neural network[J]. Geosciences, 2019, 9 (4): 159.
22	TANG Y L , JIN S H , BIAN G , et al. Shipwreck target recognition in side-scan sonar images by improved YOLOv3 model based on transfer learning[J]. IEEE Access, 2020, 8, 173450- 173460.
23	汤寓麟, 李厚朴, 张卫东, 等. 侧扫声纳检测沉船目标的轻量化DETR-YOLO法[J]. 系统工程与电子技术, 2022, 44 (8): 2427- 2436.
	TANG Y L , LI H P , ZHANG W D , et al. Lightweight DETR-YOLO method for detecting shipwreck target in side-scan sonar[J]. Systems Engineering and Electronics, 2022, 44 (8): 2427- 2436.
24	NGUYEN H T , LEE E H , LEE S , et al. Study on the classification performance of underwater sonar image classification based on convolutional neural networks for detecting a submerged human body[J]. Sensors, 2019, 20 (1): 94.
25	LI A C , YE A X , CAO B D , et al. Zero shot objects classification method of side scan sonar image based on synthesis of pseudo samples-ScienceDirect[J]. Applied Acoustics, 2021, 173, 107691.
26	COIRAS E , MIGNOTTE P Y , PETILLOT Y , et al. Supervised target detection and classification by training on augmented reality data[J]. IET Radar, Sonar & Navigation, 2007, 1 (1): 83- 90.
27	KAPETANOVI N , MIKOVI N , TAHIROVI A . Saliency and anomaly: transition of concepts from natural images to side-scan sonar images[J]. IFAC-Papers OnLine, 2020, 53 (2): 14558- 14563.
28	LEE S J, PARK B, KIM A. Deep learning from shallow dives: sonar image generation and training for underwater object detection[EB/OL]. [2023-03-10]. arxiv. org/pdf/1810.07990. pdf.
29	HUO G Y , WU Z Y , LI J B . Underwater object classification in sidescan sonar images using deep transfer learning and semisynthetic training data[J]. IEEE Access, 2020, 8, 47407- 47418.
30	HUANG C , ZHAO J H , YU Y C , et al. Comprehensive sample augmentation by fully considering side-scan imaging mechanism and environment for shipwreck detection under zero real samples[J]. IEEE Trans. on Geoscience and Remote Sensing, 2021, 60, 5906814.
31	CHAVEZ P S , ISBRECHT J A , GALANIS P , et al. Processing, mosaicking and management of the monterey bay digital sidescan-sonar images[J]. Marine Geology, 2002, 181 (1/3): 305- 315.
32	BLONDEL P . The handbook of sidescan sonar[M]. Berlin: Springer, 2010.
33	ISOLA P, ZHU J Y, ZHOU T, et al. Image-to-image translation with conditional adversarial networks[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 5967-5976.
34	ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proc. of the IEEE International Conference on Computer Vision, 2017: 2242-2251.
35	李宝奇, 黄海宁, 刘纪元, 等. 基于改进CycleGAN的光学图像迁移生成水下小目标合成孔径声纳图像算法研究[J]. 电子学报, 2021, 49 (9): 1746- 1753.
	LI B Q , HUANG H N , LIU J Y , et al. Optical image-to-underwater small target synthetic aperture sonar image translation algorithm based on improved CycleGAN[J]. Acta Electronica Sinica, 2021, 49 (9): 1746- 1753.
36	ALFARRA M, JUAN C, ANNA F, et al. On the robustness of quality measures for GANs[C]//Proc. of the 17th European Conference on Computer Vision, 2022: 18-33.
37	YI Z L, ZHANG H, TAN P L, et al. DualGAN: unsupervised dual learning for image-to-image translation[C]//Proc. of the IEEE International Conference on Computer Vision, 2017: 2868-2876.
38	KIM T, CHA M. Learning to discover cross-domain relations with generative adversarial networks[C]//Proc. of the 34th International Conference on Machine Learning, 2017: 1857-1865.

类项	参数	类项	参数
批量大小	16	裁剪尺寸	256
λ₁/λ₂	10/0.5	学习率	0.000 2
学习率策略	Linear	学习率衰减迭代次数	50
步数	1 000	优化器	Adam
β	0.5	-	-

组别	模型	FID↓	MMD↓	1-NN↓0.5
1	pix2pix	214.77	0.315	0.97
2	CycleGAN-ResNet-06	153.65	0.192	0.82
3	CycleGAN-ResNet-09	148.71	0.164	0.83
4	CycleGAN-UNet-128	155.12	0.137	0.77
5	CycleGAN-UNet-256	133.69	0.151	0.79
6	DualGAN	130.17	0.160	0.74
7	DiscoGAN	129.98	0.149	0.75
8	所提方法	123.12	0.105	0.72

组别	真实的沉船图像/张	生成的沉船图像/张
A	500	－
B	－	2 000
C	500	2 000
验证	100	－

训练集	查全率	查准率	平均精度(0.5)	平均精度(0.5∶0.95)
YOLOv5-A	79.12	84.14	80.43	46.21
YOLOv5-B	82.47	88.29	83.68	50.66
YOLOv5-C	83.22	88.65	84.71	51.14

组别	CSA模块		LSGAN	FID↓	MMD↓	1-NN↓0.5
组别	通道注意力	空间注意力	LSGAN	FID↓	MMD↓	1-NN↓0.5
1	－	－	－	136.52	0.211	0.91
2	√	－	－	132.95	0.169	0.81
3	－	√	－	133.01	0.152	0.80
4	√	√	－	126.84	0.131	0.77
5	－	－	√	130.98	0.134	0.78
6	√	√	√	123.12	0.102	0.72

基于CSLS-CycleGAN的侧扫声纳水下目标图像样本扩增法

CSLS-CycleGAN based side-scan sonar sample augmentation method for underwater target image

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 15

参考文献 38

相关文章 1

编辑推荐

Metrics

本文评价