基于改进Swin Transformer的舰船目标实例分割算法

doi:10.12305/j.issn.1001-506X.2023.10.07

系统工程与电子技术 ›› 2023, Vol. 45 ›› Issue (10): 3049-3057.doi: 10.12305/j.issn.1001-506X.2023.10.07

• 电子技术 • 上一篇

基于改进Swin Transformer的舰船目标实例分割算法

钱坤¹^,²^,*, 李晨瑄³, 陈美杉¹, 郭继伟², 潘磊²

1. 海军航空大学岸防兵学院, 山东烟台 264000
2. 中国人民解放军32127部队, 辽宁大连 116100
3. 航天工程大学研究生院, 北京 101416

收稿日期:2022-05-09 出版日期:2023-09-25 发布日期:2023-10-11
通讯作者: 钱坤
作者简介:钱坤(1986—), 男, 助理讲师, 博士研究生, 主要研究方向为图像处理、模式识别
李晨瑄(1996—), 女, 博士研究生, 主要研究方向为图像处理、模式识别
陈美杉(1991—), 女, 助理工程师, 博士研究生, 主要研究方向为作战仿真推演
郭继伟(1977—), 男, 讲师, 硕士, 主要研究方向为战役学
潘磊(1979—), 男, 讲师, 硕士, 主要研究方向为战役学
基金资助:
装备预研领域基金(6140247030216JB14004)

Ship target instance segmentation algorithm based on improved Swin Transformer

Kun QIAN¹^,²^,*, Chenxuan LI³, Meishan CHEN¹, Jiwei GUO², Lei PAN²

1. College of Coastal Defense Force, Naval Aeronautical University, Yantai 264000, China
2. Unit 32127 of the PLA, Dalian 116100, China
3. Department of Graduate Management, Space Engineering University, Beijing 101416, China

Received:2022-05-09 Online:2023-09-25 Published:2023-10-11
Contact: Kun QIAN

摘要/Abstract

摘要：

针对反舰武器图像制导目标实例分割精度低, 模型中上下文语义交互不充分, 特征融合推理速度慢, 数据集难易样本不均衡导致训练效果差等问题, 提出了一种基于改进滑动窗口的Transformer(shifted windows Transformer, Swin Transformer)的舰船目标实例分割算法。设计了局部增强感知模块用以拓展感受野, 加强语义交互能力; 采用反向特征金字塔网络进行特征融合, 提高算法处理速度; 使用在线困难样例挖掘, 改善数据集样本不均衡问题, 提升网络训练效果。实验结果表明, 改进后的算法相较基线算法在分割准确率上提升了1.5%, 在处理速度上提高了1.3%, 兼具精度和速度优势。

关键词: Swin Transformer, 反向特征金字塔, 在线困难样例挖掘, 舰船实例分割

Abstract:

Aiming at the problems of low segmentation accuracy of image guidance target instances of anti-ship weapons, insufficient semantic interaction of model context, slow inference speed of feature fusion, imbalance of hard and easy samples of data sets and poor training effect, an instance segmentation algorithm of ship target based on improved Swin Transformer is proposed. The local enhanced sensing block is designed to expand the receptive field and strengthen the ability of semantic interaction. The reverse feature pyramid network is used for feature fusion to improve the processing speed of the algorithm. Online Hard Example Mining is used to improve the sample imbalance of data set and improve the effect of network convergence. The experimental results show that, compared with the baseline algorithm, the improved algorithm improves the segmentation accuracy by 1.5% and the processing speed by 1.3%, with both accuracy and speed advantages.

Key words: shifted windows Transformer (Swin Transformer), reverse feature pyramid, online hard example mining (OHEM), ship instance segmentation

中图分类号:

TP391

钱坤, 李晨瑄, 陈美杉, 郭继伟, 潘磊. 基于改进Swin Transformer的舰船目标实例分割算法[J]. 系统工程与电子技术, 2023, 45(10): 3049-3057.

Kun QIAN, Chenxuan LI, Meishan CHEN, Jiwei GUO, Lei PAN. Ship target instance segmentation algorithm based on improved Swin Transformer[J]. Systems Engineering and Electronics, 2023, 45(10): 3049-3057.

图/表 13

图1

图2

图3

图4

图5

图6

表1

表2

表3

表4

图7

表5

表6

参考文献 30

1	苏丽, 孙雨鑫, 苑守正. 基于深度学习的实例分割研究综述[J]. 智能系统学报, 2022, 17 (1): 16- 31.
	SU L , SUN Y X , YUAN S Z . A survey of instance segmentation research based on deep learning[J]. CAAI Trans.on Intelligent Systems, 2022, 17 (1): 16- 31.
2	HARIHARAN B, ARBELÁEZ P, GIRSHICK R, et al. Simultaneous detection and segmentation[C]//Proc. of the European Conference on Computer Vision, 2014: 297-312.
3	HE K M, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[C]// Proc. of the IEEE International Conference on Computer Vision, 2017: 2980-2988.
4	HUANG Z J, HUANG L C, GONG Y C, et al. Mask scoring R-CNN[C]//Proc. of the Conference on Computer Vision and Pattern Recognition, 2019: 6402-6411.
5	CHENG T H, WANG X G, HUANG L C, et al. Boundary-preserving mask R-CNN[C]//Proc. of the European Conference on Computer Vision, 2020: 660-676.
6	LONG J , SHELHAMER E , DARRELL T . Fully convolutional networks for semantic segmentation[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2015, 39 (4): 640- 651.
7	BOLYA D, ZHOU C, XIAO F Y, et al. YOLACT: real-time instance segmentation[C]//Proc. of the IEEE/CVF International Conference on computer Vision, 2019.
8	BOLYA D , ZHOU C , XIAO F Y , et al. YOLACT++: better real-time instance segmentation[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2022, 44 (2): 1108- 1121. doi: 10.1109/TPAMI.2020.3014297
9	ASHISH V, NOAM S, NIKI P, et al. Attention is all you need[EB/OL]. [2022-05-09]. https://arxiv.org/abs/1706.03762v5.
10	HU J, CAO L J, LU Y, et al. ISTR: end-to-end instance segmentation with Transformers[EB/OL]. [2022-05-09]. https://arxiv.org/abs/2011.14503v4.
11	GUO R H, NIU D T, QU L, et al. SOTR: segmenting objects with Transformers[C]//Proc. of the Conference on Computer Vision and Pattern Recognition, 2021: 7157-7166.
12	LIU Z, LIN Y T, CAO Y, et al. Swin Transformer: hierarchical vision Transformer using shifted windows[C]//Proc. of the International Conference on Computer Vision, 2021: 10012-10022.
13	霍熠阳, 于涛, 高飞. 基于CenterMask的SAR舰船实例分割[C]// 第十三届全国DSP应用技术学术会议论文集, 2021: 150-155.
	HUO Y Y, YU T, GAO F. SAR ship instance segmentation based on CenterMask[C]//Proc. of the 13th National Confe-rence on DSP Application Technology, 2021: 150-155.
14	ZAREMBA W, SUTSKEVER I, VINYALS O. Recurrent neural network regularization[EB/OL]. [2022-05-09]. https://arxiv.org/abs/1409.2329.
15	HENDRYCKS D, GIMPEL K. Gaussian error linear units (GELUs)[EB/OL]. [2022-05-09]. https://arxiv.org/abs/1606.08415v4.
16	GLOROT X, BORDES A, BENGIO Y. Deep sparse rectifier neural networks[C]//Proc. of the 14th International Confe-rence on Artificial Intelligence and Statistics, 2011: 315-323.
17	ZONG Z F, CAO Q G, LENG B. RCNet: reverse feature pyramid and cross-scale shift network for object detection[C]// Proc. of the 29th ACM International Conference on Multimedia, 2021: 5637-5645.
18	SHRIVASTAVA A, GUPTA A, GIRSHICK R. Training region- based object detectors with online hard example mining[C]// Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 761-769.
19	CHEN L C, PAPANDREOU G, KOKKINOS I, et al. Semantic image segmentation with deep convolutional nets and fully connected CRFs[C]//Proc. of the Internation Conference on Learming Representation, 2015.
20	CHEN L C , PAPANDREOU G , KOKKINOS I , et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2018, 40 (4): 834- 848. doi: 10.1109/TPAMI.2017.2699184
21	CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segmentation[EB/OL]. [2022-05-09]. https://arxiv.org/abs/1706.05587v3
22	CHEN L C, ZHU Y, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proc. of the European Conference on Computer Vision, 2018: 801-818.
23	YU F, KOLTUN V. Multi-scale context aggregation by dilated convolutions[EB/OL]. [2022-05-09]. https://arxiv.org/abs/1511.07122.
24	李晨瑄, 钱坤, 胥辉旗. 基于深浅层特征融合的舰船要害关键点检测算法[J]. 系统工程与电子技术, 2021, 43 (11): 3239- 3249.
	LI C X , QIAN K , XU H Q . Key-points detection algorithm based on fusion of deep and shallow features for warship's vital part[J]. Systems Engineering and Electronics, 2021, 43 (11): 3239- 3249.
25	LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 936-944.
26	LIU S, QI L, QIN H, et al. Path aggregation network for instance segmentation[C]//Proc. of the Conference on Computer Vision and Pattern Recognition, 2018: 8759-8768.
27	TAN M, PANG R, LE Q V. EfficientDet: scalable and efficient object detection[C]//Proc. of the Conference on Computer Vision and Pattern Recognition, 2020: 10778-10787.
28	钱坤, 李晨瑄, 陈美杉, 等. 基于YOLOv5的舰船目标及关键部位检测算法[J]. 系统工程与电子技术, 2022, 44 (6): 1823- 1832.
	QIAN K , LI C X , CHEN M S , et al. Ship target and key parts detection algorithm based on YOLOv5[J]. Systems Engineering and Electronics, 2022, 44 (6): 1823- 1832.
29	LIN T Y, MAIRE M, BELONGIE S, et al. Microsoft COCO: common objects in context[C]//Proc. of the European Confe-rence on Computer Vision, 2014: 740-755.
30	CAI Z W, VASCONCELOS N. Cascade R-CNN: delving into high quality object detection[C]//Proc. of the Conference on Computer Vision and Pattern Recognition, 2018: 6154-6162.

标准类别	数据集总数	标注数量	航空母舰	驱逐舰	民用船舶
数量/幅	1 554	1 052	636	741	558
占比/%	-	100	60.5	70.4	53.0

项目	硬件指标(软件版本)
操作系统	Windows10
CPU	i7-9700 3.00 GHz
GPU	NVIDIA 2060 SUPER 8GB
内存/GB	16
加速环境	CUDA v11.0、CuDNN v7.6
PyTorch版本	1.7
Python版本	3.8.5

评价指标	含义
AP^segm	IoU=0.50:0.05:0.95
mAP^segm	所有类别AP的平均值
AP₅₀^segm	IoU=0.50
AP₇₅^segm	IoU=0.75
参数量	网络参数量
帧率/PFS	每秒内可以处理的图像数量

算法	主干网络	mAP^segm/%	AP₅₀^segm/%	AP₇₅^segm/%	参数量/MB	FPS
FCN^[6]	VGG16	62.2	79.5	69.2	134	13.3
Mask R-CNN^[5]	ResNet-50	69.4	89.6	77.9	110	15.0
Cascade Mask R-CNN^[30]	ResNet-50	68.7	89.4	76.3	82	18.0
YOLACT++^[8]	ResNet-50	66.3	82.3	74.6	129	32.6
基线算法	Swin-Ting	73.9	90.8	87.5	86	15.3
本文算法	改进Swin-Ting	75.4	91.3	89.4	89	15.5

算法	LES	RevFP	OHEM	mAP^segm/%	FPS
基线算法	×	×	×	73.9	15.3
消融实验1	√	×	×	74.8	15.0
消融实验2	×	√	×	74.3	15.7
消融实验3	×	×	√	74.6	15.3
消融实验4	√	√	×	75.1	15.4
消融实验5	√	×	√	75.1	15.0
消融实验6	×	√	√	74.8	15.7
本文算法	√	√	√	75.4	15.5

基于改进Swin Transformer的舰船目标实例分割算法

Ship target instance segmentation algorithm based on improved Swin Transformer

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 13

参考文献 30

相关文章 1

编辑推荐

Metrics

本文评价

算法	训练集大小	mAP
基线算法	60	61.8
	80	68.1
	100	73.9
消融实验3	60	63.0
	80	69.0
	100	74.6