基于改进Swin Transformer的舰船目标实例分割算法

doi:10.12305/j.issn.1001-506X.2023.10.07

Abstract

Abstract:

Aiming at the problems of low segmentation accuracy of image guidance target instances of anti-ship weapons, insufficient semantic interaction of model context, slow inference speed of feature fusion, imbalance of hard and easy samples of data sets and poor training effect, an instance segmentation algorithm of ship target based on improved Swin Transformer is proposed. The local enhanced sensing block is designed to expand the receptive field and strengthen the ability of semantic interaction. The reverse feature pyramid network is used for feature fusion to improve the processing speed of the algorithm. Online Hard Example Mining is used to improve the sample imbalance of data set and improve the effect of network convergence. The experimental results show that, compared with the baseline algorithm, the improved algorithm improves the segmentation accuracy by 1.5% and the processing speed by 1.3%, with both accuracy and speed advantages.

Key words: shifted windows Transformer (Swin Transformer), reverse feature pyramid, online hard example mining (OHEM), ship instance segmentation

CLC Number:

TP391

Kun QIAN, Chenxuan LI, Meishan CHEN, Jiwei GUO, Lei PAN. Ship target instance segmentation algorithm based on improved Swin Transformer[J]. Systems Engineering and Electronics, 2023, 45(10): 3049-3057.

Figures/Tables 13

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Table 1

Table 2

Table 3

Table 4

Fig.7

Table 5

Table 6

References 30

1	苏丽, 孙雨鑫, 苑守正. 基于深度学习的实例分割研究综述[J]. 智能系统学报, 2022, 17 (1): 16- 31.
	SU L , SUN Y X , YUAN S Z . A survey of instance segmentation research based on deep learning[J]. CAAI Trans.on Intelligent Systems, 2022, 17 (1): 16- 31.
2	HARIHARAN B, ARBELÁEZ P, GIRSHICK R, et al. Simultaneous detection and segmentation[C]//Proc. of the European Conference on Computer Vision, 2014: 297-312.
3	HE K M, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[C]// Proc. of the IEEE International Conference on Computer Vision, 2017: 2980-2988.
4	HUANG Z J, HUANG L C, GONG Y C, et al. Mask scoring R-CNN[C]//Proc. of the Conference on Computer Vision and Pattern Recognition, 2019: 6402-6411.
5	CHENG T H, WANG X G, HUANG L C, et al. Boundary-preserving mask R-CNN[C]//Proc. of the European Conference on Computer Vision, 2020: 660-676.
6	LONG J , SHELHAMER E , DARRELL T . Fully convolutional networks for semantic segmentation[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2015, 39 (4): 640- 651.
7	BOLYA D, ZHOU C, XIAO F Y, et al. YOLACT: real-time instance segmentation[C]//Proc. of the IEEE/CVF International Conference on computer Vision, 2019.
8	BOLYA D , ZHOU C , XIAO F Y , et al. YOLACT++: better real-time instance segmentation[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2022, 44 (2): 1108- 1121. doi: 10.1109/TPAMI.2020.3014297
9	ASHISH V, NOAM S, NIKI P, et al. Attention is all you need[EB/OL]. [2022-05-09]. https://arxiv.org/abs/1706.03762v5.
10	HU J, CAO L J, LU Y, et al. ISTR: end-to-end instance segmentation with Transformers[EB/OL]. [2022-05-09]. https://arxiv.org/abs/2011.14503v4.
11	GUO R H, NIU D T, QU L, et al. SOTR: segmenting objects with Transformers[C]//Proc. of the Conference on Computer Vision and Pattern Recognition, 2021: 7157-7166.
12	LIU Z, LIN Y T, CAO Y, et al. Swin Transformer: hierarchical vision Transformer using shifted windows[C]//Proc. of the International Conference on Computer Vision, 2021: 10012-10022.
13	霍熠阳, 于涛, 高飞. 基于CenterMask的SAR舰船实例分割[C]// 第十三届全国DSP应用技术学术会议论文集, 2021: 150-155.
	HUO Y Y, YU T, GAO F. SAR ship instance segmentation based on CenterMask[C]//Proc. of the 13th National Confe-rence on DSP Application Technology, 2021: 150-155.
14	ZAREMBA W, SUTSKEVER I, VINYALS O. Recurrent neural network regularization[EB/OL]. [2022-05-09]. https://arxiv.org/abs/1409.2329.
15	HENDRYCKS D, GIMPEL K. Gaussian error linear units (GELUs)[EB/OL]. [2022-05-09]. https://arxiv.org/abs/1606.08415v4.
16	GLOROT X, BORDES A, BENGIO Y. Deep sparse rectifier neural networks[C]//Proc. of the 14th International Confe-rence on Artificial Intelligence and Statistics, 2011: 315-323.
17	ZONG Z F, CAO Q G, LENG B. RCNet: reverse feature pyramid and cross-scale shift network for object detection[C]// Proc. of the 29th ACM International Conference on Multimedia, 2021: 5637-5645.
18	SHRIVASTAVA A, GUPTA A, GIRSHICK R. Training region- based object detectors with online hard example mining[C]// Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 761-769.
19	CHEN L C, PAPANDREOU G, KOKKINOS I, et al. Semantic image segmentation with deep convolutional nets and fully connected CRFs[C]//Proc. of the Internation Conference on Learming Representation, 2015.
20	CHEN L C , PAPANDREOU G , KOKKINOS I , et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2018, 40 (4): 834- 848. doi: 10.1109/TPAMI.2017.2699184
21	CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segmentation[EB/OL]. [2022-05-09]. https://arxiv.org/abs/1706.05587v3
22	CHEN L C, ZHU Y, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proc. of the European Conference on Computer Vision, 2018: 801-818.
23	YU F, KOLTUN V. Multi-scale context aggregation by dilated convolutions[EB/OL]. [2022-05-09]. https://arxiv.org/abs/1511.07122.
24	李晨瑄, 钱坤, 胥辉旗. 基于深浅层特征融合的舰船要害关键点检测算法[J]. 系统工程与电子技术, 2021, 43 (11): 3239- 3249.
	LI C X , QIAN K , XU H Q . Key-points detection algorithm based on fusion of deep and shallow features for warship's vital part[J]. Systems Engineering and Electronics, 2021, 43 (11): 3239- 3249.
25	LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 936-944.
26	LIU S, QI L, QIN H, et al. Path aggregation network for instance segmentation[C]//Proc. of the Conference on Computer Vision and Pattern Recognition, 2018: 8759-8768.
27	TAN M, PANG R, LE Q V. EfficientDet: scalable and efficient object detection[C]//Proc. of the Conference on Computer Vision and Pattern Recognition, 2020: 10778-10787.
28	钱坤, 李晨瑄, 陈美杉, 等. 基于YOLOv5的舰船目标及关键部位检测算法[J]. 系统工程与电子技术, 2022, 44 (6): 1823- 1832.
	QIAN K , LI C X , CHEN M S , et al. Ship target and key parts detection algorithm based on YOLOv5[J]. Systems Engineering and Electronics, 2022, 44 (6): 1823- 1832.
29	LIN T Y, MAIRE M, BELONGIE S, et al. Microsoft COCO: common objects in context[C]//Proc. of the European Confe-rence on Computer Vision, 2014: 740-755.
30	CAI Z W, VASCONCELOS N. Cascade R-CNN: delving into high quality object detection[C]//Proc. of the Conference on Computer Vision and Pattern Recognition, 2018: 6154-6162.

标准类别	数据集总数	标注数量	航空母舰	驱逐舰	民用船舶
数量/幅	1 554	1 052	636	741	558
占比/%	-	100	60.5	70.4	53.0

项目	硬件指标(软件版本)
操作系统	Windows10
CPU	i7-9700 3.00 GHz
GPU	NVIDIA 2060 SUPER 8GB
内存/GB	16
加速环境	CUDA v11.0、CuDNN v7.6
PyTorch版本	1.7
Python版本	3.8.5

评价指标	含义
AP^segm	IoU=0.50:0.05:0.95
mAP^segm	所有类别AP的平均值
AP₅₀^segm	IoU=0.50
AP₇₅^segm	IoU=0.75
参数量	网络参数量
帧率/PFS	每秒内可以处理的图像数量

算法	主干网络	mAP^segm/%	AP₅₀^segm/%	AP₇₅^segm/%	参数量/MB	FPS
FCN^[6]	VGG16	62.2	79.5	69.2	134	13.3
Mask R-CNN^[5]	ResNet-50	69.4	89.6	77.9	110	15.0
Cascade Mask R-CNN^[30]	ResNet-50	68.7	89.4	76.3	82	18.0
YOLACT++^[8]	ResNet-50	66.3	82.3	74.6	129	32.6
基线算法	Swin-Ting	73.9	90.8	87.5	86	15.3
本文算法	改进Swin-Ting	75.4	91.3	89.4	89	15.5

算法	LES	RevFP	OHEM	mAP^segm/%	FPS
基线算法	×	×	×	73.9	15.3
消融实验1	√	×	×	74.8	15.0
消融实验2	×	√	×	74.3	15.7
消融实验3	×	×	√	74.6	15.3
消融实验4	√	√	×	75.1	15.4
消融实验5	√	×	√	75.1	15.0
消融实验6	×	√	√	74.8	15.7
本文算法	√	√	√	75.4	15.5

Ship target instance segmentation algorithm based on improved Swin Transformer

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 13

References 30

Related Articles 15

Recommended Articles

Metrics

Comments

算法	训练集大小	mAP
基线算法	60	61.8
	80	68.1
	100	73.9
消融实验3	60	63.0
	80	69.0
	100	74.6

[1]	Haigang SUI, Jiajie LI, Guohua GOU. Online fast localization method of UAVs based on heterologous image matching [J]. Systems Engineering and Electronics, 2023, 45(10): 3008-3015.
[2]	Shan GAO, Yongfeng ZHI, Pu ZHANG, Xuan ZUO. Consistency validation method of simulation results based on improved grey relational analysis for aerospace product performance prototype [J]. Systems Engineering and Electronics, 2023, 45(9): 2777-2783.
[3]	Haoliang REN, Jianchao ZHANG, Huichuan CHENG. Modeling and analysis method of weapon equipment system capability requirements based on SysML [J]. Systems Engineering and Electronics, 2023, 45(9): 2843-2851.
[4]	Zhenhai XIE, Ming HE, Minggang YU, Kaohua YU, Guodong YUAN. Modeling and simulation of cooperative evolution of unmanned swarms for strategy diversity [J]. Systems Engineering and Electronics, 2023, 45(9): 2852-2859.
[5]	Haijun LI, Fancheng KONG, Yun LIN. Infrared ship detection algorithm based on improved YOLOv5s [J]. Systems Engineering and Electronics, 2023, 45(8): 2415-2422.
[6]	Huiying WANG, Chunping WANG, Qiang FU, Zishuo HAN, Dongdong ZHANG. Infrared and low illumination image fusion based on image features [J]. Systems Engineering and Electronics, 2023, 45(8): 2395-2404.
[7]	Meng WANG, Bing ZHU. Application of uncertainty modeling in 2D and 3D object detection [J]. Systems Engineering and Electronics, 2023, 45(8): 2370-2376.
[8]	Yanyan HUANG, Kaisheng WANG, Yu'ang SHI. Research on an evaluation model for data link operational support capability based on networking index [J]. Systems Engineering and Electronics, 2023, 45(8): 2361-2369.
[9]	Fan YANG, Ping MA, Wei LI, Ming YANG. Intelligent ranking evaluation method of simulation models based on siamese network [J]. Systems Engineering and Electronics, 2023, 45(7): 2060-2068.
[10]	Siqiang DONG, Nianmao DENG, Yan LIU. Optimization method of pixel pose location based on multi-scale features [J]. Systems Engineering and Electronics, 2023, 45(7): 2203-2210.
[11]	Jingrong SUN, Zhezhe CHEN, Linchang XIE, Mengxin DU, Shibin SONG. Haze removal algorithm based on image sky region segmentation [J]. Systems Engineering and Electronics, 2023, 45(6): 1606-1615.
[12]	Qian CHENG, Jia LI, Juan DU. Ship target detection algorithm of optical remote sensing image based on YOLOv5 [J]. Systems Engineering and Electronics, 2023, 45(5): 1270-1276.
[13]	Xin GUAN, Jiaen GUO, Xiao YI. Ship target recognition based on low rank bilinear pooling attention network [J]. Systems Engineering and Electronics, 2023, 45(5): 1305-1314.
[14]	Li CHEN, Zihan FANG, Liquan MEI. DSS signal generation algorithm based on GAN [J]. Systems Engineering and Electronics, 2023, 45(5): 1544-1552.
[15]	Weiguang FANG, Zhaowei NIE, Chenning LIU, Hao LI, Yang NA, Huixiong WANG, Dongpao HONG. Research on digital twin driven intelligent weaponry support technology [J]. Systems Engineering and Electronics, 2023, 45(4): 1247-1260.