融合域自适应和弱监督策略的遥感影像建筑物提取算法

doi:10.12305/j.issn.1001-506X.2023.06.04

摘要/Abstract

摘要：

常见的建筑物提取算法主要采用全监督的方式实现, 得到的模型通常在训练数据集上表现良好, 而在跨区域使用时效果不佳。基于生成对抗网络的域自适应方法虽然在一定程度上能够增强网络的迁移能力, 但由于缺乏目标域关键信息, 效果难以保证。设计了一种全新的端到端弱监督建筑物提取网络。首先, 采用像素关联模块来提升生成网络的性能; 然后, 在此基础上综合运用域自适应和图像级弱标签两种策略来优化训练过程, 从而大幅提升了网络模型的泛化扩展性能。采用3组数据对所提方法的有效性进行了验证，通过大量实验证明了所提方法可以有效提升建筑物提取的性能。同时，通过消融实验验证了网络中各个模块的有效性。

关键词: 语义分割, 建筑物提取, 弱监督, 域自适应, 生成对抗网络

Abstract:

Common building extraction algorithms were mainly implemented in a fully-supervised manner, and the resulting model usually only perform well on training data sets, but operate badly on cross domain. The domain adaptation method based on generative adversarial network (GAN) can enhance the migration ability of the network to a certain extent. However, due to the lack of key information of the target domain, it is difficult to guarantee its ideal performance. A new end-to-end weakly-supervised building extraction network is proposed in this paper, which adopts pixel correlation module (PCM) to improve the performance of the generative network, and then based on this, uses two strategies, domain adaptation and image-level weak label, to optimize the training process, thereby improving the ability of generalization and expansion of the network greatly. Three datasets are used to verify the effectiveness of the proposed method. Extensive experiments show that the proposed method can effectively improve the performance of building extraction. At the same time, ablation studies are conducted to verify the effectiveness of each module in the network.

Key words: semantic segmentation, building extraction, weakly supervision, domain adaptation, generative adversarial network (GAN)

中图分类号:

P237

庞世燕, 郝京京, 邢立宁, 谭旭. 融合域自适应和弱监督策略的遥感影像建筑物提取算法[J]. 系统工程与电子技术, 2023, 45(6): 1616-1623.

Shiyan PANG, Jingjing HAO, Lining XING, Xu TAN. Weakly supervised domain adaptation algorithm for building extraction from remote sensing images[J]. Systems Engineering and Electronics, 2023, 45(6): 1616-1623.

图/表 12

图1

图2

图3

表1

表2

表3

表4

表5

表6

图4

表7

表8

参考文献 30

12	XU J, SCHWING A G, URTASUN R. Learning to segment under various forms of weak supervision[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2015.
13	ARASLANOV N, ROTH S. Single-stage semantic segmentation from image labels[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 4253- 4262.
14	FAN J S, ZHANG Z X, SONG C, et al. Learning integral objects with intra-class discriminator for weakly-supervised semantic segmentation[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
15	LIU J H , YU C Q , YANG B B , et al. CSENet: cascade semantic erasing network for weakly-supervised semantic segmentation[J]. Neurocomputing, 2021, 453, 885- 895.
16	李阳, 刘扬, 刘国军, 等. 基于对象位置线索的弱监督图像语义分割方法[J]. 软件学报, 2020, 31 (11): 3640- 3656.
	LI Y , LIU Y , LIU G J , et al. Weakly supervised image semantic segmentation method based on object location cues[J]. Journal of Software, 2020, 31 (11): 3640- 3656.
17	XU Y J , MAO Z D , CHEN Z N , et al. Context propagation embedding network for weakly supervised semantic segmentation[J]. Multimedia Tools and Applications, 2020, 79 (45): 33925- 33942.
18	白雪飞, 李文静, 王文剑. 基于显著性背景引导的弱监督语义分割网络[J]. 模式识别与人工智能, 2021, 34 (9): 824- 835.
	BAI X F , LI W J , WANG W J . Saliency background guidance network for weakly-supervised semantic segmentation[J]. Pattern Recognition and Artificial Intelligence, 2021, 34 (9): 824- 835.
19	TSAI Y H, HUNG W C, SCHULTER S, et al. Learning to adapt structured output space for semantic segmentation[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 7472-7481.
20	ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proc. of the IEEE International Conference on Computer Vision, 2017: 2223-2232.
21	PAN F, SHIN I, RAMEAU F, et al. Unsupervised intra-domain adaptation for semantic segmentation through self-supervision[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 3764-3773.
1	GOODFEllOW I J , POUGET-ABADIE J , MIRZA M , et al. Generative adversarial networks[J]. Advances in Neural Information Processing Systems, 2014, 3, 2672- 2680.
2	SHELHAMER E , LONG J , DARRELL T . Fully convolutional networks for semantic segmentation[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2017, 39 (4): 640- 651. doi: 10.1109/TPAMI.2016.2572683
3	BADRINARAYANAN V , KENDALL A , CIPOLLA R . SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Trans.on Pattern Analysis & Machine Intelligence, 2017, 1- 1.
4	RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]//Proc. of the International Conference on Medical Image Computing and Computer-Assisted Intervention, 2015.
5	CHEN L C , PAPANDREOU G , KOKKINOS I , et al. Semantic image segmentation with deep convolutional nets and fully connected CRFs[J]. Computer Science, 2014, (4): 357- 361.
6	WANG L, LI D, ZHU Y, et al. Dual super resolution learning for semantic segmentation[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
7	ZHONG Z, LIN Z Q, BIDART R, et al. Squeeze-and-attention networks for semantic segmentation[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020.
8	XING F Z, CAMBRIA E, HUANG W B, et al. Weakly supervised semantic segmentation with superpixel embedding[C]//Proc. of the IEEE International Conference on Image Processing, 2016.
9	韩铮, 肖志涛. 基于纹元森林和显著性先验的弱监督图像语义分割方法[J]. 电子与信息学报, 2018, 40 (3): 610- 617.
	HAN Z , XIAO Z T . Weakly supervised semantic segmentation based on semantic texton forest and saliency prior[J]. Journal of Electronics & Information Technology, 2018, 40 (3): 610- 617.
10	SHI Z Y , YANG Y X , HOSPEDALES T M , et al. Weakly-supervised image annotation and segmentation with objects and attributes[J]. IEEE Trans.on Pattern Analysis & Machine Intelligence, 2017, 39 (12): 2525- 2538.
11	YI L , GUO Y Q , KAO Y Y , et al. Image piece learning for weakly supervised semantic segmentation[J]. IEEE Trans.on Systems Man & Cybernetics Systems, 2017, 47 (4): 648- 659.
22	KIM M, BYUN H. Learning texture invariant representation for domain adaptation of semantic segmentation[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 12975-12984.
23	LV F M, LIANG T, CHEN X, et al. Crossdomain semantic segmentation via domain-invariant interactive relation transfer[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 4334-4343.
24	YANG Y C, SOATTO S. FDA: Fourier domain adaptation for semantic segmentation[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 4085-4095.
25	LIU Y, ZHANG W, WANG J. Source-free domain adaptation for semantic segmentation[C]//Proc. of the IEEE/CVF Conferenceon Computer Vision and Pattern Recognition, 2021: 1215-1224.
26	MA H Y, LIN X R, WU Z F, et al. Coarse-to-fine domain adaptive semantic segmentation with photometric alignment and category-center regularization[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 4051-4060.
27	GUO X Q, YANG C, LI B P, et al. Metacor-rection: domain-aware meta loss correction for unsupervised domain adaptation in semantic segmentation[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 3927- 3936.
28	LUO Y W, ZHENG L, GUAN T, et al. Taking a closer look at domain shift: category-level adversaries for semantics consistent domain adaptation[C]//Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 2507- 2516.
29	HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//Proc of the IEEE Conference on Computer Vision and Pattern recognition, 2018: 7132-7141.
30	WANG Y D, ZHANG J, KAN M N, et al. Self-supervised equivariant attention mechanism for weakly supervised semantic segmentation[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 12275-12284.

数据集	尺寸/像素	数量(训练/验证/测试)	类别
WHU2016	256×256	18 944/4 144/9 644	2(建筑物/背景)
WHU2012	256×256	5 040/1 000/1 760	2(建筑物/背景)
Massachusetts	256×256	4 374/377/360	2(建筑物/背景)

网络模型	性能指标
网络模型	Accuracy	Precision	Recall	IoU	F1-Score
Deeplab v3	89.38	77.52	60.75	51.65	68.12
Deeplab Xception	89.66	76.85	63.92	53.60	69.79
PSPNet	90.90	80.33	67.92	58.23	73.60
Unet	91.95	84.05	70.25	61.99	76.53
PCMUnet	92.10	83.12	72.40	63.12	77.39

方法	性能指标
方法	Accuracy	Precision	Recall	IoU	F1-Score
Generator	83.18	76.59	14.33	13.73	24.14
Generator+BuiltNet	84.77	70.44	31.39	27.73	43.43
Generator+BuiltNet+Discriminator	86.75	68.93	52.98	42.77	59.91
Upper Bound	92.10	83.12	72.40	63.12	77.39

方法	性能指标
方法	Accuracy	Precision	Recall	IoU	F1-Score
Baseline (Unet)	92.38	92.64	53.05	50.91	67.47
Generator+BuiltNet	94.92	85.18	79.82	70.08	82.41
Generator+Discriminator	96.08	92.56	80.15	75.30	85.91
Generator+BuiltNet+Discriminator	96.48	90.10	85.77	78.38	87.88
Upper Bound	97.94	94.94	91.05	86.84	92.95

方法	性能指标
方法	Accuracy	Precision	Recall	IoU	F1-score
主干网络	92.38	92.64	53.05	50.91	67.47
AdaptSegNet	92.80	78.08	71.82	59.77	74.82
CLAN	95.03	89.35	75.68	69.42	81.95
WDAN	96.48	90.10	85.77	78.38	87.88