融合域自适应和弱监督策略的遥感影像建筑物提取算法

doi:10.12305/j.issn.1001-506X.2023.06.04

Abstract

Abstract:

Common building extraction algorithms were mainly implemented in a fully-supervised manner, and the resulting model usually only perform well on training data sets, but operate badly on cross domain. The domain adaptation method based on generative adversarial network (GAN) can enhance the migration ability of the network to a certain extent. However, due to the lack of key information of the target domain, it is difficult to guarantee its ideal performance. A new end-to-end weakly-supervised building extraction network is proposed in this paper, which adopts pixel correlation module (PCM) to improve the performance of the generative network, and then based on this, uses two strategies, domain adaptation and image-level weak label, to optimize the training process, thereby improving the ability of generalization and expansion of the network greatly. Three datasets are used to verify the effectiveness of the proposed method. Extensive experiments show that the proposed method can effectively improve the performance of building extraction. At the same time, ablation studies are conducted to verify the effectiveness of each module in the network.

Key words: semantic segmentation, building extraction, weakly supervision, domain adaptation, generative adversarial network (GAN)

CLC Number:

P237

Shiyan PANG, Jingjing HAO, Lining XING, Xu TAN. Weakly supervised domain adaptation algorithm for building extraction from remote sensing images[J]. Systems Engineering and Electronics, 2023, 45(6): 1616-1623.

Figures/Tables 12

Fig.1

Fig.2

Fig.3

Table 1

Table 2

Table 3

Table 4

Table 5

Table 6

Fig.4

Table 7

Table 8

References 30

12	XU J, SCHWING A G, URTASUN R. Learning to segment under various forms of weak supervision[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2015.
13	ARASLANOV N, ROTH S. Single-stage semantic segmentation from image labels[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 4253- 4262.
14	FAN J S, ZHANG Z X, SONG C, et al. Learning integral objects with intra-class discriminator for weakly-supervised semantic segmentation[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
15	LIU J H , YU C Q , YANG B B , et al. CSENet: cascade semantic erasing network for weakly-supervised semantic segmentation[J]. Neurocomputing, 2021, 453, 885- 895.
16	李阳, 刘扬, 刘国军, 等. 基于对象位置线索的弱监督图像语义分割方法[J]. 软件学报, 2020, 31 (11): 3640- 3656.
	LI Y , LIU Y , LIU G J , et al. Weakly supervised image semantic segmentation method based on object location cues[J]. Journal of Software, 2020, 31 (11): 3640- 3656.
17	XU Y J , MAO Z D , CHEN Z N , et al. Context propagation embedding network for weakly supervised semantic segmentation[J]. Multimedia Tools and Applications, 2020, 79 (45): 33925- 33942.
18	白雪飞, 李文静, 王文剑. 基于显著性背景引导的弱监督语义分割网络[J]. 模式识别与人工智能, 2021, 34 (9): 824- 835.
	BAI X F , LI W J , WANG W J . Saliency background guidance network for weakly-supervised semantic segmentation[J]. Pattern Recognition and Artificial Intelligence, 2021, 34 (9): 824- 835.
19	TSAI Y H, HUNG W C, SCHULTER S, et al. Learning to adapt structured output space for semantic segmentation[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 7472-7481.
20	ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proc. of the IEEE International Conference on Computer Vision, 2017: 2223-2232.
21	PAN F, SHIN I, RAMEAU F, et al. Unsupervised intra-domain adaptation for semantic segmentation through self-supervision[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 3764-3773.
1	GOODFEllOW I J , POUGET-ABADIE J , MIRZA M , et al. Generative adversarial networks[J]. Advances in Neural Information Processing Systems, 2014, 3, 2672- 2680.
2	SHELHAMER E , LONG J , DARRELL T . Fully convolutional networks for semantic segmentation[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2017, 39 (4): 640- 651. doi: 10.1109/TPAMI.2016.2572683
3	BADRINARAYANAN V , KENDALL A , CIPOLLA R . SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Trans.on Pattern Analysis & Machine Intelligence, 2017, 1- 1.
4	RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]//Proc. of the International Conference on Medical Image Computing and Computer-Assisted Intervention, 2015.
5	CHEN L C , PAPANDREOU G , KOKKINOS I , et al. Semantic image segmentation with deep convolutional nets and fully connected CRFs[J]. Computer Science, 2014, (4): 357- 361.
6	WANG L, LI D, ZHU Y, et al. Dual super resolution learning for semantic segmentation[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
7	ZHONG Z, LIN Z Q, BIDART R, et al. Squeeze-and-attention networks for semantic segmentation[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020.
8	XING F Z, CAMBRIA E, HUANG W B, et al. Weakly supervised semantic segmentation with superpixel embedding[C]//Proc. of the IEEE International Conference on Image Processing, 2016.
9	韩铮, 肖志涛. 基于纹元森林和显著性先验的弱监督图像语义分割方法[J]. 电子与信息学报, 2018, 40 (3): 610- 617.
	HAN Z , XIAO Z T . Weakly supervised semantic segmentation based on semantic texton forest and saliency prior[J]. Journal of Electronics & Information Technology, 2018, 40 (3): 610- 617.
10	SHI Z Y , YANG Y X , HOSPEDALES T M , et al. Weakly-supervised image annotation and segmentation with objects and attributes[J]. IEEE Trans.on Pattern Analysis & Machine Intelligence, 2017, 39 (12): 2525- 2538.
11	YI L , GUO Y Q , KAO Y Y , et al. Image piece learning for weakly supervised semantic segmentation[J]. IEEE Trans.on Systems Man & Cybernetics Systems, 2017, 47 (4): 648- 659.
22	KIM M, BYUN H. Learning texture invariant representation for domain adaptation of semantic segmentation[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 12975-12984.
23	LV F M, LIANG T, CHEN X, et al. Crossdomain semantic segmentation via domain-invariant interactive relation transfer[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 4334-4343.
24	YANG Y C, SOATTO S. FDA: Fourier domain adaptation for semantic segmentation[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 4085-4095.
25	LIU Y, ZHANG W, WANG J. Source-free domain adaptation for semantic segmentation[C]//Proc. of the IEEE/CVF Conferenceon Computer Vision and Pattern Recognition, 2021: 1215-1224.
26	MA H Y, LIN X R, WU Z F, et al. Coarse-to-fine domain adaptive semantic segmentation with photometric alignment and category-center regularization[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 4051-4060.
27	GUO X Q, YANG C, LI B P, et al. Metacor-rection: domain-aware meta loss correction for unsupervised domain adaptation in semantic segmentation[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 3927- 3936.
28	LUO Y W, ZHENG L, GUAN T, et al. Taking a closer look at domain shift: category-level adversaries for semantics consistent domain adaptation[C]//Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 2507- 2516.
29	HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//Proc of the IEEE Conference on Computer Vision and Pattern recognition, 2018: 7132-7141.
30	WANG Y D, ZHANG J, KAN M N, et al. Self-supervised equivariant attention mechanism for weakly supervised semantic segmentation[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 12275-12284.

数据集	尺寸/像素	数量(训练/验证/测试)	类别
WHU2016	256×256	18 944/4 144/9 644	2(建筑物/背景)
WHU2012	256×256	5 040/1 000/1 760	2(建筑物/背景)
Massachusetts	256×256	4 374/377/360	2(建筑物/背景)

网络模型	性能指标
网络模型	Accuracy	Precision	Recall	IoU	F1-Score
Deeplab v3	89.38	77.52	60.75	51.65	68.12
Deeplab Xception	89.66	76.85	63.92	53.60	69.79
PSPNet	90.90	80.33	67.92	58.23	73.60
Unet	91.95	84.05	70.25	61.99	76.53
PCMUnet	92.10	83.12	72.40	63.12	77.39

方法	性能指标
方法	Accuracy	Precision	Recall	IoU	F1-Score
Generator	83.18	76.59	14.33	13.73	24.14
Generator+BuiltNet	84.77	70.44	31.39	27.73	43.43
Generator+BuiltNet+Discriminator	86.75	68.93	52.98	42.77	59.91
Upper Bound	92.10	83.12	72.40	63.12	77.39

方法	性能指标
方法	Accuracy	Precision	Recall	IoU	F1-Score
Baseline (Unet)	92.38	92.64	53.05	50.91	67.47
Generator+BuiltNet	94.92	85.18	79.82	70.08	82.41
Generator+Discriminator	96.08	92.56	80.15	75.30	85.91
Generator+BuiltNet+Discriminator	96.48	90.10	85.77	78.38	87.88
Upper Bound	97.94	94.94	91.05	86.84	92.95

方法	性能指标
方法	Accuracy	Precision	Recall	IoU	F1-score
主干网络	92.38	92.64	53.05	50.91	67.47
AdaptSegNet	92.80	78.08	71.82	59.77	74.82
CLAN	95.03	89.35	75.68	69.42	81.95
WDAN	96.48	90.10	85.77	78.38	87.88

Weakly supervised domain adaptation algorithm for building extraction from remote sensing images

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 12

References 30

Related Articles 5

Recommended Articles

Metrics

Comments

[1]	Li CHEN, Zihan FANG, Liquan MEI. DSS signal generation algorithm based on GAN [J]. Systems Engineering and Electronics, 2023, 45(5): 1544-1552.
[2]	Rongxiang LIU, Lin WU, Zhige XIE, Honglin LIU. Auxiliary situation analysis for air defense system based on generative adversarial network [J]. Systems Engineering and Electronics, 2022, 44(8): 2522-2529.
[3]	Dong CHEN, Yanwei JU. Ship object detection SAR images based on semantic segmentation [J]. Systems Engineering and Electronics, 2022, 44(4): 1195-1201.
[4]	Kai SHAO, Miaomiao ZHU, Guangyu WANG. Modulation recognition method based on generative adversarial andconvolutional neural network [J]. Systems Engineering and Electronics, 2022, 44(3): 1036-1043.
[5]	Zhiying LIU, Chunsi XIE, Jinjun LI, Yu SANG. Smoke region segmentation recognition algorithm based on improved Deeplabv3+ [J]. Systems Engineering and Electronics, 2021, 43(2): 328-335.

方法	性能指标
方法	Accuracy	Precision	Recall	IoU	F1-score
Baseline(Unet)	93.65	75.68	62.93	52.35	68.72
WDAN	96.39	86.30	80.16	71.11	83.11

方法	性能指标
方法	Accuracy	Precision	Recall	IoU	F1-score
Baseline(Unet)	90.84	62.25	44.24	34.88	51.72
WDAN	92.09	68.76	52.45	42.36	59.51