基于多任务学习的建筑毁伤评估方法

doi:10.12305/j.issn.1001-506X.2024.10.15

Abstract

Abstract:

Building damage assessment plays an important role in the disaster relief process, influencing the formulation of rescue strategies and optimization of resource allocation. Currently, damage assessment methods based on semantic segmentation face challenges in extracting fine-grained semantic information for damaged buildings. Thus, a multi-task learning based approach for building damage assessment is proposed, dividing the damage assessment into two subtasks as coarse-grained building area extraction and fine-grained damage segmentation. The proposed method utilizes a shared encoder-decoder and context fusion module to achieve coarse-grained extraction of building areas and fine-grained segmentation of building damage. The results of these two tasks are fused using the Hadamard product to obtain the final assessment. Experimental results demonstrate that the proposed multi-task learning based building damage assessment method performs well.

Key words: building damage assessment, deep learning, multi-task learning

CLC Number:

TP391.41

Yibo WANG, Lefei ZHANG, Xinde LI. Multi-task learning based building damage assessment method[J]. Systems Engineering and Electronics, 2024, 46(10): 3375-3382.

Figures/Tables 10

Fig.1

Fig.2

Fig.3

Fig.4

Table 1

Fig.5

Table 2

Fig.6

Fig.7

Table 3

References 31

1	ZHENG S Q, LI X D, ZHU L L. Optimal time-efficient UAV area coverage path planning based on raster map[C]//Proc. of the International Conference on Advanced Robotics and Mechatronics, 2023: 727-732.
2	王建峰, 贾高伟, 郭正, 等. 多无人机协同任务规划方法研究综述[J/OL]. 系统工程与电子技术: 1-18[2024-09-23]. http://kns.cnki.net/kcms/detail/11.2422.TN.20230419.1331.010.html.
	WANG J F, JIA G W, GUO Z, et al. Research status and development of multi-UAV system mission planning[J/OL]. Systems Engineering and Electronics: 1-18[2024-09-23]. http://kns.cnki.net/kcms/detail/11.2422.TN.20230419.1331.010.html.
3	TCHINDA N E , PANOFF K M , KWADJO T D , et al. Semi-supervised image stitching from unstructured camera arrays[J]. Sensors, 2023, 23 (23): 9481- 9501. doi: 10.3390/s23239481
4	杨丹宁. 基于受灾点应急需求分析的地震初期应急物资分配问题研究[D]. 北京: 北京交通大学, 2023.
	YANG D N. A study on the distribution of emergency supplies at the early stage of earthquake based on the analysis of emergency needs at the affected sites[D]. Beijing: Beijing Jiao Tong University, 2023.
5	李阳, 张玉梅, 赵彦东, 等. 基于云贝叶斯网络的毁伤效果评估方法[J]. 火力与指挥控制, 2020, 45 (3): 144- 149.
	LI Y , ZHANG Y M , ZHAO Y D , et al. Method of battle da-mage assessment based on cloudy Bayesian network[J]. Fire Control & Command Control, 2020, 45 (3): 144- 149.
6	LI X D , FIR D K , JEAN D . Multi-source information fusion: progress and future[J]. Chinese Journal of Aeronautics, 2024, 37 (7): 24- 58. doi: 10.1016/j.cja.2023.12.009
7	王广学, 黄晓涛, 周智敏. 基于图像分割的VHF SAR叶簇隐蔽目标差值变化检测[J]. 电子学报, 2020, 38 (9): 1969- 1974.
	WANG G X , HUANG X T , ZHOU Z M . VHF SAR difference change detection of target in foliage based on image segmentation[J]. Acta Electronica Sinica, 2020, 38 (9): 1969- 1974.
8	CHEN J , CHEN X H , CUI X H , et al. Change vector analysis in posterior probability space: a new method for land cover change detection[J]. IEEE Geoscience and Remote Sensing Letters, 2010, 8 (2): 317- 321.
9	MARIUS P , ANDREAS D , TOBIAS U , et al. Automated extraction of annual erosion rates for arctic permafrost coasts using sentinel-1, deep learning, and change vector analysis[J]. Remote Sensing, 2022, 14 (15): 3656- 3681. doi: 10.3390/rs14153656
10	苗添, 曾虹程, 王贺, 等. 基于迭代阈值分割的星载SAR洪水区域快速提取[J]. 系统工程与电子技术, 2022, 44 (9): 2760- 2768. doi: 10.12305/j.issn.1001-506X.2022.09.08
	MIAO T , ZENG H C , WANG H , et al. A fast extraction method of flood areas based on iterative threshold segmentation using spaceborne SAR data[J]. Systems Engineering and Electronics, 2022, 44 (9): 2760- 2768. doi: 10.12305/j.issn.1001-506X.2022.09.08
11	杨青青, 樊桂花. 基于改进模糊综合评判法的建筑物毁伤效果评估[J]. 系统工程与电子技术, 2018, 40 (9): 2026- 2031. doi: 10.3969/j.issn.1001-506X.2018.09.19
	YANG Q Q , FAN G H . Battle damage assessment of building based on improved fuzzy comprehensive evaluation method[J]. Systems Engineering and Electronics, 2018, 40 (9): 2026- 2031. doi: 10.3969/j.issn.1001-506X.2018.09.19
12	RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[C]//Proc. of the Medical Image Computing and Computer-assisted Intervention, 2015: 234-241.
13	XIE E Z , WANG W H , YU Z D , et al. SegFormer: simple and efficient design for semantic segmentation with transfor-mers[J]. Advances in Neural Information Processing Systems, 2021, 34, 12077- 12090.
14	ZHONG A Q , FU Q , HUANG D F , et al. A topology based automatic registration method for infrared and polarized coupled imaging[J]. Applied Sciences, 2022, 12 (24): 12596- 12607. doi: 10.3390/app122412596
15	DENG X , LIU E P , LI S X , et al. Interpretable multi-model image registration network based on disentangled convolutional sparse coding[J]. IEEE Trans.on Image Processing, 2023, 32, 1078- 1091. doi: 10.1109/TIP.2023.3240024
16	徐艺博, 于清华, 王炎娟, 等. 基于多源信息融合的巡飞弹对地目标识别与毁伤评估[J]. 系统仿真学报, 2024, 36 (2): 511- 521.
	XU Y B , YU Q H , WANG Y J , et al. Ground target recognition and damage assessment of patrol missiles based on multi-source information fusion[J]. Journal of System Simulation, 2024, 36 (2): 511- 521.
17	李琳, 孙世岩, 曾雅琴, 等. 基于改进CRITIC-Bayes网络的激光武器毁伤效果评估方法[J]. 兵器装备工程学报, 2023, 44 (7): 109- 115.
	LI L , SUN S Y , ZENG Y Q , et al. Evaluation methods of laser weapon damage effect based on improved CRITIC-Bayes network[J]. Journal of Ordnance Equipment Engineering, 2023, 44 (7): 109- 115.
18	NIA K R, MORI G. Building damage assessment using deep learning and ground-level image data[C]//Proc. of the 14th Conference on Computer and Robot, 2017: 95-102.
19	麻连伟, 宁卫远, 焦利伟, 等. 基于U-Net卷积神经网络的遥感影像变化检测方法研究[J]. 能源与环保, 2022, 44 (11): 102- 106.
	MA L W , NING W Y , JIAO L W , et al. Research on remote sensing image change detection method based on U-Net convolutional neural network[J]. China Energy and Environmental Protection, 2022, 44 (11): 102- 106.
20	FAN Z P , WANG S Q , PU X T , et al. Fusion-former: fusion features across transformer and convolution for building change detection[J]. Electronics, 2023, 12 (23): 4823- 4836. doi: 10.3390/electronics12234823
21	WANG X D , TIAN M L , ZHANG Z J , et al. SDSNet: building extraction in high-resolution remote sensing images using a deep convolutional network with cross-layer feature information interaction filtering[J]. Remote Sensing, 2023, 16 (1): 169- 191. doi: 10.3390/rs16010169
22	王凌霄, 贾婧. 基于改进DeepLabv3plus算法的遥感图像海岛建筑提取方法[J]. 热带地理, 2021, 41 (4): 834- 844.
	WANG L X , JIA J . Extraction method of island buildings in remote sensing images based on improved DeepLabv3plus algorithm[J]. Tropical Geography, 2021, 41 (4): 834- 844.
23	ZHENG Z , ZHONG Y F , WANG J J , et al. Building damage assessment for rapid disaster response with a deep object-based semantic change detection framework: from natural disasters to man-made disasters[J]. Remote Sensing of Environment, 2021, 265, 112636- 112653. doi: 10.1016/j.rse.2021.112636
24	SHENG W J , LI X D . Multi-task learning for gait-based identity recognition and emotion recognition using attention enhanced temporal graph convolutional network[J]. Pattern Recognition, 2021, 114, 107868- 107880. doi: 10.1016/j.patcog.2021.107868
25	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.
26	LIN T Y, DOLLAR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]// Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 2117-2125.
27	MILLETARI F, NAVAB N, AHMADI S. V-net: fully convolutional neural networks for volumetric medical image segmentation[C]//Proc. of the 4th Conference on 3D Vision, 2016: 565-571.
28	GUPTA R, GOODMAN B, PATEL N, et al. Creating xBD: a dataset for assessing building damage from satellite imagery[C]// Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2019: 10-17.
29	LOSHCHILOV I, HUTTER F. Decoupled weight decay regularization[C]// Proc. of the International Conference on Learning Representations, 2018.
30	GUPTA R, HOSFELT R, SAJEEV S, et al. xBD: a dataset for assessing building damage from satellite imagery[EB/OL]. [2024-01-30]. https://arXiv preprint arXiv:1911.09296, 2019.
31	WEBER E, KANE H. Building disaster damage assessment in sate- llite imagery with multi-temporal fusion[EB/OL]. [2024-01-30]. https://arXiv preprint arXiv:2004.05525, 2020.

方法	主干网	F₁^overall	F₁^loc	F₁^dam	F₁^dam_1	F₁^dam_2	F₁^dam_3	F₁^dam_4
xBD-Baseline^[30]	ResNet50	26.5	-	-	66.3	14.4	0.9	46.6
方法a^[31]	ResNet50	74.1	-	-	90.6	49.3	72.2	83.7
方法b^[23]	ResNet50	74.5	85.4	69.9	88.9	51.4	71.6	79.6
MLDA(本文)	ResNet50	76.1	83.1	73.1	93.5	56.1	70.3	83.3

方法	F₁^overall	F₁^loc	F₁^dam	F₁^dam_1	F₁^dam_2	F₁^dam_3	F₁^dam_4
Res-U-Net	72.6	80.1	69.5	87.2	51.6	69.6	80.9
MLDA-	71.5	81.4	67.2	91.5	50.1	63.2	78.1
MLDA	76.1	83.1	73.1	93.5	56.1	70.3	83.3

方法	λ₁	λ₂	F₁^overall	F₁^loc	F₁^dam
MLDA	0.1	2	73.4	80.3	70.5
MLDA	1	2	76.1	83.1	73.1

[1]	Wei CAI, Xin WANG, Xinhao JIANG, Zhiyong YANG, Dong CHEN. Research on few shot target detection method based on decoupling [J]. Systems Engineering and Electronics, 2024, 46(9): 2941-2950.
[2]	Xiaoxuan CHEN, Shuwen XU, Shaohai HU, Xiaole MA. Infrared and visible light image fusion based on convolution and self attention [J]. Systems Engineering and Electronics, 2024, 46(8): 2641-2649.
[3]	Qianglong WANG, Xiaoguang GAO, Bicong WU, Zijian HU, Kaifang WAN. Review of research on restricted Boltzmann machine and its variants [J]. Systems Engineering and Electronics, 2024, 46(7): 2323-2345.
[4]	Xiantao SUN, Wangyang JIANG, Wenjie CHEN, Weihai CHEN, Yali ZHI. Object grasp pose detection based on the region of interest [J]. Systems Engineering and Electronics, 2024, 46(6): 1867-1877.
[5]	Xuemei CHEN, Zhiheng LIU, Suiping ZHOU, Hang YU, Yanming LIU. Road extraction from high-resolution remote sensing images based on HRNet [J]. Systems Engineering and Electronics, 2024, 46(4): 1167-1173.
[6]	Tianwen ZHANG, Xiaoling ZHANG, Zikang SHAO, Tianjiao ZENG. Mask attention interaction for SAR ship instance segmentation [J]. Systems Engineering and Electronics, 2024, 46(3): 831-838.
[7]	Ran JI, Maosen XIAO, Shuo LI, Yu LIU, Zhanyi LUO, Jiawei CHENG. Research on MRTD objective testing method based on machine learning [J]. Systems Engineering and Electronics, 2024, 46(10): 3265-3270.
[8]	Qianqi NIE, Minghui SHA, Yingshen ZHU. Radar signal recognition method based on improved residual neural network [J]. Systems Engineering and Electronics, 2024, 46(10): 3356-3364.
[9]	Duanyang SHI, Qiang LIN, Bing HU, Xiaoshuai DU. Target detection method of primary surveillance radar based on YOLO [J]. Systems Engineering and Electronics, 2024, 46(1): 143-151.
[10]	Meng WANG, Bing ZHU. Application of uncertainty modeling in 2D and 3D object detection [J]. Systems Engineering and Electronics, 2023, 45(8): 2370-2376.
[11]	Kai SHAO, Ziqun DU, Guangyu WANG. CSI feedback method for dynamically adjusting compression rate based on model pruning [J]. Systems Engineering and Electronics, 2023, 45(8): 2615-2622.
[12]	Tianshu CUI, Dong WANG, Zhen HUANG. Automatic modulation classification based on lightweight network for space cognitive communication [J]. Systems Engineering and Electronics, 2023, 45(7): 2220-2226.
[13]	Yu JIANG, Qi YUAN, Zhitao HU, Weiwei WU, Xin GU. Airport arrival and departure delay time prediction based on meteorological factors [J]. Systems Engineering and Electronics, 2023, 45(6): 1722-1731.
[14]	Yang CHEN, Canhui LIAO, Kun ZHANG, Jian LIU, Pengju WANG. A signal modulation indentification algorithm based on self-supervised contrast learning [J]. Systems Engineering and Electronics, 2023, 45(4): 1200-1206.
[15]	Ye ZHANG, Yi HOU, Kewei OUYANG, Shilin ZHOU. Survey of univariate sequence data classification methods [J]. Systems Engineering and Electronics, 2023, 45(2): 313-335.

Multi-task learning based building damage assessment method

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 31

Related Articles 15

Recommended Articles

Metrics

Comments