基于空谱信息协同与Gram-Schmidt变换的多源遥感图像融合方法

doi:10.12305/j.issn.1001-506X.2022.07.02

摘要/Abstract

摘要：

多光谱和合成孔径雷达图像的融合可以保留每个数据的优势, 有利于提高土地覆盖分类精度。然而, 当前的一些图像融合方法不能完全利用原始数据的光谱信息与纹理细节。为了克服上述问题, 提出一种基于空谱信息协同和Gram-Schmidt变换的融合方法。在所提方法中, Sentinel-2A图像和高分三号(GaoFen-3, GF-3)图像分别经过不同的预处理操作。由于灰度共生矩阵能有效提取图像的纹理信息, 因此将其应用于Sentinel-2A图像以提取结构特征, 并将空谱信息协同的多光谱图像与GF-3图像通过Gram-Schmidt变换进行融合。实验采用主成分分析法和传统的Gram-Schmidt变换作为比较方法。为了确定融合算法的有效性, 采用5项评价指标(包括平均梯度、空间频率、均值、标准差和相关系数)来衡量融合图像的质量。此外, 由于随机森林具有优秀的训练速度和出色的分类性能, 将其用于土地覆盖分类。随机森林的分类精度、Kappa系数和分类结果图作为融合方法的评价标准。实验结果表明, 与单独使用原始Sentinel-2A相比, 所提方法可以将整体精度提高多达5%, 具有提高遥感卫星图像土地覆盖分类精度的潜力。

关键词: 图像融合, 分类, 多光谱, 遥感

Abstract:

The fusion of multispectral and synthetic aperture radar (SAR) images can retain the advantages of each data and improve the accuracy of land cover classification. However, some current image fusion methods cannot fully utilize the spectral information and texture details of the original data. In order to overcome these problems, a fusion method based on space-spectrum information collaboration and Gram-Schmidt transform is proposed. In the proposed method, Sentinel-2A images and GaoFen-3 (GF-3) images are preprocessed by different methods. Since the gray co-occurrence matrix can effectively extract the texture information of the image, it is applied to the Sentinel-2A image to extract the structural features, and the multispectral image coordinated by the space-spectrum information is fused with GF-3 image by the Gram-Schmidt transform. Principal component analysis (PCA) and the traditional Gram-Schmidt transform are used as the comparison methods in this experiment. In order to determine the effectiveness of the fusion algorithm, this paper uses five evaluation indicators including average gradient, spatial frequency, mean, standard deviation and correlation coefficient to measure the quality of the fusion image. In addition, due to its excellent training speed and excellent classification performance, random forest is used for land cover classification. The classification accuracy of random forest, Kappa coefficient and classification result graph are used as the evaluation criteria of the fusion method. Experimental results show that, compared with the original Sentinel-2A alone, the proposed fusion method can improve the overall accuracy by up to 5%, and has the potential to improve the accuracy of land cover classification in remote sensing satellite images.

Key words: image fusion, classification, multispectral, remote sensing

中图分类号:

TP701

童莹萍, 全英汇, 冯伟, 邢孟道. 基于空谱信息协同与Gram-Schmidt变换的多源遥感图像融合方法[J]. 系统工程与电子技术, 2022, 44(7): 2074-2083.

Yingping TONG, Yinghui QUAN, Wei FENG, Mengdao XING. Multi-source remote sensing image fusion method based on spatial-spectrum information collaboration and Gram-Schmidt transform[J]. Systems Engineering and Electronics, 2022, 44(7): 2074-2083.

图/表 8

表1

图1

图2

表2

表3

图3

表4

表5

参考文献 33

1	SHIRAISHI T , MOTIHKA T , THAPA R B , et al. Comparative assessment of supervised classifiers for land use-land cover classification in a tropical region using time-series PALSAR mosaic data[J]. IEEE Journal of Selected Topics in Applied Earth Observations & Remote Sensing, 2014, 7 (4): 1186- 1199.
2	THOMAS C , RANCHIN T , WALD L , et al. Synthesis of multispectral images to high spatial resolution: a critical review of fusion methods based on remote sensing physics[J]. IEEE Trans.on Geoscience and Remote Sensing, 2008, 46 (5): 1301- 1312. doi: 10.1109/TGRS.2007.912448
3	MING C , CHENG W , LI J . Sparse representation based pansharpening using trained dictionary[J]. IEEE Geoscience and Remote Sensing Letters, 2014, 11 (1): 293- 297. doi: 10.1109/LGRS.2013.2256875
4	WANG Q M , SHI W Z , LI Z B , et al. Fusion of Sentinel-2A images[J]. Remote Sensing of Environment, 2016, 187, 241- 252. doi: 10.1016/j.rse.2016.10.030
5	MA X S , WU P H , WU Y L , et al. A review on recent developments in fully polarimetric SAR image despeckling[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018, 11 (3): 743- 758. doi: 10.1109/JSTARS.2017.2768059
6	张庆君. 高分三号卫星总体设计与关键技术[J]. 测绘学报, 2017, 46 (3): 269- 277.
	ZHANG Q J . System design and key technologies of the GF-3 satellite[J]. Acta Geodaetica et Cartographica Sinica, 2017, 4, 6.
7	ZHU Y H , LIU K , MYINT S W , et al. Integration of GF2 optical, GF3 SAR, and UAV data for estimating aboveground biomass of China's largest artificially planted mangroves[J]. Remote Sensing, 2020, 12 (12): 2039. doi: 10.3390/rs12122039
8	SILVERIRA M , HELENO S . Separation between water and land in SAR images using region-based level sets[J]. IEEE Geoscience & Remote Sensing Letters, 2009, 6 (3): 471- 475.
9	YE Z , PRASAD S , LI W , et al. Classification based on 3-D DWT and decision fusion for hyperspectral image analysis[J]. IEEE Geoscience and Remote Sensing Letters, 2013, 11 (1): 173- 177.
10	VOHRA R , TIWARI K C . Comparative analysis of SVM and ANN classifiers using multilevel fusion of multi-sensor data in urban land classification[J]. Sensing and Imaging, 2020, 21 (1): 17. doi: 10.1007/s11220-020-00280-9
11	TAHA E D , ELBEIH S F . Investigation of fusion of SAR and Landsat data for shoreline super resolution mapping: the northeastern Mediterr anean Sea coast in Egypt[J]. Applied Geomatics, 2010, 2 (4): 177- 186. doi: 10.1007/s12518-010-0033-x
12	BYUN Y , CHOI J , HAN Y . An area-based image fusion scheme for the integration of SAR and optical satellite imagery[J]. Selected Topics in Applied Earth Observations & Remote Sensing, 2013, 6 (5): 2212- 2220.
13	MONTGOMERY J , BRISCO B , CHASMER L , et al. SAR and Lidar temporal data fusion approaches to boreal wetland ecosystem monitoring[J]. Remote Sensing, 2019, 11 (2): 161. doi: 10.3390/rs11020161
14	IERVOLINO P , GUIDA R , RICCIO D , et al. A novel multispectral, panchromatic and SAR data fusion for land classification[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019, 12 (10): 3966- 3979. doi: 10.1109/JSTARS.2019.2945188
15	GAO H , WANG C C , WANG G Y , et al. A crop classification method integrating GF-3 PolSAR and Sentinel-2A optical data in the Dongting Lake Basin[J]. Sensors, 2018, 18 (9): 3139. doi: 10.3390/s18093139
16	李树涛, 李聪妤, 康旭东. 多源遥感图像融合发展现状与未来展望[J]. 遥感学报, 2021, 25 (1): 148- 166.
	LI S T , LI C Y , KANG X D . Development status and future prospects of multi-source remote sensing image fusion[J]. National Remote Sensing Bulletin, 2021, 25 (1): 148- 166.
17	尹雯, 李元祥, 周则明, 等. 基于稀疏表示的遥感图像融合方法[J]. 光学学报, 2013, 33 (4): 267- 274.
	YIN W , LI Y X , ZHOU Z M , et al. Remote sensing image fusion based on sparse representation[J]. Acta Optica Sinica, 2013, 33 (4): 267- 274.
18	王海荣, 郭擎, 李安. 波段自适应细节注入的高分五号与Sentinel-2遥感影像空谱融合[J]. 中国图象图形学报, 2021, 26 (8): 1896- 1909.
	WANG H R , GUO Q , LI A . Spatial-spectral fusion based on band-adaptive detail injection for GF-5 and Sentinel-2 remote sensing images[J]. Journal of Image and Graphics, 2021, 26 (8): 1896- 1909.
19	王志社, 杨风暴, 陈磊, 等. 基于纹理分割和Top-Hat变换的合成孔径雷达与可见光图像增强融合[J]. 光学学报, 2014, 34 (10): 85- 92.
	WANG Z S , YANG F B , CHEN L , et al. SAR and visible im age enhanced fusion based on texture segmentation and Top-Hat transformation[J]. Acta Optica Sinica, 2014, 34 (10): 85- 92.
20	范新南, 张继, 李敏, 等. 基于局部特征差异的异源图像融合算法[J]. 光电子·激光, 2014, 25 (10): 2025- 2032.
	FAN X N , ZHANG J , LI M , et al. A multi-sensor image fusion algorithm based on local feature difference[J]. Journal of Optoelectronics·Laser, 2014, 25 (10): 2025- 2032.
21	RODRIGUEZ-GALIANO V F , CHICA-OLMO M , ABARCA-HERNANDEZ F , et al. Random forest classification of mediterranean land cover using multi-seasonal imagery and multi-seasonal texture[J]. Remote Sensing of Environment, 2012, 121, 93- 107. doi: 10.1016/j.rse.2011.12.003
22	ZHANG R , TANG X M , YOU S C , et al. A novel feature-level fusion framework using optical and SAR remote sensing images for land use/land cover (LULC) classification in cloudy mountainous area[J]. Applied Sciences, 2020, 10 (8): 2928. doi: 10.3390/app10082928
23	FENG W , HUANG W J , REN J C . Class Imbalance ensemble learning based on the Margin Theory[J]. Applied Sciences, 2018, 8 (5): 815. doi: 10.3390/app8050815
24	FENG W , DAUPHIN G , HUANG W J , et al. Dynamic synthetic minority over-sampling technique-based rotation forest for the classification of imbalanced hyperspectral data[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019, 12 (7): 2159- 2169. doi: 10.1109/JSTARS.2019.2922297
25	侯群群, 王飞, 严丽. 基于灰度共生矩阵的彩色遥感图像纹理特征提取[J]. 国土资源遥感, 2013, 25 (4): 26- 32.
	HOU Q Q , WANG F , YAN L . Extraction of color image texture feature based on gray-level co-occurrence matrix[J]. Remote Sensing for Land and Resources, 2013, 25 (4): 26- 32.
26	刘丽, 匡纲要. 图像纹理特征提取方法综述[J]. 中国图象图形学报, 2009, 14 (4): 622- 635.
	LIU L , KUANG G Y . Overview of image textural feature ex traction methods[J]. Journal of Image and Graphics, 2009, 14 (4): 622- 635.
27	李智峰, 朱谷昌, 董泰锋. 基于灰度共生矩阵的图像纹理特征地物分类应用[J]. 地质与勘探, 2011, 47 (3): 456- 461.
	LI Z F , ZHU G C , DONG T F . Application of GLCM-based texture features to remote sensing image classification[J]. Geo-logy and Exploration, 2011, 47 (3): 456- 461.
28	李存军, 刘良云, 王纪华, 等. 两种高保真遥感影像融合方法比较[J]. 中国图象图形学报, 2004, 9 (11): 1376- 1385.
	LI C J , LIU L Y , WANG J H , et al. Comparison of two methods of fusing remote sensing images with fidelity of spectral information[J]. Journal of Image and Graphics, 2004, 9 (11): 1376- 1385.
29	张涛, 刘军, 杨可明, 等. 结合Gram-Schmidt变换的高光谱影像谐波分析融合算法[J]. 测绘学报, 2015, 44 (9): 1042- 1047.
	ZHANG T , LIU J , YANG K M , et al. Fusion algorithm for hyperspectral remote sensing image combined with harmonic analysis and Gram-Schmidt transform[J]. Acta Geodaetica et Cartograpgica Sinica, 2015, 44 (9): 1042- 1047.
30	张小利, 李雄飞, 李军. 融合图像质量评价指标的相关性分析及性能评估[J]. 自动化学报, 2014, 40 (2): 306- 315.
	ZHANG X L , LI X F , LI J . Validation and correlation analysis of metrics for evaluating performance of image fusion[J]. Acta Automatica Sinica, 2014, 40 (2): 306- 315.
31	杨艳春, 李娇, 王阳萍. 图像融合质量评价方法研究综述[J]. 计算机科学与探索, 2018, 12 (7): 1021- 1035.
	YANG Y C , LI J , WANG Y P . Review of image fusion quality evaluation methods[J]. Journal of Frontiers of Computer Science and Technology, 2018, 12 (7): 1021- 1035.
32	FENG W , QUAN Y H , DAUPHIN G , et al. Semi-supervised rotation forest based on ensemble margin theory for the classification of hyperspectral image with limited training data[J]. Information Sciences, 2021, 575, 611- 638. doi: 10.1016/j.ins.2021.06.059
33	FENG W , DAUPHIN G , HUANG W J , et al. New margin-based subsampling iterative technique in modified random forests for classification[J]. Knowledge-Based Systems, 2019, 182, 104845. doi: 10.1016/j.knosys.2019.07.016

符号	含义
S	Sentinel-2A图像中的可用波长数
G	GF-3图像中的可用波长数
N_s	Sentinel-2A图像中的像素总数
N_g	GF-3图像中的像素总数
S	Sentinel-2A图像的相关矩阵
G	GF-3图像的相关矩阵

评价指标	PCA融合	GS融合	M-GS融合
平均梯度	6.75	8.38	9.37
均值	1 455.76	1 356.10	1 765.20
标准差	569.63	515.78	707.60
空间频率	240.62	256.17	312.62
相关系数	0.94	0.95	0.97

评价指标	PCA融合	GS融合	M-GS融合
平均梯度	7.28	9.06	9.69
均值	1 461.96	1 356.30	1 784.20
标准差	500.48	444.31	818.25
空间频率	140.08	156.25	214.35
相关系数	0.94	0.95	0.96

研究区域	Sentinel-2A图像	PCA融合	GS融合	M-GS融合
区域1	83.97	79.58	85.64	86.28
区域2	88.71	86.50	91.78	95.50

研究区域	Sentinel-2A图像	PCA融合	GS融合	M-GS融合
区域1	0.791 0	0.733 4	0.813 6	0.821 1
区域2	0.852 4	0.823 6	0.892 3	0.941 0