基于三维注意力与混合卷积的高光谱图像分类

doi:10.12305/j.issn.1001-506X.2023.09.04

Abstract

Abstract:

Aiming at the lack of effective attention in the process of feature extraction in existing hyperspectral image classification models, a classification model based on hybrid convolution and three-dimensional attention mechanism is proposed. The method realizes the extraction of spatial-spectral features of hyperspectral images by tandem three-dimensioal (3D) convolution and two-dimensional (2D) convolution. An attention mechanism in the 3D convolution stage is designed, and the attention mechanism is implemented in the 3D convolution stage to realize the attention and activation of the effective spatial-spectral features of hyperspectral images while the model is extracting the underlying features. Compared with the traditional 3D convolution-based model, the classification model proposed in this paper reduces the complexity of operations, improves the model's ability to suppress interference noise, and enhances the classification effect. Ablation experiments against the method demonstrate the effectiveness of the proposed 3D convolution attention mechanism, and the optimal classification accuracy is achieved in comparison experiments with five other classification models on two publicly available datasets, Indian Pines and Pavia University.

Key words: hyperspectral image classification, three-dimensional attention mechanism, hybrid convolution, spatial-spectral feature extraction

CLC Number:

TP751

Xiaofeng ZHAO, Jiahui NIU, Chuntong LIU, Yuting XIA. Hyperspectral image classification based on hybrid convolution with three-dimensional attention mechanism[J]. Systems Engineering and Electronics, 2023, 45(9): 2673-2680.

Figures/Tables 11

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Table 1

Table 2

Fig.6

Table 3

Fig.7

Fig.8

References 32

1	GHAMISI P , RASTI B , YOKOYA N , et al. Multisource and multitemporal data fusion in remote sensing: a comprehensive review of the state of the art[J]. IEEE Geoscience and Remote Sensing Magazine, 2019, 7 (1): 6- 39. doi: 10.1109/MGRS.2018.2890023
2	ZHU Q Q , DENG W H , ZHENG Z , et al. A spectral-spatial-dependent global learning framework for insufficient and imbalanced hyperspectral image classification[J]. IEEE Trans.on Cybernetics, 2021, 52 (11): 11709- 11723.
3	AHMAD M, SHABBIR S, RAZA R A, et al. Hyperspectral image classification: artifacts of dimension reduction on hybrid CNN[C]//Proc. of the Computer Vision and Pattern Recognition Conference, 2021.
4	张朝阳, 程海峰, 陈朝辉, 等. 高光谱遥感的发展及其对军事装备的威胁[J]. 光电技术应用, 2008, (1): 10- 12. doi: 10.3969/j.issn.1673-1255.2008.01.003
	ZHANG C Y , CHENG H F , CHEN C H , et al. The development of hyperspectral remote sensing and its threatening to military equipments[J]. Electro-optic Technology Application, 2008, (1): 10- 12. doi: 10.3969/j.issn.1673-1255.2008.01.003
5	YANG X G , YU Y . Estimating soil salinity under various moisture conditions: an experimental study[J]. IEEE Trans.on Geoscience and Remote Sensing, 2017, 55 (5): 2525- 2533. doi: 10.1109/TGRS.2016.2646420
6	GOVENDER M , CHETTY K , BULCOCK H . A review of hyperspectral remote sensing and its application in vegetation and water resource studies[J]. Water SA, 2007, 33 (2): 145- 151.
7	MOLERO J M , GARZON E M , GARCIA I , et al. Efficient implementation of hyperspectral anomaly detection techniques on GPUs and multicore processors[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014, 7 (6): 2256- 2266. doi: 10.1109/JSTARS.2014.2328614
8	SUMARSONO A , DU Q . Low-rank subspace representation for estimating the number of signal subspaces in hyperspectral imagery[J]. IEEE Trans.on Geoscience and Remote Sensing, 2015, 53 (11): 6286- 6292. doi: 10.1109/TGRS.2015.2438079
9	CAMPSVALLS G , GOMEZCHOVA L , MUNOZMARI J , et al. Composite kernels for hyperspectral image classification[J]. IEEE Geoscience Remote Sensing Letters, 2006, 105 (1): 23- 33.
10	LI J , HUANG X , GAMBA P , et al. Multiple feature learning for hyperspectral image classification[J]. IEEE Trans.on Geoscience and Remote Sensing, 2015, 53 (3): 1592- 1606. doi: 10.1109/TGRS.2014.2345739
11	CIOTOLA M , VITALE S , MAZZA A , et al. Pansharpening by convolutional neural networks in the full resolution framework[J]. IEEE Trans.on Geoscience and Remote Sensing, 2022, 60, 5408717.
12	GUO M H , XU T X , LIU J J , et al. Attention mechanisms in computer vision: a survey[J]. Computational Visual Media, 2022, 8 (3): 331- 368. doi: 10.1007/s41095-022-0271-y
13	HU W , HUANG Y Y , LI W , et al. Deep convolutional neural networks for hyperspectral image classification[J]. Journal of Sensors, 2015, 2015, 258619.
14	BOULCH A, TROUVE P, KOENIGUER E, et al. Learning speckle suppression in SAR images without ground truth: application to Sentinel-1 time-series[C]//Proc. of the IEEE International Geoscience and Remote Sensing Symposium, 2018: 2366-2369.
15	MOU L , GHAMISI P , ZHU X X . Deep recurrent neural networks for hyperspectral image classification[J]. IEEE Trans.on Geoscience and Remote Sensing, 2017, 55 (7): 3639- 3655. doi: 10.1109/TGRS.2016.2636241
16	MAKANTASIS K, KARANTZALOS K, DOULAMIS A, et al. Deep supervised learning for hyperspectral data classification through convolutional neural networks[C]//Proc. of the IEEE International Geoscience and Remote Sensing Symposium, 2015: 4959-4962.
17	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778
18	SZEGEDY C, LIU W, JIA Y Q, et al. Going deeper with convolutions[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2015.
19	韩彦岭, 崔鹏霞, 杨树瑚, 等. 基于残差网络特征融合的高光谱图像分类[J]. 国土资源遥感, 2021, 33 (2): 11- 19.
	HAN Y L , CUI P X , YANG S H , et al. Classification of hyperspectral image based on feature fusion of residual network[J]. Remote Sensing for Land and Resources, 2021, 33 (2): 11- 19.
20	王溢琴, 董云云, 刘慧玲. 基于GoogLeNet和空间谱变换的高光谱图像超分辨率方法[J]. 光学技术, 2022, 48 (1): 93- 101.
	WANG Y Q , DONG Y Y , LIU H L . Super-resolution method of hyperspectral image based on GoogLeNet and spatial spectrum transformation[J]. Optical Technique, 2022, 48 (1): 93- 101.
21	HAMIDA A B , BENOIT A , LAMBERT P , et al. 3-D deep learning approach for remote sensing image classification[J]. IEEE Trans.on Geoscience and Remote Sensing, 2018, 56 (8): 4420- 4434. doi: 10.1109/TGRS.2018.2818945
22	CHEN Y S , JIANG H L , LI C Y , et al. Deep feature extraction and classification of hyperspectral images based on convolutional neural networks[J]. IEEE Trans.on Geoscience and Remote Sensing, 2016, 54 (10): 6232- 6251. doi: 10.1109/TGRS.2016.2584107
23	ROY S K , KRISHNA G , DUBEY S R , et al. HybridSN: exploring 3D-2D CNN feature hierarchy for hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 17 (2): 277- 281.
24	张祥东, 王腾军, 朱劭俊, 等. 基于扩张卷积注意力神经网络的高光谱图像分类[J]. 光学学报, 2021, 41 (3): 49- 59.
	ZHANG X D , WANG T J , ZHU S J , et al. Hyperspectral image classification based on dilated convolutional attention neural network[J]. Acta Optica Sinica, 2021, 41 (3): 49- 59.
25	DING Y , ZHAO X F , ZHANG Z L , et al. Graph sample and aggregate-attention network for hyperspectral image classification[J]. IEEE Geoscience Remote Sensing Letters, 2021, 19, 5504205.
26	LIU Q C , XIAO L , YANG J X , et al. CNN-enhanced graph convolutional network with pixel and superpixel level feature fusion for hyperspectral image classification[J]. IEEE Trans. on Geoscience and Remote Sensing, 2021, 59 (10): 8657- 8671.
27	HONG D F , HAN Z , YAO J , et al. SpectralFormer: rethinking hyperspectral image classification with Transformers[J]. IEEE Trans.on Geoscience and Remote Sensing, 2022, 60 (5518615)
28	HU J , SHEN L , ALBANIE S , et al. Squeeze-and-excitation networks[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2020, 42 (8): 2011- 2023.
29	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proc. of the European Conference on Computer Vision, 2018.
30	HAMIDA A B , BENOIT A , LAMBERT P , et al. 3-D deep learning approach for remote sensing image classification[J]. IEEE Trans.on Geoscience and Remote Sensing, 2018, 56 (8): 4420- 4434.
31	HE M Y, BO L, CHEN H H. Multi-scale 3D deep convolutional neural network for hyperspectral image classification[C]//Proc. of the IEEE International Conference on Image Processing, 2018: 3904-3908.
32	SUN H , ZHENG X T , LU X Q , et al. Spectral-spatial attention networks for hyperspectral image classification[J]. Remote Sensing, 2019, 11 (8): 3232- 3245.

模型	分类精度
模型	OA	AA	Kappa
2D-CNN-DF	77.33	71.95	68.71
FE-3D-CNN	80.74	85.04	83.00
Muti-3DCNN	79.39	84.29	82.10
3D-2D-ADCNN	97.25	94.89	96.32
SSAN	97.26	94.88	96.31
3DA-FCNN	97.73	95.24	97.41

模型	分类精度
模型	OA	AA	Kappa
2D-CNN-DF	94.15	94.23	91.25
FE-3D-CNN	93.17	91.49	88.7
Muti-3DCNN	97.57	94.69	93.10
3D-2D-ADCNN	98.74	98.93	98.60
SSAN	99.12	98.03	98.11
3DA-FCNN	99.61	98.96	99.48

方法	IP			PU
方法	OA	AA	Kappa	OA	AA	Kappa
FCNN	94.84	90.89	94.09	99.19	98.06	98.93
3DA-FCNN	97.73	95.24	97.41	99.61	98.96	99.48

[1]	Yiqiang TANG, Xiaopeng YANG, Shengming ZHU. Low-orbit satellite channel prediction algorithm based on the hybrid CNN-BiLSTM using attention mechanism [J]. Systems Engineering and Electronics, 2022, 44(12): 3863-3870.
[2]	Shanxue CHEN, Xinxin WANG. Joint sparse representation hyperspectral image classification based on spatial preprocessing [J]. Systems Engineering and Electronics, 2021, 43(9): 2422-2429.
[3]	Shanxue CHEN, Wenwen CHEN. Joint sparse representation of hyperspectral image classification based on secondary dictionary [J]. Systems Engineering and Electronics, 2020, 42(3): 550-556.
[4]	CHEN Shanxue, ZHOU Yanfa, QI Ruolan. Joint sparse representation of hyperspectral image classification based on kernel function [J]. Systems Engineering and Electronics, 2018, 40(3): 692-698.
[5]	CHEN Shan-xue, QU Long-yao, HU Can. Spatial correlation constrained weighted conditional sparse representation for hyperspectral image classification [J]. Systems Engineering and Electronics, 2016, 38(2): 442-449.

Hyperspectral image classification based on hybrid convolution with three-dimensional attention mechanism

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 32

Related Articles 5

Recommended Articles

Metrics

Comments