二阶逐层特征融合网络的图像超分辨重建

doi:10.12305/j.issn.1001-506X.2024.02.03

Abstract

Abstract:

Some super-resolution networks ignore the reuse of the features of different levels, and there is no fusing of the features. In order to solve those problems, a second-order progressive feature fusion super-resolution network with strong feature reuse and fuse ability is constructed to realize the reconstructed image with high resolution and high fidelity. The core of the network is progressive feature fusion block. Progressive feature fusion block enhances the reuse of features through feature fusion operation. In addition, a second-order feature fusion mechanism is proposed, which adopts progressive feature fusion method for feature fusion at the local and global levels of the network. The experimental results show that the reconstructed image of the network is clearer than that of other networks on line and contour, and better results are obtained in peak signal to noise ratio (SNR) and structural similarity. For example, when the scaling factor is 2, the peak SNR/structure similarity on each test set is 38.20 dB/0.961 2, 33.81 dB/0.919 5, 32.28 dB/0.901 0, 32.65 dB/0.932 4, and 39.11 dB/0.977 9 respectively, which proves that the proposed model acheives improvent compared to other models. The advantages of the second-order progressive feature fusion super-resolution network is proven from the objective standard and subjective point of view.

Key words: super-resolution reconstruction, convolutional neural network (CNN), feature fusion, second-order feature fusion mechanism

CLC Number:

TN911.73

Lei YU, Qiuyue DENG, Liying ZHENG, Haoyu WU. Second-order progressive feature fusion network for image super-resolution reconstruction[J]. Systems Engineering and Electronics, 2024, 46(2): 391-400.

Figures/Tables 19

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Table 1

Table 2

Fig.6

Table 3

Table 4

Table 5

Table 6

Table 7

Fig.7

Fig.8

Fig.9

Fig.10

Fig.11

Fig.12

References 25

1	LIU A R , LIU Y H , GU J J , et al. Blind image super-resolution: a survey and beyond[J]. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2022, 45 (5): 5461- 5480.
2	TIAN C W, ZHANG X Y, LIN J C W, et al. Generative adversarial networks for image super-resolution: a survey[EB/OL]. [2023-04-09]. https://arxiv.org/abs/2204.13620.
3	毕笃彦, 王世平, 刘坤, 等. 基于并行映射卷积网络的超分辨率重建算法[J]. 系统工程与电子技术, 2018, 40 (8): 1873- 1880.
	BI D Y , WANG S P , LIU K , et al. Super-resolution algorithm based on parallel mapping convolution network[J]. Systems Engi- neering and Electronics, 2018, 40 (8): 1873- 1880.
4	TAO H J , LU X B . Contour-based smoky vehicle detection from surveillance video for alarm systems[J]. Signal, Image and Video Processing, 2019, 13 (2): 217- 225. doi: 10.1007/s11760-018-1348-z
5	王钢, 周若飞, 邹昳琨. 基于压缩感知理论的图像优化技术[J]. 电子与信息学报, 2020, 42 (1): 222- 233.
	WANG G , ZHOU R F , ZOU Y K . Research on image optimization technology based on compressed sensing[J]. Journal of Electronics &Information Technology, 2020, 42 (1): 222- 233.
6	陈书贞, 曹世鹏, 崔美玥, 等. 基于深度多级小波变换的图像盲去模糊算法[J]. 电子与信息学报, 2021, 43 (1): 154- 161.
	CHEN S Z , CAO S P , CUI M Y , et al. Image blind deblurring algorithm based on deep multi-level wavelet transform[J]. Journal of Electronics &Information Technology, 2021, 43 (1): 154- 161.
7	CHEN H G , HE X H , QING L B , et al. Real-world single image super resolution: a brief review[J]. Information Fusion, 2022, 79, 124- 145. doi: 10.1016/j.inffus.2021.09.005
8	BASHIR S M A , WANG Y , KHAN M , et al. A comprehensive review of deep learning based single image super-resolution[J]. PeerJ Computer Science, 2021, 7, e621.
9	ZHU X B , LI Z Z , LOU J G , et al. Video super-resolution based on a spatio-temporal matching network[J]. Pattern Re-cognition, 2021, 110 (2): 107619.
10	DONG C, LOY C C, HE K M, et al. Learning a deep convolutional network for image super-resolution[C]//Proc. of the 13th European Conference on Computer Vision,, 2014: 184-199.
11	DONG C, LOY C C, TANG X O. Accelerating the super-resolution convolutional neural network[C]//Proc. of the 14th European Conference on Computer Vision, 2016: 391-407.
12	SHI W Z, CABALLERO J, HUSZÁR F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 1874-1883.
13	KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 1646-1654.
14	AHN N, KANG B, SOHN K A. Fast, accurate, and lightweight super-resolution with cascading residual network[C]//Proc. of the 15th European Conference on Computer Vision, 2018: 256-272.
15	TAI Y, YANG J, LIU X M. Image super-resolution via deep recursive residual network[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 2790-2798.
16	TAI Y, YANG J, LIU X M, et al. MemNet: a persistent memory network for image restoration[C]//Proc. of the IEEE International Conference on Computer Vision, 2017: 4549-4557.
17	程德强, 郭昕, 陈亮亮, 等. 多通道递归残差网络的图像超分辨率重建[J]. 中国图象图形学报, 2021, 26 (3): 605- 618.
	CHENG D Q , GUO X , CHEN L L , et al. Image super-resolution reconstruction from multi-channel recursive residual network[J]. Journal of Image and Graphics, 2021, 26 (3): 605- 618.
18	QIN J H , HUANG Y J , WEN W S . Multi-scale feature fusion residual network for single image super-resolution[J]. Neurocomputing, 2020, 379, 334- 342.
19	LI J C, FANG F M, MEI K F, et al. Multi-scale residual network for image super-resolution[C]//Proc. of the 15th European Conference on Computer Vision, 2018: 517-532.
20	MUQEET A , IQBAL M T B , BAE S H . Hybrid residual attention network for single image super resolution[J]. IEEE Access, 2019, 137020- 137029.
21	KEYS R . Cubic convolution interpolation for digital image processing[J]. IEEE Trans. on Acoustics, Speech, and Signal Processing, 1981, 29 (6): 1153- 1160.
22	KIM J, LEE J K, LEE K M. Deeply-recursive convolutional network for image super-resolution[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 1637-1645.
23	LAI W S, HUANG J B, AHUJA N, et al. Deep Laplacian pyramid networks for fast and accurate super-resolution[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 5838-5843.
24	HUI Z, WANG X M, GAO X B. Fast and accurate single image super-resolution via information distillation network[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018: 723-731.
25	ZHANG K, ZUO W M, ZHANG L. Learning a single convolutional super-resolution network for multiple degradations[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018: 3262-3271.

特征融合方法	PSNR/SSIM
特征融合方法	Set5	Set14	BSD100	Urban100	Manga109
HFF	38.17/0.961 1	33.78/0.919 2	32.25/0.900 5	32.48/0.931 0	38.95/0.977 8
BFF	38.16/0.961 0	33.78/0.918 7	32.25/0.900 4	32.47/0.930 9	38.96/0.977 8
IFF	38.16/0.961 0	33.81/0.918 9	32.24/0.900 2	32.49/0.931 1	38.95/0.977 7
PFF	38.17/0.961 1	33.78/0.919 3	32.26/0.900 5	32.52/0.931 4	39.07/0.977 8

模块	PSNR/SSIM
模块	Set5	Set14	BSD100	Urban100	Manga109
BRB	38.14/0.960 9	33.75/0.918 9	32.24/0.900 4	32.39/0.930 2	38.94/0.977 7
PFFB	38.17/0.961 1	33.78/0.919 3	32.26/0.900 5	32.52/0.931 4	39.07/0.977 8

二阶特征融合机制	PSNR/SSIM
二阶特征融合机制	Set5	Set14	BSD100	Urban100	Manga109
无	38.16/0.960 9	33.77/0.918 9	32.25/0.900 5	32.47/0.930 6	39.03/0.977 7
有	38.17/0.961 1	33.78/0.919 3	32.26/0.900 5	32.52/0.931 4	39.07/0.977 8

模块数量	PSNR/SSIM
模块数量	Set5	Set14	BSD100	Urban100	Manga109
n=6	38.17/0.961 1	33.78/0.919 3	32.26/0.900 5	32.52/0.931 4	39.07/0.977 8
n=8	38.20↑/0.961 2↑	33.81↑/0.919 5↑	32.28↑/0.901 0↑	32.65↑/0.932 4↑	39.11↑/0.977 9↑
n=10	38.21↑/0.961 2	33.81/0.919 2↓	32.29↑/0.900 9↓	32.66↑/0.932 5↑	39.08↓/0.977 9

模型	PSNR/SSIM
模型	Set5	Set14	BSD100	Urban100	Manga109
Bicubic	33.66/0.929 9	30.24/0.868 8	29.56/0.843 1	26.88/0.840 3	30.80/0.933 9
SRCNN	36.66/0.954 2	32.45/0.906 7	31.36/0.887 9	29.50/0.894 6	35.60/0.966 3
FSRCNN	37.00/0.955 8	32.63/0.908 8	31.53/0.892 0	29.88/0.902 0	36.67/0.971 0
VDSR	37.53/0.958 7	33.03/0.912 4	31.90/0.896 0	30.76/0.914 0	37.22/0.975 0
DRCN	37.63/0.958 8	33.04/0.911 8	31.85/0.894 2	30.75/0.913 3	37.55/0.973 2
LapSRN	37.52/0.959 1	32.99/0.912 4	31.80/0.895 2	30.41/0.910 3	37.27/0.974 0
IDN	37.83/0.960 0	33.30/0.914 8	32.08/0.898 5	31.27/0.919 6	38.01/0.974 9
CARN	37.76/0.959 0	33.52/0.916 6	32.09/0.897 8	31.92/0.925 6	38.36/0.976 5
MCSR	38.03/0.960 0	33.58/0.917 0	32.18/0.899 0	31.94/0.926 0	-/-
SPFFSR	38.20/0.961 2	33.81/0.919 5	32.28/0.901 0	32.65/0.932 4	39.11/0.977 9

Second-order progressive feature fusion network for image super-resolution reconstruction

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 19

References 25

Related Articles 15

Recommended Articles

Metrics

Comments

[1]	Jiayi LUO, Xuanpeng LI, Jianghao LI, Qifan XUE, Feng YANG, Weigong ZHANG. Radiation signal sorting method based on PDW multi-feature fusion [J]. Systems Engineering and Electronics, 2024, 46(1): 80-87.
[2]	Haijun LI, Fancheng KONG, Yun LIN. Infrared ship detection algorithm based on improved YOLOv5s [J]. Systems Engineering and Electronics, 2023, 45(8): 2415-2422.
[3]	Yanyan HUANG, Shaoyan GAI, Feipeng DA. Image matching algorithm based on attention mechanism of three branch spatial transformation [J]. Systems Engineering and Electronics, 2023, 45(11): 3363-3373.
[4]	Caiyun WANG, Yida WU, Jianing WANG, Lu MA, Huanyue ZHAO. SAR image target recognition based on combinatorial optimization convolutional neural network [J]. Systems Engineering and Electronics, 2022, 44(8): 2483-2487.
[5]	Xiaofeng ZHAO, Fei WU, Yebin XU, Jiahui NIU, Wei CAI, Zhili ZHANG. Evaluation method of infrared camouflage effect based on background restoration [J]. Systems Engineering and Electronics, 2022, 44(8): 2554-2561.
[6]	Dong CHEN, Yanwei JU. Ship object detection SAR images based on semantic segmentation [J]. Systems Engineering and Electronics, 2022, 44(4): 1195-1201.
[7]	Jingming SUN, Shengkang YU, Jun SUN. Pose sensitivity analysis of HRRP recognition based on deep learning [J]. Systems Engineering and Electronics, 2022, 44(3): 802-807.
[8]	Hengyan LIU, Limin ZHANG, Wenjun YAN, Zhaogen ZHONG, Qing LING, Xiaojun LIANG. LDPC decoding based on WBP-CNN algorithm [J]. Systems Engineering and Electronics, 2022, 44(3): 1030-1035.
[9]	Kai SHAO, Miaomiao ZHU, Guangyu WANG. Modulation recognition method based on generative adversarial andconvolutional neural network [J]. Systems Engineering and Electronics, 2022, 44(3): 1036-1043.
[10]	Xi ZHANG, Zhengmeng JIN, Yaqin JIANG. Total variation algorithm with depth image priors for image colorization [J]. Systems Engineering and Electronics, 2022, 44(2): 385-393.
[11]	Qinzhe LYU, Yinghui QUAN, Minghui SHA, Shuxian DONG, Mengdao XING. Ensemble deep learning-based intelligent classification of active jamming [J]. Systems Engineering and Electronics, 2022, 44(12): 3595-3602.
[12]	Yonggang LI, Weigang ZHU, Qiongnan HUANG, Yuntao LI, Yonghua HE. Near-shore ship target detection with SAR images in complex background [J]. Systems Engineering and Electronics, 2022, 44(10): 3096-3103.
[13]	Bo DAN, Zhequan FU, Shan GAO, Tao JIAN. Full-polarization high resolution range profile recognition technology for sea surface target based on convolutional neural network [J]. Systems Engineering and Electronics, 2022, 44(1): 108-116.
[14]	Ziyan LIU, Shanshan MA, Jing LIANG, Mingcheng ZHU, Lei YUAN. Attention mechanism based CNN channel estimation algorithm in millimeter-wave massive MIMO system [J]. Systems Engineering and Electronics, 2022, 44(1): 307-312.
[15]	Bangyan CUI, Runlan TIAN, Dongfeng WANG, Gang CUI, Jingyuan SHI. Radar emitter identification based on attention mechanism and improved CLDNN [J]. Systems Engineering and Electronics, 2021, 43(5): 1224-1231.

模型	PSNR/SSIM
模型	Set5	Set14	BSD100	Urban100	Manga109
Bicubic	30.39/0.868 2	27.55/0.774 2	27.21/0.738 5	24.46/0.734 9	26.95/0.855 6
SRCNN	32.75/0.909 0	29.30/0.821 5	28.41/0.786 3	26.24/0.798 9	30.48/0.911 7
FSRCNN	33.18/0.914 0	29.37/0.824 0	28.53/0.791 0	26.43/0.808 0	31.10/0.921 0
VDSR	33.66/0.921 3	29.77/0.831 4	28.82/0.797 6	27.14/0.827 9	32.01/0.934 0
DRCN	33.82/0.922 6	29.76/0.831 1	28.80/0.796 3	27.15/0.827 6	32.24/0.934 3
LapSRN	33.81/0.922 0	29.79/0.832 5	28.82/0.798 0	27.07/0.827 5	32.21/0.935 0
IDN	34.11/0.925 3	29.99/0.835 4	28.95/0.801 3	27.42/0.835 9	32.71/0.938 1
CARN	34.29/0.925 5	30.29/0.840 7	29.06/0.803 4	28.06/0.849 3	33.50/0.944 0
MCSR	34.44/0.926 0	30.37/0.842 0	29.11/0.805 0	28.10/0.851 0	-/-
SPFFSR	34.65/0.928 7	30.55/0.845 6	29.22/0.808 0	28.63/0.860 4	34.13/0.947 8

模型	PSNR/SSIM
模型	Set5	Set14	BSD100	Urban100	Manga109
Bicubic	28.42/0.810 4	26.00/0.702 7	25.96/0.667 5	23.14/0.657 7	24.89/0.786 6
SRCNN	30.48/0.862 6	27.50/0.751 3	26.90/0.710 1	24.52/0.722 1	27.58/0.855 5
FSRCNN	30.72/0.866 0	27.61/0.755 0	26.98/0.715 0	24.62/0.728 0	27.90/0.861 0
VDSR	31.35/0.883 8	28.01/0.767 4	27.29/0.725 1	25.18/0.752 4	28.83/0.887 0
DRCN	31.53/0.885 4	28.02/0.767 0	27.23/0.723 3	25.14/0.751 0	28.93/0.885 4
LapSRN	31.54/0.885 2	28.09/0.770 0	27.32/0.727 5	25.21/0.756 2	29.09/0.890 0
IDN	31.82/0.890 3	28.25/0.773 0	27.41/0.729 7	25.41/0.763 2	29.41/0.894 2
CARN	32.13/0.893 7	28.60/0.780 6	27.58/0.734 9	26.07/0.783 7	30.47/0.908 4
MCSR	32.19/0.894 0	28.63/0.782 0	27.58/0.736 0	26.04/0.783 0	-/-
SPFFSR	32.44/0.898 2	28.79/0.786 1	27.70/0.739 7	26.49/0.796 6	31.14/0.915 5