基于稀疏性深度学习的航拍图像超分辨重构

doi:10.12305/j.issn.1001-506X.2021.08.04

Abstract

Abstract:

In order to reduce the cost of unmanned aerial vehicle (UAV) hardware upgrades, super-resolution (SR) based on deep learning of aerial images is studied. SR based on sparse convolutional neural network (SRSCNN) is proposed to compress network structure and reduce the training time by selectively screening the weights of the neural network connections. The experimental results show that the method can effectively shorten the network learning time required under conditions of superiority of the reconstruction effect and computing time. Meanwhile, a saliency-map-based image quality assessment method is designed, which is more suitable for the follow-up processing of aerial image.

Key words: image super-resolution (SR), deep learning, convolutional neural network, aerial image, image quality assessment

CLC Number:

TN911.73

Caiyun WANG, Yangyu LI, Xiaofei LI, Jianing WANG, Wenyi WEI. Aerial image super-resolution restruction based on sparsity and deep learning[J]. Systems Engineering and Electronics, 2021, 43(8): 2045-2050.

Figures/Tables 11

Fig.1

Fig.2

Fig.3

Fig.4

Table 1

Fig.5

Fig.6

Table 2

Table 3

Table 4

Table 5

References 21

1	郭萌. 基于改进POCS的红外弱小目标超分辨率复原算法研究[D]. 吉林: 吉林大学, 2016.
	GUO M. Super-resolution restoration algorithm of small infrared targets based on improved POCS[D]. Jilin: Jilin University, 2016.
2	DAI S S, XIANG H Y, DU Z H, et al. Adaptive regularization of infrared image super-resolution reconstruction[C]//Proc. of the 5th International Conference on Computing, Communications and Networking Technologies, 2014.
3	CASTRO E B , NAKANO M , PEREZ G S , et al. Improvement of image super-resolution algorithms using iterative back projection[J]. IEEE Latin America Transactions, 2017, 15 (11): 2214- 2219. doi: 10.1109/TLA.2017.8070429
4	黎海雪, 林海涛, 姜栋瀚. 基于马尔可夫随机场的图像超分辨技术研究综述[J]. 通信技术, 2018, 51 (10): 2356- 2364. doi: 10.3969/j.issn.1002-0802.2018.10.015
	LI H X , LIN H T , JIANG D H . Research review of image superresolution based on Markov random field[J]. Communications Technology, 2018, 51 (10): 2356- 2364. doi: 10.3969/j.issn.1002-0802.2018.10.015
5	熊亚辉, 陈东方, 王晓峰. 基于多尺度反向投影的图像超分辨率重建算法[J]. 计算机工程, 2020, 46 (7): 251- 259.
	XIONG Y H , CHEN D F , WANG X F . Super-resolution image reconstruction algorithm based on multi-scale back projection[J]. Computer Engineering, 2020, 46 (7): 251- 259.
6	FREEMAN W T , JONES T R , PASZTOR E C . Example-based super-resolution[J]. IEEE Computer Graphics and Applications, 2002, (2): 56- 65.
7	DANG C , AGHAGOLZADEH M , RADHA H . Image super-resolution via local self-learning manifold approximation[J]. IEEE Signal Processing Letters, 2014, 21 (10): 1245- 1249. doi: 10.1109/LSP.2014.2332118
8	杨学峰, 程耀瑜, 王高. 基于小波域压缩感知的遥感图像超分辨算法[J]. 计算机应用, 2017, 37 (5): 1430- 1433.
	YANG X F , CHENG Y Y , WANG G . A super-resolution algorithm for remote sensing images based on wavelet compressed sensing[J]. Journal of Computer Applications, 2017, 37 (5): 1430- 1433.
9	王彩云, 胡允侃, 李晓飞, 等. 基于卷积稀疏编码与多分类器融合的雷达HRRP目标识别方法[J]. 系统工程与电子技术, 2018, 40 (11): 2433- 2437. doi: 10.3969/j.issn.1001-506X.2018.11.07
	WANG C Y , HU Y K , LI X F , et al. Radar HRRP target recognition based on convolutional sparse coding and multi-classifier fusion[J]. Systems Engineering and Electronics, 2018, 40 (11): 2433- 2437. doi: 10.3969/j.issn.1001-506X.2018.11.07
10	张万绪, 史剑雄, 陈晓璇, 等. 基于稀疏表示与引导滤波的图像超分辨率重建[J]. 计算机工程, 2018, 44 (9): 212- 217.
	ZHANG W X , SHI J X , CHEN X X , et al. Image super-resolution reconstruction based on sparse representation and guided filtering[J]. Computer Engineering, 2018, 44 (9): 212- 217.
11	TIMOFTE R, DE S V, VAN G L. A+: adjusted anchored neighborhood regression for fast super-resolution[C]//Proc. of the Asian Conference on Computer Vision, 2014: 111-126.
12	YANG C Y, YANG M H. Fast direct super-resolution by simple functions[C]//Proc. of the IEEE International Conference on Computer Vision, 2014: 561-568.
13	DONG C , LOY C C , HE K , et al. Image super-resolution using deep convolutional networks[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2016, 38 (2): 295- 307. doi: 10.1109/TPAMI.2015.2439281
14	吴磊, 吕国强, 薛治天, 等. 基于多尺度递归网络的图像超分辨率重建[J]. 光学学报, 2019, 39 (6): 90- 97.
	WU L , LYU G Q , XUE Z T , et al. Super-resolution reconstruction of images based on multi-scale recursive network[J]. Acta Optica Sinica, 2019, 39 (6): 90- 97.
15	WANG Y, ZHENG J C. Real-time face detection based on YOLO[C]//Proc. of the IEEE 1st International Conference on Knowledge Innovation and Invention, 2018: 221-224.
16	刘帆, 刘鹏远, 李兵, 等. TensorFlow平台下的视频目标跟踪深度学习模型设计[J]. 激光与光电子学进展, 2017, 54 (9): 277- 285.
	LIU F , LIU P Y , LI B , et al. Design of video target tracking depth learning model under TensorFlow platform[J]. Laser & Optoelectronics Progress, 2017, 54 (9): 277- 285.
17	朱新山, 姚思如, 孙彪, 等. 图像质量评价: 融合视觉特性与结构相似性指标[J]. 哈尔滨工业大学学报, 2018, 50 (5): 121- 128.
	ZHU X S , YAO S R , SUN B , et al. Image quality evaluation: fusion of visual characteristics and structural similarity indicators[J]. Journal of Harbin Institute of Technology, 2018, 50 (5): 121- 128.
18	PREEDANAN W, KONDO T, BUNNUN P, et al. Image quality assessment for medical images based on gradient information[C]//Proc. of the 5th International Conference on Business and Industrial Research, 2018: 189-194.
19	XUE W , ZHANG L , MOU X , et al. Gradient magnitude similarity deviation: a highly efficient perceptual image quality index[J]. IEEE Trans.on Image Processing, 2014, 23 (2): 684- 695. doi: 10.1109/TIP.2013.2293423
20	DONG C, LOY C C, TANG X. Accelerating the super-resolution convolutional neural network[C]//Proc. of the European Conference on Computer Vision, 2016: 391-407.
21	LIU Z, LI J G, SHEN Z Q, et al. Learning efficient convolutional networks through network slimming[C]//Proc. of IEEE International Conference on Computer Vision, 2017: 2755-2762.

网络层	卷积核尺寸	卷积核个数	步长	填充
卷积层1	5×5×1	56	1	2
卷积层2	3×3×12	12	1	0
反卷积层	9×9×56	1	3	4

算法	图像
算法	lena	baboon	comic	flower	face
Bicubic	31.705	23.059	23.122	27.296	32.852
FSRCNN	32.516	23.370	23.816	28.292	33.224
SRSCNN	32.722	23.405	23.881	28.354	33.278

算法	图像
算法	lena	baboon	comic	flower	face
Bicubic	0.860 2	0.539 9	0.704 8	0.805 6	0.799 6
FSRCNN	0.860 8	0.592 7	0.748 4	0.831 6	0.814 3
SRSCNN	0.874 9	0.597 1	0.754 5	0.835 4	0.817 7

算法	图像
算法	lena	baboon	comic	flower	face
Bicubic	0.413 3	0.434 6	0.388 9	0.408 9	0.427 1
FSRCNN	0.414 9	0.437 0	0.394 1	0.410 4	0.431 9
SRSCNN	0.415 5	0.436 9	0.394 4	0.410 6	0.432 3

算法	图像
算法	lena	baboon	comic	flower	face
Bicubic	0.006 3	0.006 0	0.004 3	0.005 4	0.004 1
FSRCNN	0.171 9	0.182 1	0.064 0	0.118 5	0.053 4
SRSCNN	0.168 1	0.159 4	0.066 9	0.120 2	0.057 5

Aerial image super-resolution restruction based on sparsity and deep learning

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 21

Related Articles 15

Recommended Articles

Metrics

Comments

[1]	Xiao HAN, Shiwen CHEN, Meng CHEN, Jincheng YANG. Open-set recognition of LPI radar signal based on reciprocal point learning [J]. Systems Engineering and Electronics, 2022, 44(9): 2752-2759.
[2]	Caiyun WANG, Yida WU, Jianing WANG, Lu MA, Huanyue ZHAO. SAR image target recognition based on combinatorial optimization convolutional neural network [J]. Systems Engineering and Electronics, 2022, 44(8): 2483-2487.
[3]	Limin ZHANG, Kaiwen TAN, Wenjun YAN, Yuyuan ZHANG. Radar emitter recognition based on multi-level jumper residual network [J]. Systems Engineering and Electronics, 2022, 44(7): 2148-2156.
[4]	Guodong JIN, Yuanliang XUE, Lining TAN, Jiankun XU. Advances in object tracking algorithm based on siamese network [J]. Systems Engineering and Electronics, 2022, 44(6): 1805-1822.
[5]	Juan WEI, Huangwei YANG, Fangli NING. Acoustic scene classification based on joint optimization of NMF and CNN [J]. Systems Engineering and Electronics, 2022, 44(5): 1433-1438.
[6]	Xiaofeng ZHAO, Yebin XU, Fei WU, Jiahui NIU, Wei CAI, Zhili ZHANG. Ground infrared target detection method based on global sensing mechanism [J]. Systems Engineering and Electronics, 2022, 44(5): 1461-1467.
[7]	Hong ZOU, Chenyang BAI, Peng HE, Yaping CUI, Ruyan WANG, Dapeng WU. Edge service placement strategy based on distributed deep learning [J]. Systems Engineering and Electronics, 2022, 44(5): 1728-1737.
[8]	Dong CHEN, Yanwei JU. Ship object detection SAR images based on semantic segmentation [J]. Systems Engineering and Electronics, 2022, 44(4): 1195-1201.
[9]	Jingming SUN, Shengkang YU, Jun SUN. Pose sensitivity analysis of HRRP recognition based on deep learning [J]. Systems Engineering and Electronics, 2022, 44(3): 802-807.
[10]	Hengyan LIU, Limin ZHANG, Wenjun YAN, Zhaogen ZHONG, Qing LING, Xiaojun LIANG. LDPC decoding based on WBP-CNN algorithm [J]. Systems Engineering and Electronics, 2022, 44(3): 1030-1035.
[11]	Kai SHAO, Miaomiao ZHU, Guangyu WANG. Modulation recognition method based on generative adversarial andconvolutional neural network [J]. Systems Engineering and Electronics, 2022, 44(3): 1036-1043.
[12]	Xi ZHANG, Zhengmeng JIN, Yaqin JIANG. Total variation algorithm with depth image priors for image colorization [J]. Systems Engineering and Electronics, 2022, 44(2): 385-393.
[13]	Yunxiang YAO, Ying CHEN. Target tracking network based on dual-modal interactive fusion under attention mechanism [J]. Systems Engineering and Electronics, 2022, 44(2): 410-419.
[14]	Qinzhe LYU, Yinghui QUAN, Minghui SHA, Shuxian DONG, Mengdao XING. Ensemble deep learning-based intelligent classification of active jamming [J]. Systems Engineering and Electronics, 2022, 44(12): 3595-3602.
[15]	Yiqiang TANG, Xiaopeng YANG, Shengming ZHU. Low-orbit satellite channel prediction algorithm based on the hybrid CNN-BiLSTM using attention mechanism [J]. Systems Engineering and Electronics, 2022, 44(12): 3863-3870.