基于图像特征的红外与低照度图像融合

doi:10.12305/j.issn.1001-506X.2023.08.13

Abstract

Abstract:

Aiming at the problem that it is difficult to analyze the target and background information in the scene of infrared images and low illumination images independently, an infrared and low illumination image fusion algorithm based on image features is proposed. Firstly, the infrared and low illumination images are processed to optimize their characteristics respectively. Secondly, the non-subsampled shearlet transform is used to decompose them in high and low frequency. Thirdly, the improved Laplace weighted algorithm is used to fuse the low-frequency images. The improved pulse coupled neural network is used to fuse the high-frequency images. Finally, the fused image is obtained by non-subsampled shearlet inverse transform. Experimental results show that the proposed algorithm can effectively fuse infrared and low illumination images, reduce the impact of noise on the fused image, improve the clarity of the fused image, and the contrast of the image is moderate, which is suifable the visual perception effect of human eyes.

Key words: low illumination image, infrared image, non-subsampled shearlet transform, pulse-coupled neural network, image fusion algorithm

CLC Number:

TP391.41

Huiying WANG, Chunping WANG, Qiang FU, Zishuo HAN, Dongdong ZHANG. Infrared and low illumination image fusion based on image features[J]. Systems Engineering and Electronics, 2023, 45(8): 2395-2404.

Figures/Tables 14

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Fig.9

Fig.10

Fig.11

Fig.12

Fig.13

Table 1

References 31

1	邢雅琼, 王晓丹, 毕凯, 等. 基于NSCT和ICA的红外和可见光图像融合方法[J]. 系统工程与电子技术, 2013, 35 (11): 2251- 2256.
	XING Y Q , WANG X D , BI K , et al. Infrared and visible image fusion method based on NSCT and ICA[J]. Systems Engineering and Electronics, 2013, 35 (11): 2251- 2256.
2	江泽涛, 覃露露, 秦嘉奇, 等. 一种基于MDARNet的低照度图像增强方法[J]. 软件学报, 2021, 32 (12): 3977- 3991. doi: 10.13328/j.cnki.jos.006112
	JIANG Z T , QIN L L , QIN J Q , et al. Low-light image enhancement method based on MDARNet[J]. Journal of Software, 2021, 32 (12): 3977- 3991. doi: 10.13328/j.cnki.jos.006112
3	周晓玲, 江泽涛. 结合脉冲耦合神经网络与引导滤波的红外与可见光图像融合[J]. 光学学报, 2019, 39 (11): 1110003.
	ZHOU X L , JIANG Z T . Infrared and visible image fusion combining pulse-coupled neural network and guided filtering[J]. Acta Optical Sinica, 2019, 39 (11): 1110003.
4	LIU K , GUO L , LI H H , et al. Fusion of infrared and visible light images based on region segmentation[J]. Chinese Journal of Aeronautics, 2009, 22 (1): 75- 80. doi: 10.1016/S1000-9361(08)60071-0
5	GAI D , SHEN X J , CHEN H P , et al. Medical image fusion using the PCNN based on IQPSO in NSST domain[J]. IET Image Processing, 2020, 14 (9): 1870- 1880. doi: 10.1049/iet-ipr.2020.0040
6	CHEN J , LI X J , LUO L B , et al. Infrared and visible image fusion based on target-enhanced multiscale transform decomposition[J]. Information Sciences, 2020, 508, 64- 78. doi: 10.1016/j.ins.2019.08.066
7	BHAT S , KOUNDAL D . Multi-focus image fusion using neutrosophic based wavelet transform[J]. Applied Soft Computing, 2021, 106, 107307. doi: 10.1016/j.asoc.2021.107307
8	GUO L Q , CAO X , LIU L . Dual-tree biquaternion wavelet transform and its application to color image fusion[J]. Signal Processing, 2020, 171, 107513. doi: 10.1016/j.sigpro.2020.107513
9	AGHAMALEKI J A , GHORBANI A . Image fusion using dual tree discrete wavelet transform and weights optimization[J]. The Visual Computer, 2023, 39, 1181- 1191. doi: 10.1007/s00371-021-02396-9
10	ZHAO C H , GUO Y T , WANG Y L . A fast fusion scheme for infrared and visible light images in NSCT domain[J]. Infrared Physics and Technology, 2015, 72, 266- 275. doi: 10.1016/j.infrared.2015.07.026
11	YUE J J , LI M Z , CHEN J , et al. Multi-focus infrared image fusion based on pulse coupled neural networks in a nonsubsampled contourlet transform domain[J]. Infrared Technology, 2017, 39 (9): 798- 806.
12	JIAO J , WU L D . Pansharpening with a gradient domain GIF based on NSST[J]. Electronics, 2019, 8 (2): 229. doi: 10.3390/electronics8020229
13	LI L L , SI Y J , WANG L L , et al. A novel approach for multi-focus image fusion based on SF-PAPCNN and ISML in NSST domain[J]. Multimedia Tools and Applications, 2020, 79 (6): 24303- 24328.
14	WEI T , LIU Y , CHENG J , et al. A phase congruency-based green fluorescent protein and phase contrast image fusion method in nonsubsampled shearlet transform domain[J]. Microscopy Research and Technique, 2020, 83 (10): 1225- 1234.
15	WANG J , YANG K , REN P , et al. Multi-source image fusion algorithm based on fast weighted guided filter[J]. Journal of Systems Engineering and Electronics, 2019, 30 (5): 831- 840. doi: 10.21629/JSEE.2019.05.02
16	GANASALA P , PRASAD A D . Contrast enhanced multi sensor image fusion based on guided image filter and NSST[J]. IEEE Trans.on Sensors Journal, 2020, 20 (2): 939- 946. doi: 10.1109/JSEN.2019.2944249
17	ZHANG S , HUANG F Y , ZHONG H , et al. Multi-modal image fusion via sparse representation and multi-scale anisotropic guided measure[J]. IEEE Access, 2020, (8): 35638- 35649.
18	BIRGIT K , LORENZ D A , LENA V . Denoising of image gradients and total generalized variation denoising[J]. Journal of Mathematical Imaging & Vision, 2019, 61, 21- 39.
19	EASLEY G , LABATE D , LIM W Q . Sparse directional image representations using the discrete shearlet transform[J]. Applied Computational Harmonic Analysis, 2008, 25 (1): 25- 46. doi: 10.1016/j.acha.2007.09.003
20	ECKHORN R , REIBOECK H J , ARNDT M , et al. Feature linking via synchronization among distributed assemblies: simulation of results from cat cortex[J]. Neural Computation, 1990, 2 (3): 293- 307. doi: 10.1162/neco.1990.2.3.293
21	YIN M , LIU X N , LIU Y , et al. Medical image fusion with parameter adaptive pulse coupled neural network in nonsubsampled shearlet transform domain[J]. IEEE Trans.on Instrumentation and Measurement, 2019, 68 (1): 49- 64. doi: 10.1109/TIM.2018.2838778
22	LI X X , GUO X P , HAN P F , et al. Laplacian re-decomposition for multimodal medical image fusion[J]. IEEE Trans.on Instrumentation and Measurement, 2020, 69 (9): 6880- 6890. doi: 10.1109/TIM.2020.2975405
23	LI Y Q , ZHAO H T , HU Z W , et al. IVFuseNet: fusion of infrared and visible light images for depth prediction[J]. Information Fusion, 2019, 58, 1- 12.
24	TOET Alexander. TNO image fusion dataset[EB/OL]. [2021-07-30]. https://figshare.com/articles/TNO-Image-Fusion-Dataset/1008029.
25	WU C , CHEN L . Infrared and visible image fusion method of dual NSCT and PCNN[J]. PLoS One, 2020, 15 (9): e0239535. doi: 10.1371/journal.pone.0239535
26	WANG L , LUO W , CHEN J H , et al. Fusion of infrared and visible images based on adaptive PCNN and information extraction[J]. Computer Engineering and Applications, 2018, 54 (4): 192- 198.
27	XING X X, LIU C, LUO C, et al. Infrared and visible image fusion based on nonlinear enhancement and NSST decomposition[EB/OL]. [2022-03-25]. https//www.research.square.com/article/rs-15346v1.
28	DU Q L , XU H , MA Y , et al. Fusing infrared and visible images of different resolutions via total variation model[J]. Sensors, 2018, 18 (11): 3827. doi: 10.3390/s18113827
29	MA J Y , CHEN C , LI C , et al. Infrared and visible image fusion via gradient transfer and total variation minimization[J]. Information Fusion, 2016, 31, 100- 109. doi: 10.1016/j.inffus.2016.02.001
30	JIN X , JIANG Q , YAO S W , et al. A survey of infrared and visual image fusion methods[J]. Infrared Physics and Technology, 2017, 85, 478- 501. doi: 10.1016/j.infrared.2017.07.010
31	KONG W W , LEI Y , REN M M . Fusion method for infrared and visible images based on improved quantum theory model[J]. Neuro Computing, 2016, 212, 12- 21.

源图像	指标	NSCT_PCNN	PCNN	NSST	本文算法
第1组	IE	6.561 8	6.668 0	6.569 1	7.791 8
	SD	45.329 1	45.423 4	45.356 8	60.576 2
	AG	7.453 2	7.612 8	7.642 0	8.921 0
	Q^AB/F	0.504 0	0.564 1	0.521 0	0.582 8
第2组	IE	6.803 2	5.521 9	6.674 2	7.313 0
	SD	38.751 7	38.721 0	38.854 2	74.012 5
	AG	5.781 6	5.359 6	5.742 9	8.672 0
	Q^AB/F	0.340 2	0.561 6	0.551 3	0.584 2
第3组	IE	6.541 2	6.211 2	6.421 3	7.294 6
	SD	26.324 0	24.104 2	26.214 0	50.321 1
	AG	4.401 5	3.014 2	4.025 6	6.121 0
	Q^AB/F	0.391 8	0.401 2	0.354 2	0.471 3
第4组	IE	6.701 2	7.399 5	7.066 8	7.651 6
	SD	23.191 2	26.146 2	25.486 9	30.151 6
	AG	2.295 2	4.352 8	3.961 6	5.659 6
	Q^AB/F	0.382 4	0.401 5	0.375 2	0.542 1
第5组	IE	6.954 8	6.707 8	6.575 3	7.213 4
	SD	31.492 6	27.451 2	25.617 1	45.142 3
	AG	4.809 8	4.754 3	4.017 6	5.013 6
	Q^AB/F	0.409 6	0.391 2	0.301 9	0.553 8

[1]	Haijun LI, Fancheng KONG, Yun LIN. Infrared ship detection algorithm based on improved YOLOv5s [J]. Systems Engineering and Electronics, 2023, 45(8): 2415-2422.
[2]	Jian WANG, Zihao HE, Jie LIU, Ke YANG. Image fusion algorithm based on gradient domain guided filtering and improved PCNN [J]. Systems Engineering and Electronics, 2022, 44(8): 2381-2392.
[3]	Xu LI, Meng DING, Donghui WEI, Xiaozhou WU, Yunfeng CAO. Depth estimation method based on monocular infrared image in VDAS [J]. Systems Engineering and Electronics, 2021, 43(5): 1210-1217.
[4]	Siqi HUA, Wei ZHAO, Jianye LIU. Background suppression algorithms based on improved filter and image multi-scale transformation [J]. Systems Engineering and Electronics, 2020, 42(8): 1679-1684.
[5]	LI Cui-yun, CAO Xiao-nan, LIAO Liang-xiong, JIANG Zhou. Track before detect using Gaussian particle probability hypothesis density [J]. Systems Engineering and Electronics, 2015, 37(4): 740-745.
[6]	LIU Gang, LIANG Xiao-geng, ZHANG Jing-guo. Contourlet transform and improved fuzzy c-means clustering based infrared image segmentation [J]. Journal of Systems Engineering and Electronics, 2011, 33(2): 443-448.
[7]	ZHU Ying-hong, LI Jun-shan, TANG Yu. Matching algorithm for IR/visible images based on CSS corner extraction [J]. Journal of Systems Engineering and Electronics, 2011, 33(11): 2540-2545.

Infrared and low illumination image fusion based on image features

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 14

References 31

Related Articles 7

Recommended Articles

Metrics

Comments