基于梯度域导向滤波器和改进PCNN的图像融合算法

doi:10.12305/j.issn.1001-506X.2022.08.01

Abstract

Abstract:

In order to solve the problems of halo artefacts and unfavourable visual perception in the fused images, this paper proposes an image fusion algorithm based on gradient-domain guided filtering and an improved pulse-coupled neural network (PCNN). First, an image fusion model is constructed using the image features of image structure, sharpness and contrast saliency. Secondly, the initial decision map is optimised by inter-pixel correlation using gradient-domain guided filtering instead of the traditional optimisation method. Then, the optimised decision map is used as external input to stimulate the improved PCNN model to obtain the fusion weight map. Finally, the source image and the fusion weight map are weighted to obtain the final Finally, the source image and the fusion weight map are weighted to obtain the final fused image. The experimental results show that this method can better preserve the image edge, texture and detail information, avoid halo artefacts on the target edge, and facilitate visual observation.

Key words: guided filtering, improved pulse-coupled neural network (PCNN), image fusion

CLC Number:

TP391

Jian WANG, Zihao HE, Jie LIU, Ke YANG. Image fusion algorithm based on gradient domain guided filtering and improved PCNN[J]. Systems Engineering and Electronics, 2022, 44(8): 2381-2392.

Figures/Tables 14

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Fig.9

Table 1

Fig.10

Table 2

Fig.11

Table 3

References 31

1	BEN H A , HE Y , KRIM H , et al. A multiscale approach to pixel-level image fusion[J]. Integrated Computer Aided Engineering, 2005, 12 (2): 135- 146. doi: 10.3233/ICA-2005-12201
2	叶传奇, 王宝树, 苗启广. 基于区域分割的NSCT域SAR与全色图像融合算法[J]. 系统工程与电子技术, 2010, 32 (3): 609- 613.
	YE C Q , WANG B S , MIAO Q G . Fusion algorithm of SAR and panchromatic images based on region segmentation in NSCT domain[J]. Systems Engineering and Electronics, 2010, 32 (3): 609- 613.
3	郭明, 王书满. 基于区域和方向方差加权信息熵的图像融合[J]. 系统工程与电子技术, 2013, 35 (4): 720- 724. doi: 10.3969/j.issn.1001-506X.2013.04.07
	GUO M , WANG S M . Image fusion based on region and directional variance weighted entropy[J]. Systems Engineering and Electronics, 2013, 35 (4): 720- 724. doi: 10.3969/j.issn.1001-506X.2013.04.07
4	HE K M, JIAN S, TANG X O. Guided image filtering[C]//Proc. of the European Conference on Computer Vision, 2010.
5	LI S T , KANG X D , HU J W . Image fusion with guided filtering[J]. IEEE Trans.on Image Processing, 2013, 22 (7): 2864- 2875. doi: 10.1109/TIP.2013.2244222
6	江泽涛, 吴辉, 周晓玲. 基于改进引导滤波和双通道脉冲发放皮层模型的红外与可见光图像融合算法[J]. 光学学报, 2018, 32 (2): 112- 120.
	JIANG Z T , WU H , ZHOU X L . Infrared and visible image fusion algorithm based on improved guided filtering and dual-channel spiking cortical model[J]. Acta Optica Sinica, 2018, 32 (2): 112- 120.
7	PEI C Y , FAN K G , WANG W S . Two-scale multimodal medical image fusion based on guided filtering and sparse representation[J]. IEEE Access, 2020, 8, 140216- 140233. doi: 10.1109/ACCESS.2020.3013027
8	ECKHORN R , REITBOECK H J , ARNDT M . Feature linking via synchronization among distributed assemblies: simulations of results from cat visual cortex[J]. Neural Computation, 1999, 2 (3): 293- 307.
9	PANIGRAHY C , SEAL A , MAHATO N K . Fractal dimension based parameter adaptive dual channel PCNN for multi-focus image fusion[J]. Optics and Lasers in Engineering, 2020, 133, 106141. doi: 10.1016/j.optlaseng.2020.106141
10	ZHANG K , HUANG Y D , ZHAO C . Remote sensing image fusion via RPCA and adaptive PCNN in NSST domain[J]. International Journal of Wavelets, Multiresolution and Information Processing, 2018, 16 (5): 1850037. doi: 10.1142/S0219691318500376
11	NIE R C , CAO J D , ZHOU D M , et al. Multi-source information exchange encoding with PCNN for medical image fusion[J]. IEEE Trans.on Circuits and Systems for Video Technology, 2020, 31 (3): 986- 1000.
12	BROUSSARD R P , ROGERS S K . Physiologically motivated image fusion for object detection using a pulse coupled neural network[J]. IEEE Trans.on Neural Networks, 1999, 10 (3): 554- 563. doi: 10.1109/72.761712
13	王健, 张修飞, 任萍, 等. 基于增补小波变换和PCNN的NSCT域图像融合算法[J]. 计算机工程与科学, 2018, 40 (10): 1822- 1828.
	WANG J , ZHANG X F , REN P , et al. An image fusion algorithm based on complementary wavelet transform and PCNN in NSCT domain[J]. Computer Engineering and Science, 2018, 40 (10): 1822- 1828.
14	GAI D , SHEN X J , CHEN H P , et al. Medical image fusion using the PCNN based on IQPSO in NSST domain[J]. IET Image Processing, 2020, 14 (9): 1870- 1880. doi: 10.1049/iet-ipr.2020.0040
15	LI Z G , ZHENG J H , ZHU Z J , et al. Weighted guided image filtering[J]. IEEE Trans.on Image Processing, 2014, 24 (1): 120- 129.
16	KOU F , CHEN W H , WEN C Y , et al. Gradient domain guided image filtering[J]. IEEE Trans.on Image Processing, 2015, 24 (11): 4528- 4539. doi: 10.1109/TIP.2015.2468183
17	张军英, 卢涛. 通过脉冲耦合神经网络来增强图像[J]. 计算机工程与应用, 2003, 39 (19): 93- 95.
	ZHANG J Y , LU T . Enhancement of image by PCNN[J]. Computer Engineering and Applications, 2003, 39 (19): 93- 95.
18	HASSEN R , WANG Z . Objective quality assessment for multiexposure multifocus image fusion[J]. IEEE Trans.on Image Processing, 2015, 24 (9): 2712- 2724. doi: 10.1109/TIP.2015.2428051
19	WANG J , YANG K , REN P , et al. Multi-source image fusion algorithm based on fast weighted guided filter[J]. Journal of Systems Engineering and Electronics, 2019, 30 (5): 831- 840. doi: 10.21629/JSEE.2019.05.02
20	王帅. 基于PCNN的多聚焦图像融合算法研究[D]. 兰州: 兰州大学, 2017.
	WANG S. Research on multi focus image fusion algorithm based on PCNN[D]. Lanzhou: Lanzhou University, 2017.
21	ZHOU Z Q , WANG B , LI S , et al. Perceptual fusion of infrared and visible images through a hybrid multi-scale decomposition with Gaussian and bilateral filters[J]. Information Fusion, 2016, 30, 15- 26. doi: 10.1016/j.inffus.2015.11.003
22	XIANG T Z , YAN L , GAO R R . A fusion algorithm for infrared and visible images based on adaptive dual-channel unit-linking PCNN in NSCT domain[J]. Infrared Physics & Technology, 2015, 69, 53- 61.
23	LIU Y , LIU S P , WANG Z F . A general framework for image fusion based on multi-scale transform and sparse representation[J]. Information Fusion, 2015, 24, 147- 164. doi: 10.1016/j.inffus.2014.09.004
24	ZHU Z Q , ZHENG M Y , QI G Q , et al. A phase congruency and local laplacian energy based multi-modality medical image fusion method in NSCT domain[J]. IEEE Access, 2019, 7, 20811- 20824.
25	CHEN J , LI X J , LUO L B , et al. Infrared and visible image fusion based on target-enhanced multiscale transform decomposition[J]. Information Sciences, 2020, 508, 64- 78.
26	郑伟, 赵成晨, 郝冬梅. NSST与改进PCNN相结合的甲状腺图像融合[J]. 光电工程, 2016, 43 (10): 42- 48.
	ZHENG W , ZHAO C C , HAO D M . Thyroid image fusion based on NSST and improved PCNN[J]. Opto-Electronic Engineering, 2016, 43 (10): 42- 48.
27	CHAI Y , LI H F , QU J F . Image fusion scheme using a novel dual-channel PCNN in lifting stationary wavelet domain[J]. Optics Communications, 2010, 283 (19): 3591- 3602.
28	LIU Z D , YIN H P , YI C , et al. A novel approach for multimodal medical image fusion[J]. Expert Systems with Applications an International Journal, 2014, 41 (16): 7425- 7435.
29	XYDEAS C S , PETROVIC V . Objective image fusion performance measure[J]. Military Technical Courier, 2000, 56 (4): 181- 193.
30	PIELLA G, HEIJMANS H. A new quality metric for image fusion[C]//Proc. of the International Conference on Image Processing, 2003.
31	YIN C , BLUM R S . A new automated quality assessment algorithm for image fusion[J]. Image & Vision Computing, 2009, 27 (10): 1421- 1432.

源图像	指标	HMSD	NSCT-PCNN	MST-SR	文献[24]	文献[25]	本文算法
第1组	Q_EN Q_AG Q_W Q_MI	7.321 6 5.260 3 0.900 3 3.490 9	7.294 3 3.681 7 0.847 9 3.444 0	7.334 1 5.281 8 0.898 4 3.446 2	4.593 3 5.223 9 0.907 8 3.561 3	4.818 1 4.817 0 0.901 3 3.572 7	7.364 8 5.167 4 0.909 2 3.368 9
第2组	Q_EN Q_AG Q_W Q_MI	7.410 4 4.495 0 0.844 0 4.499 4	7.501 7 2.791 3 0.815 2 4.204 9	7.406 3 4.542 2 0.838 7 4.403 0	7.508 3 4.558 8 0.846 4 4.498 8	7.524 9 4.139 8 0.834 6 5.080 4	7.513 6 4.611 5 0.876 5 5.428 4
第3组	Q_EN Q_AG Q_W Q_MI	7.123 8 7.183 7 0.863 2 5.204 7	7.106 4 5.728 9 0.883 7 5.521 9	7.112 3 7.138 5 0.896 2 5.916 0	7.106 8 7.161 4 0.870 3 5.861 2	7.119 6 6.981 2 0.869 0 5.321 9	7.150 2 7.116 0 0.899 5 5.609 2

源图像	指标	HMSD	NSCT-PCNN	MST-SR	文献[24]	文献[25]	本文算法
第1组	Q_EN Q_AG Q_W Q_MI	6.274 2 6.802 2 0.582 7 1.747 2	5.908 5 3.559 5 0.570 6 2.074 9	6.893 8 6.115 4 0.761 3 1.408 5	6.878 6 6.511 3 0.808 3 1.939 6	6.259 3 6.666 7 0.866 2 3.495 4	6.496 5 6.166 7 0.812 4 3.565 4
第2组	Q_EN Q_AG Q_W Q_MI	5.235 8 7.926 4 0.844 8 2.334 9	5.172 3 3.559 5 0.734 8 2.210 4	5.312 7 7.463 8 0.893 0 2.646 1	5.288 2 7.421 5 0.892 3 2.562 9	4.819 6 6.510 8 0.764 9 2.146 7	5.618 5 7.150 3 0.898 7 2.710 7
第3组	Q_EN Q_AG Q_W Q_MI	4.934 0 6.039 4 0.815 3 2.504 5	4.663 1 3.165 9 0.824 9 2.332 2	5.141 5 5.313 2 0.812 0 2.162 7	4.866 1 5.210 7 0.819 8 2.233 5	4.418 5 5.281 9 0.811 4 2.529 4	5.321 5 5.863 2 0.902 2 3.234 5

源图像	指标	HMSD	NSCT-PCNN	MST-SR	文献[24]	文献[25]	本文算法
第1组	Q_EN Q_AG Q_W Q_MI	6.863 6 5.252 9 0.774 6 1.522 9	6.264 2 3.535 7 0.459 9 1.124 1	6.954 9 4.498 3 0.682 3 1.247 5	6.902 2 4.679 9 0.620 3 1.234 5	6.646 0 4.419 8 0.579 4 2.035 8	6.976 8 4.070 6 0.805 4 1.373 4
第2组	Q_EN Q_AG Q_W Q_MI	7.309 5 9.247 7 0.726 7 1.938 6	7.013 6 6.448 5 0.749 1 1.748 4	7.378 7 8.484 3 0.535 3 1.543 7	7.370 7 8.686 2 0.454 6 1.706 7	7.220 6 8.001 3 0.362 1 2.830 3	7.385 8 8.004 3 0.846 6 2.195 3
第3组	Q_EN Q_AG Q_W Q_MI	6.563 6 8.673 9 0.753 2 2.520 9	6.484 2 6.060 3 0.741 5 2.603 3	6.836 1 8.070 1 0.665 0 2.707 3	6.811 5 8.153 7 0.910 4 3.007 2	6.516 5 7.647 9 0.868 5 2.801 7	6.886 4 7.894 6 0.896 4 3.270 0

[1]	Yingping TONG, Yinghui QUAN, Wei FENG, Mengdao XING. Multi-source remote sensing image fusion method based on spatial-spectrum information collaboration and Gram-Schmidt transform [J]. Systems Engineering and Electronics, 2022, 44(7): 2074-2083.
[2]	Ying AN, Xunli FAN, Li CHEN, Pei LIU. Image fusion combining FABEMD with improved saliency detection [J]. Systems Engineering and Electronics, 2020, 42(2): 292-300.
[3]	Guoming XU, Hongwu YUAN, Mogen XUE, Feng WANG. Full polarization parameters low-rank and sparse factorization for pseudo color image fusion [J]. Systems Engineering and Electronics, 2020, 42(11): 2450-2460.
[4]	ZONG Jingjing, QIU Tianshuang. Characteristic analysis for the l-1 norm of sparse coefficients in sparse representation [J]. Systems Engineering and Electronics, 2019, 41(12): 2692-2696.
[5]	WANG Chen, FAN Yangyu. Saliency detection method based on dynamic guided filtering [J]. Systems Engineering and Electronics, 2018, 40(6): 1391-1397.
[6]	LIU Bin, FU Zhongwang. Multi-focus image fusion based on four-channel non-separable lifting wavelet [J]. Systems Engineering and Electronics, 2018, 40(2): 463-471.
[7]	CHANG Lihong, FENG Xiangchu, ZHANG Rui. Image fusion scheme based on quaternion wavelet transform and sparse representation [J]. Systems Engineering and Electronics, 2017, 39(7): 1633-1639.
[8]	ZHAO Lanfei, WANG Aili, WANG Bo, Lv Xinmiao. Image enhancement algorithm based on sub-image fusion [J]. Systems Engineering and Electronics, 2017, 39(12): 2840-2848.
[9]	HU Xiao-wei, TONG Ning-ning, HE Xing-yu, LEI Teng, WANG Yu-chen. High-speed spinning target imaging based on narrow-band radar networks [J]. Systems Engineering and Electronics, 2016, 38(4): 792-798.
[10]	ZHOU Zhi-qiang, WANG Bo, LI Li-guang, LI Sun. Image fusion based on a hybrid decomposition via bilateral and Gaussian filters [J]. Systems Engineering and Electronics, 2016, 38(1): 8-13.
[11]	LIU Bin, LIU Wei-jie, WEI YAN-ing. Construction of the six channel multi-scale singular value decomposition and its application in multi-focus image fusion [J]. Systems Engineering and Electronics, 2015, 37(9): 2191-2197.
[12]	XING Ya-qiong, WANG Xiao-dan, LIANG Bing-jie, QIN Zhuo. Fusion technique for multi-focus images based on orientation information measurement in non-subsample shearlet transform domain [J]. Systems Engineering and Electronics, 2015, 37(1): 191-198.
[13]	MA Zhong-li， WEN Jie， HAO Liang-liang. Video image defogging algorithm for surface ship scenes [J]. Systems Engineering and Electronics, 2014, 36(9): 1860-1867.
[14]	GUO Ming, WANG Shu-man. Image fusion based on region and directional variance weighted entropy [J]. Journal of Systems Engineering and Electronics, 2013, 35(4): 720-724.
[15]	WANG Xiaowen, ZHAO Zonggui, TANG Lei. Novel quality metric for infrared and visible image fusion [J]. Journal of Systems Engineering and Electronics, 2012, 34(5): 871-875.

Image fusion algorithm based on gradient domain guided filtering and improved PCNN

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 14

References 31

Related Articles 15

Recommended Articles

Metrics

Comments