基于梯度域导向滤波器和改进PCNN的图像融合算法

doi:10.12305/j.issn.1001-506X.2022.08.01

摘要/Abstract

摘要：

为解决当前融合后图像存在的光晕伪影现象以及不利于视觉感知的问题, 提出了一种基于梯度域导向滤波(gradient domain guided filtering, GDGF)和改进的脉冲耦合神经网络(pulse-coupled neural network, PCNN)的图像融合算法。首先, 利用图像结构、清晰度以及对比度显著性的图像特征构建图像融合模型。其次, 采用梯度域导向滤波取代传统优化方法, 通过像素间相关性优化初始决策图。然后, 将优化决策图作为外部输入刺激改进PCNN模型, 得到融合权重图。最后, 对源图像和融合权重图进行加权操作得到最终融合图像。实验结果表明, 所提方法更好地保留图像边缘、纹理和细节信息, 避免目标边缘的光晕伪影现象, 且利于视觉观察。

关键词: 导向滤波器, 改进的脉冲耦合神经网络, 图像融合

Abstract:

In order to solve the problems of halo artefacts and unfavourable visual perception in the fused images, this paper proposes an image fusion algorithm based on gradient-domain guided filtering and an improved pulse-coupled neural network (PCNN). First, an image fusion model is constructed using the image features of image structure, sharpness and contrast saliency. Secondly, the initial decision map is optimised by inter-pixel correlation using gradient-domain guided filtering instead of the traditional optimisation method. Then, the optimised decision map is used as external input to stimulate the improved PCNN model to obtain the fusion weight map. Finally, the source image and the fusion weight map are weighted to obtain the final Finally, the source image and the fusion weight map are weighted to obtain the final fused image. The experimental results show that this method can better preserve the image edge, texture and detail information, avoid halo artefacts on the target edge, and facilitate visual observation.

Key words: guided filtering, improved pulse-coupled neural network (PCNN), image fusion

中图分类号:

TP391

王健, 何自豪, 刘洁, 杨珂. 基于梯度域导向滤波器和改进PCNN的图像融合算法[J]. 系统工程与电子技术, 2022, 44(8): 2381-2392.

Jian WANG, Zihao HE, Jie LIU, Ke YANG. Image fusion algorithm based on gradient domain guided filtering and improved PCNN[J]. Systems Engineering and Electronics, 2022, 44(8): 2381-2392.

图/表 14

图1

图2

图3

图4

图5

图6

图7

图8

图9

表1

图10

表2

图11

表3

参考文献 31

1	BEN H A , HE Y , KRIM H , et al. A multiscale approach to pixel-level image fusion[J]. Integrated Computer Aided Engineering, 2005, 12 (2): 135- 146. doi: 10.3233/ICA-2005-12201
2	叶传奇, 王宝树, 苗启广. 基于区域分割的NSCT域SAR与全色图像融合算法[J]. 系统工程与电子技术, 2010, 32 (3): 609- 613.
	YE C Q , WANG B S , MIAO Q G . Fusion algorithm of SAR and panchromatic images based on region segmentation in NSCT domain[J]. Systems Engineering and Electronics, 2010, 32 (3): 609- 613.
3	郭明, 王书满. 基于区域和方向方差加权信息熵的图像融合[J]. 系统工程与电子技术, 2013, 35 (4): 720- 724. doi: 10.3969/j.issn.1001-506X.2013.04.07
	GUO M , WANG S M . Image fusion based on region and directional variance weighted entropy[J]. Systems Engineering and Electronics, 2013, 35 (4): 720- 724. doi: 10.3969/j.issn.1001-506X.2013.04.07
4	HE K M, JIAN S, TANG X O. Guided image filtering[C]//Proc. of the European Conference on Computer Vision, 2010.
5	LI S T , KANG X D , HU J W . Image fusion with guided filtering[J]. IEEE Trans.on Image Processing, 2013, 22 (7): 2864- 2875. doi: 10.1109/TIP.2013.2244222
6	江泽涛, 吴辉, 周晓玲. 基于改进引导滤波和双通道脉冲发放皮层模型的红外与可见光图像融合算法[J]. 光学学报, 2018, 32 (2): 112- 120.
	JIANG Z T , WU H , ZHOU X L . Infrared and visible image fusion algorithm based on improved guided filtering and dual-channel spiking cortical model[J]. Acta Optica Sinica, 2018, 32 (2): 112- 120.
7	PEI C Y , FAN K G , WANG W S . Two-scale multimodal medical image fusion based on guided filtering and sparse representation[J]. IEEE Access, 2020, 8, 140216- 140233. doi: 10.1109/ACCESS.2020.3013027
8	ECKHORN R , REITBOECK H J , ARNDT M . Feature linking via synchronization among distributed assemblies: simulations of results from cat visual cortex[J]. Neural Computation, 1999, 2 (3): 293- 307.
9	PANIGRAHY C , SEAL A , MAHATO N K . Fractal dimension based parameter adaptive dual channel PCNN for multi-focus image fusion[J]. Optics and Lasers in Engineering, 2020, 133, 106141. doi: 10.1016/j.optlaseng.2020.106141
10	ZHANG K , HUANG Y D , ZHAO C . Remote sensing image fusion via RPCA and adaptive PCNN in NSST domain[J]. International Journal of Wavelets, Multiresolution and Information Processing, 2018, 16 (5): 1850037. doi: 10.1142/S0219691318500376
11	NIE R C , CAO J D , ZHOU D M , et al. Multi-source information exchange encoding with PCNN for medical image fusion[J]. IEEE Trans.on Circuits and Systems for Video Technology, 2020, 31 (3): 986- 1000.
12	BROUSSARD R P , ROGERS S K . Physiologically motivated image fusion for object detection using a pulse coupled neural network[J]. IEEE Trans.on Neural Networks, 1999, 10 (3): 554- 563. doi: 10.1109/72.761712
13	王健, 张修飞, 任萍, 等. 基于增补小波变换和PCNN的NSCT域图像融合算法[J]. 计算机工程与科学, 2018, 40 (10): 1822- 1828.
	WANG J , ZHANG X F , REN P , et al. An image fusion algorithm based on complementary wavelet transform and PCNN in NSCT domain[J]. Computer Engineering and Science, 2018, 40 (10): 1822- 1828.
14	GAI D , SHEN X J , CHEN H P , et al. Medical image fusion using the PCNN based on IQPSO in NSST domain[J]. IET Image Processing, 2020, 14 (9): 1870- 1880. doi: 10.1049/iet-ipr.2020.0040
15	LI Z G , ZHENG J H , ZHU Z J , et al. Weighted guided image filtering[J]. IEEE Trans.on Image Processing, 2014, 24 (1): 120- 129.
16	KOU F , CHEN W H , WEN C Y , et al. Gradient domain guided image filtering[J]. IEEE Trans.on Image Processing, 2015, 24 (11): 4528- 4539. doi: 10.1109/TIP.2015.2468183
17	张军英, 卢涛. 通过脉冲耦合神经网络来增强图像[J]. 计算机工程与应用, 2003, 39 (19): 93- 95.
	ZHANG J Y , LU T . Enhancement of image by PCNN[J]. Computer Engineering and Applications, 2003, 39 (19): 93- 95.
18	HASSEN R , WANG Z . Objective quality assessment for multiexposure multifocus image fusion[J]. IEEE Trans.on Image Processing, 2015, 24 (9): 2712- 2724. doi: 10.1109/TIP.2015.2428051
19	WANG J , YANG K , REN P , et al. Multi-source image fusion algorithm based on fast weighted guided filter[J]. Journal of Systems Engineering and Electronics, 2019, 30 (5): 831- 840. doi: 10.21629/JSEE.2019.05.02
20	王帅. 基于PCNN的多聚焦图像融合算法研究[D]. 兰州: 兰州大学, 2017.
	WANG S. Research on multi focus image fusion algorithm based on PCNN[D]. Lanzhou: Lanzhou University, 2017.
21	ZHOU Z Q , WANG B , LI S , et al. Perceptual fusion of infrared and visible images through a hybrid multi-scale decomposition with Gaussian and bilateral filters[J]. Information Fusion, 2016, 30, 15- 26. doi: 10.1016/j.inffus.2015.11.003
22	XIANG T Z , YAN L , GAO R R . A fusion algorithm for infrared and visible images based on adaptive dual-channel unit-linking PCNN in NSCT domain[J]. Infrared Physics & Technology, 2015, 69, 53- 61.
23	LIU Y , LIU S P , WANG Z F . A general framework for image fusion based on multi-scale transform and sparse representation[J]. Information Fusion, 2015, 24, 147- 164. doi: 10.1016/j.inffus.2014.09.004
24	ZHU Z Q , ZHENG M Y , QI G Q , et al. A phase congruency and local laplacian energy based multi-modality medical image fusion method in NSCT domain[J]. IEEE Access, 2019, 7, 20811- 20824.
25	CHEN J , LI X J , LUO L B , et al. Infrared and visible image fusion based on target-enhanced multiscale transform decomposition[J]. Information Sciences, 2020, 508, 64- 78.
26	郑伟, 赵成晨, 郝冬梅. NSST与改进PCNN相结合的甲状腺图像融合[J]. 光电工程, 2016, 43 (10): 42- 48.
	ZHENG W , ZHAO C C , HAO D M . Thyroid image fusion based on NSST and improved PCNN[J]. Opto-Electronic Engineering, 2016, 43 (10): 42- 48.
27	CHAI Y , LI H F , QU J F . Image fusion scheme using a novel dual-channel PCNN in lifting stationary wavelet domain[J]. Optics Communications, 2010, 283 (19): 3591- 3602.
28	LIU Z D , YIN H P , YI C , et al. A novel approach for multimodal medical image fusion[J]. Expert Systems with Applications an International Journal, 2014, 41 (16): 7425- 7435.
29	XYDEAS C S , PETROVIC V . Objective image fusion performance measure[J]. Military Technical Courier, 2000, 56 (4): 181- 193.
30	PIELLA G, HEIJMANS H. A new quality metric for image fusion[C]//Proc. of the International Conference on Image Processing, 2003.
31	YIN C , BLUM R S . A new automated quality assessment algorithm for image fusion[J]. Image & Vision Computing, 2009, 27 (10): 1421- 1432.

源图像	指标	HMSD	NSCT-PCNN	MST-SR	文献[24]	文献[25]	本文算法
第1组	Q_EN Q_AG Q_W Q_MI	7.321 6 5.260 3 0.900 3 3.490 9	7.294 3 3.681 7 0.847 9 3.444 0	7.334 1 5.281 8 0.898 4 3.446 2	4.593 3 5.223 9 0.907 8 3.561 3	4.818 1 4.817 0 0.901 3 3.572 7	7.364 8 5.167 4 0.909 2 3.368 9
第2组	Q_EN Q_AG Q_W Q_MI	7.410 4 4.495 0 0.844 0 4.499 4	7.501 7 2.791 3 0.815 2 4.204 9	7.406 3 4.542 2 0.838 7 4.403 0	7.508 3 4.558 8 0.846 4 4.498 8	7.524 9 4.139 8 0.834 6 5.080 4	7.513 6 4.611 5 0.876 5 5.428 4
第3组	Q_EN Q_AG Q_W Q_MI	7.123 8 7.183 7 0.863 2 5.204 7	7.106 4 5.728 9 0.883 7 5.521 9	7.112 3 7.138 5 0.896 2 5.916 0	7.106 8 7.161 4 0.870 3 5.861 2	7.119 6 6.981 2 0.869 0 5.321 9	7.150 2 7.116 0 0.899 5 5.609 2

源图像	指标	HMSD	NSCT-PCNN	MST-SR	文献[24]	文献[25]	本文算法
第1组	Q_EN Q_AG Q_W Q_MI	6.274 2 6.802 2 0.582 7 1.747 2	5.908 5 3.559 5 0.570 6 2.074 9	6.893 8 6.115 4 0.761 3 1.408 5	6.878 6 6.511 3 0.808 3 1.939 6	6.259 3 6.666 7 0.866 2 3.495 4	6.496 5 6.166 7 0.812 4 3.565 4
第2组	Q_EN Q_AG Q_W Q_MI	5.235 8 7.926 4 0.844 8 2.334 9	5.172 3 3.559 5 0.734 8 2.210 4	5.312 7 7.463 8 0.893 0 2.646 1	5.288 2 7.421 5 0.892 3 2.562 9	4.819 6 6.510 8 0.764 9 2.146 7	5.618 5 7.150 3 0.898 7 2.710 7
第3组	Q_EN Q_AG Q_W Q_MI	4.934 0 6.039 4 0.815 3 2.504 5	4.663 1 3.165 9 0.824 9 2.332 2	5.141 5 5.313 2 0.812 0 2.162 7	4.866 1 5.210 7 0.819 8 2.233 5	4.418 5 5.281 9 0.811 4 2.529 4	5.321 5 5.863 2 0.902 2 3.234 5

源图像	指标	HMSD	NSCT-PCNN	MST-SR	文献[24]	文献[25]	本文算法
第1组	Q_EN Q_AG Q_W Q_MI	6.863 6 5.252 9 0.774 6 1.522 9	6.264 2 3.535 7 0.459 9 1.124 1	6.954 9 4.498 3 0.682 3 1.247 5	6.902 2 4.679 9 0.620 3 1.234 5	6.646 0 4.419 8 0.579 4 2.035 8	6.976 8 4.070 6 0.805 4 1.373 4
第2组	Q_EN Q_AG Q_W Q_MI	7.309 5 9.247 7 0.726 7 1.938 6	7.013 6 6.448 5 0.749 1 1.748 4	7.378 7 8.484 3 0.535 3 1.543 7	7.370 7 8.686 2 0.454 6 1.706 7	7.220 6 8.001 3 0.362 1 2.830 3	7.385 8 8.004 3 0.846 6 2.195 3
第3组	Q_EN Q_AG Q_W Q_MI	6.563 6 8.673 9 0.753 2 2.520 9	6.484 2 6.060 3 0.741 5 2.603 3	6.836 1 8.070 1 0.665 0 2.707 3	6.811 5 8.153 7 0.910 4 3.007 2	6.516 5 7.647 9 0.868 5 2.801 7	6.886 4 7.894 6 0.896 4 3.270 0

[1]	童莹萍, 全英汇, 冯伟, 邢孟道. 基于空谱信息协同与Gram-Schmidt变换的多源遥感图像融合方法[J]. 系统工程与电子技术, 2022, 44(7): 2074-2083.
[2]	安影, 范训礼, 陈莉, 刘佩. 结合FABEMD和改进的显著性检测的图像融合[J]. 系统工程与电子技术, 2020, 42(2): 292-300.
[3]	徐国明, 袁宏武, 薛模根, 王峰. 全偏振参量低秩稀疏分解伪彩色图像融合[J]. 系统工程与电子技术, 2020, 42(11): 2450-2460.
[4]	宗静静, 邱天爽. 稀疏表示中稀疏系数的 l-1 范数的特性分析[J]. 系统工程与电子技术, 2019, 41(12): 2692-2696.
[5]	刘斌, 付忠旺. 基于四通道不可分提升小波的多聚焦图像融合[J]. 系统工程与电子技术, 2018, 40(2): 463-471.
[6]	常莉红, 冯象初, 张瑞. 四元数小波变换联合稀疏表示的图像融合[J]. 系统工程与电子技术, 2017, 39(7): 1633-1639.
[7]	赵蓝飞, 王爱丽, 王博, 吕鑫淼. 基于子图融合技术的图像增强算法[J]. 系统工程与电子技术, 2017, 39(12): 2840-2848.
[8]	胡晓伟, 童宁宁, 何兴宇, 雷腾, 王宇晨. 基于窄带雷达组网的高速自旋目标成像[J]. 系统工程与电子技术, 2016, 38(4): 792-798.
[9]	周志强，汪渤，李立广，李笋. 基于双边与高斯滤波混合分解的图像融合方法[J]. 系统工程与电子技术, 2016, 38(1): 8-13.
[10]	刘斌, 刘维杰, 魏艳萍. 6通道MSVD构造及其在多聚焦图像融合中的应用[J]. 系统工程与电子技术, 2015, 37(9): 2191-2197.
[11]	邢雅琼, 王晓丹, 梁兵杰, 秦卓. 基于非下采样剪切波变换域方向信息测度的多聚焦图像融合方法[J]. 系统工程与电子技术, 2015, 37(1): 191-198.
[12]	郭明，王书满. 基于区域和方向方差加权信息熵的图像融合[J]. Journal of Systems Engineering and Electronics, 2013, 35(4): 720-724.
[13]	邢雅琼，王晓丹，毕　凯，雷　蕾. 基于NSCT和ICA的红外和可见光图像融合方法[J]. 系统工程与电子技术, 2013, 35(11): 2251-2257.
[14]	王晓文，赵宗贵，汤磊. 一种新的红外与可见光图像融合评价方法[J]. Journal of Systems Engineering and Electronics, 2012, 34(5): 871-875.
[15]	周浦城，王峰，崔逊学，薛模根. 基于逼近理想解排序的图像融合效果评价方法[J]. Journal of Systems Engineering and Electronics, 2011, 33(3): 681-684.