基于多级别融合的可见光和SAR图像联合识别

doi:10.12305/j.issn.1001-506X.2026.03.09

系统工程与电子技术 ›› 2026, Vol. 48 ›› Issue (3): 817-825.doi: 10.12305/j.issn.1001-506X.2026.03.09

• 传感器与信号处理 • 上一篇

基于多级别融合的可见光和SAR图像联合识别

翟佳¹^,²^,*, 画麒嘉³, 王梓权², 张子恺², 王睿琦², 刘金玲², 胡美琪², 武晨辉²

1. 中国传媒大学信息与通信工程学院，北京 100024
2. 散射辐射全国重点实验室，北京 100854
3. 大连理工大学国际信息与软件学院，辽宁大连 116081

收稿日期:2025-02-06 出版日期:2026-03-25 发布日期:2026-04-13
通讯作者: 翟佳
作者简介:画麒嘉（2002—），男，主要研究方向为人工智能、计算机视觉
王梓权（1997—），男，工程师，硕士，主要研究方向为人工智能、计算机视觉、机器学习
张子恺（1996—），男，工程师，硕士，主要研究方向为数据分析、人工智能
王睿琦（1992—），男，工程师，博士，主要研究方向为人工智能、大模型学习、多模态学习
刘金玲（1997—），女，工程师，硕士，主要研究方向为图像处理、计算机视觉
胡美琪（1999—），女，助理工程师，硕士，主要研究方向为计算机网络、人工智能
武晨辉（1999—），男，助理工程师，硕士，主要研究方向为人工智能、计算机视觉

Joint recognition of visible light and SAR images based on multi-level fusion

Jia ZHAI¹^,²^,*, Qijia HUA³, Ziquan WANG², Zikai ZHANG², Ruiqi WANG², Jinling LIU², Meiqi HU², Chenhui WU²

1. School of Information and Communication Engineering，Communication University of China，Beijing 100024，China
2. National Key Laboratory of Scattering and Radiation，Beijing 100854，China
3. International School of Information Science & Engineering，Dalian University of Technology，Dalian 116081，China

Received:2025-02-06 Online:2026-03-25 Published:2026-04-13
Contact: Jia ZHAI

摘要/Abstract

摘要：

为解决合成孔径雷达影像和可见光影像融合过程中存在信息丢失、细节模糊，造成目标检测与识别精度不足的问题，提出一种贯穿像素、特征、决策3个层级的多级图像融合方法。首先，在像素级别上初步融合，提高检测模型对纹理和形态上细节信息的敏感度。之后，在特征级别上进一步融合，使模型更关注全局的语义信息，捕捉复杂背景下的目标特征。最后，在决策级别上整合融合信息并筛选出具有高可信度和精确度的检测结果。模型在自制目标识别数据集上进行了对比实验和消融实验，实验结果验证了其有效性。

关键词: 多级别融合, 联合识别, 合成孔径雷达, 可见光

Abstract:

To address the issues of information loss and detail blurring in the fusion process of synthetic aperture radar （SAR） and visible light images, which lead to insufficient accuracy in target detection and recognition, a multi-level image fusion method spanning pixel, feature, and decision levels is proposed. Firstly, perform preliminary fusion at the pixel level to enhance the sensitivity of the detection model to texture and morphological details. Then, conduct further fusion at the feature level to make the model focus more on global semantic information and capture target features in complex backgrounds. Finally, integrate and fuse the information at the decision level, selecting detection results with high reliability and accuracy. The model is compared and ablated experiments on a self-constructed target recognition dataset, with experimental results confirming its effectiveness.

Key words: multi-level fusion, joint recognition, synthetic aperture radar (SAR), visible light

中图分类号:

TP 391.4

翟佳, 画麒嘉, 王梓权, 张子恺, 王睿琦, 刘金玲, 胡美琪, 武晨辉. 基于多级别融合的可见光和SAR图像联合识别[J]. 系统工程与电子技术, 2026, 48(3): 817-825.

Jia ZHAI, Qijia HUA, Ziquan WANG, Zikai ZHANG, Ruiqi WANG, Jinling LIU, Meiqi HU, Chenhui WU. Joint recognition of visible light and SAR images based on multi-level fusion[J]. Systems Engineering and Electronics, 2026, 48(3): 817-825.

图/表 11

图1

图2

图3

图4

图5

表1

表2

图6

表3

图7

表4

参考文献 31

1	GHAMISI P, RASTI B, YOKOYA N, et al. Multisource and multitemporal data fusion in remote sensing: a comprehensive review of the state of the art[J]. IEEE Geoscience and Remote Sensing Magazine, 2019, 7 (1): 6- 39. doi: 10.1109/MGRS.2018.2890023
2	HAME T, SIRRO L, KILPI J, et al. A hierarchical clustering method for land cover change detection and identification[J]. Remote Sensing, 2020, 12 (11): 1751. doi: 10.3390/rs12111751
3	TSOKAS A, RYSZ M, PARDALOS P M, et al. SAR data applications in earth observation: an overview[J]. Expert Systems with Applications, 2022, 205, 117342. doi: 10.1016/j.eswa.2022.117342
4	ZHU X X, MONTAZERI S, ALI M, et al. Deep learning meets SAR: concepts, models, pitfalls, and perspectives[J]. IEEE Geoscience and Remote Sensing Magazine, 2021, 9 (4): 143- 172.
5	FREEMAN A. SAR calibration: an overview[J]. IEEE Trans. on Geoscience and Remote Sensing, 1992, 30 (6): 1107- 1121.
6	MIAO Z H, ZHU L J, ZHAO C C, et al. Image enhancement of low-light parking space based on Retinex[J]. Automotive Engineering, 2023, 45 (6): 989- 996.
7	AMARSAIKHAN D, BLOTEVOGEL H H, GENDEREN J L, et al. Fusing high-resolution SAR and optical imagery for improved urban land cover study and classification[J]. International Journal of Image and Data Fusion, 2010, 1 (1): 83- 97. doi: 10.1080/19479830903562041
8	GUO Q, LIU J N, KALIUZHNYI M. YOLOX-SAR: high-precision object detection system based on visible and infrared sensors for SAR remote sensing[J]. IEEE Sensors Journal, 2022, 22 (17): 17243- 17253. doi: 10.1109/JSEN.2022.3186889
9	BHAT S, KOUNDADL D. Multi-focus image fusion using neutrosophic based wavelet transform[J]. Applied Soft Computing, 2021, 106, 107307. doi: 10.1016/j.asoc.2021.107307
10	萧明伟, 李素敏, 李雁. 基于区域性IHS_NSST的国产高分合成孔径雷达和光学图像融合[J]. 激光与光电子学进展, 2020, 57(18): 181013.
	XIAO M W, LI S M, LI Y. Fusion of domestic high resolution synthetic aperture radar and optical images based on regional IHS_NSST[J]. Laser & Optoelectronics Progress, 2020, 57(18): 181013.
11	SHAH E, JAYAPRASAD P, JAMES M E. Image fusion of SAR and optical images for identifying antarctic ice features[J]. Journal of the Indian Society of Remote Sensing, 2019, 47, 2113- 2127. doi: 10.1007/s12524-019-01040-3
12	KARAKAYA D, ULUCAN O, TURKAN M. PAS-MEF: multi-exposure image fusion based on principal component analysis, adaptive well-exposedness and saliency map[C]//Proc. of the IEEE International Conference on Acoustics, Speech and Signal Processing, 2022: 2345−2349.
13	CHEN Y X, BRUZZONE L. Self-supervised SAR-optical data fusion of sentinel-1/-2 images[J]. IEEE Trans. on Geoscience and Remote Sensing, 2022, 60, 5406011. doi: 10.1109/tgrs.2021.3128072
14	LI J, LI C C, XU W M, et al. Fusion of optical and SAR images based on deep learning to reconstruct vegetation NDVI time series in cloud-prone regions[J]. International Journal of Applied Earth Observation and Geoinformation, 2022, 112, 102818. doi: 10.1016/j.jag.2022.102818
15	ZHOU K X, ZHANG M, WANG H, et al. Ship detection in SAR images based on multi-scale feature extraction and adaptive feature fusion[J]. Remote Sensing, 2022, 14 (3): 755. doi: 10.3390/rs14030755
16	ZHANG H, XU H, TIAN X, et al. Image fusion meets deep learning: a survey and perspective[J]. Information Fusion, 2021, 76, 323- 336. doi: 10.1016/j.inffus.2021.06.008
17	WANG Z B, MA Y K, ZHANG Y N. Review of pixel-level remote sensing image fusion based on deep learning[J]. Information Fusion, 2023, 90, 36- 58. doi: 10.1016/j.inffus.2022.09.008
18	JIANG C C, REN H Z, YANG H, et al. M2FNet: multi-modal fusion network for object detection from visible and thermal infrared images[J]. International Journal of Applied Earth Observation and Geoinformation, 2024, 130, 103918. doi: 10.1016/j.jag.2024.103918
19	DENG B, LV H. Research on image fusion method of SAR and visible image based on CNN[C]//Proc. of the IEEE 4th International Conference on Civil Aviation Safety and Information Technology, 2022: 1400−1403.
20	WANG J P, SU N, ZHAO C H, et al. Multi-modal object detection method based on dual-branch asymmetric attention backbone and feature fusion pyramid network[J]. Remote Sensing, 2024, 16 (20): 3904. doi: 10.3390/rs16203904
21	SONG B Z, LIU P, LI J. MLFF-GAN: a multilevel feature fusion with GAN for spatiotemporal remote sensing images[J]. IEEE Trans. on Geoscience and Remote Sensing, 2022, 60, 4410816. doi: 10.1109/tgrs.2022.3169916
22	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770−778.
23	LIN T Y, DOLLAR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//Proc. of the IEEE Conference on Computer Vision And Pattern Recognition, 2017: 2117−2125.
24	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal network[J]. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2017, 39 (6): 1137- 1149. doi: 10.1109/TPAMI.2016.2577031
25	ZHANG J. RPN 2: on interdependence function learning towards unifying and advancing CNN, RNN, GNN, and transformer[EB/OL]. [2025-01-10]. https://arxiv.org/pdf/2411.11162.
26	GIRSHICK R. Fast R-CNN[C]//Proc. of the IEEE International Conference on Computer Vision, 2015.
27	HUANG M Y, XU Y, QIAN L X, et al. The QXS-SAROPT dataset for deep learning in SAR-optical data fusion[EB/OL]. [2025-01-10]. https://arxiv.org/abs/2103.08259.
28	郝永平, 曹昭睿, 白帆, 等. 基于兴趣区域掩码卷积神经网络的红外-可见光图像融合与目标识别算法研究[J]. 光子学报, 2021, 50 (2): 76- 90. doi: 10.3788/gzxb20215002.0210002
	HAO Y P, CAO Z R, BAI F, et al. Research on infrared visible image fusion and target recognition algorithm based on region of interest mask convolution neural network[J]. Acta Photonica Sinica, 2021, 50 (2): 76- 90. doi: 10.3788/gzxb20215002.0210002
29	邝楚文, 何望. 基于红外与可见光图像的目标检测算法[J]. 红外技术, 2022, 44 (9): 912- 919.
	KUANG C W, HE W. Object detection algorithm based on infrared and visible light images[J]. Infrared Technology, 2022, 44 (9): 912- 919.
30	赵松璞, 杨利萍, 赵昕, 等. 基于自适应注意力机制的红外与可见光图像目标检测[J]. 红外技术, 2024, 46 (4): 443- 451.
	ZHAO S P, YANG L P, ZHAO X, et al. Object detection in visible light and infrared images based on adaptive attention mechanism[J]. Infrared Technology, 2024, 46 (4): 443- 451.
31	何南, 石书桦. 融合红外特征的可见光图像目标检测算法研究[J]. 微电子学与计算机, 2023, 40 (3): 29- 36. doi: 10.19304/J.ISSN1000-7180.2022.0388
	HE N, SHI S H. Research on visible image object detection algorithm based on infrared features fusion[J]. Microelectronics & Computer, 2023, 40 (3): 29- 36. doi: 10.19304/J.ISSN1000-7180.2022.0388

类别	物体个数
船	672
车	988
飞机	468

类别	物体个数
岸边	328
陆地	556
海面	244

方法	车		船		飞机
方法	漏警率	精确率	漏警率	精确率	漏警率	精确率
文献[28]	12.4	95.7	12.2	96.3	14.3	96.6
文献[29]	13.1	96.1	12.7	97.0	13.4	97.7
文献[30]	12.1	95.4	14.4	96.2	12.8	97.1
文献[31]	12.6	96.4	13.1	96.9	12.5	97.4
本文方法	11.2	97.1	11.3	97.8	11.5	98.9

实验配置	最大值	最小值	平均值
去除像素级	0.910	0.845	0.887
去除特征级	0.915	0.856	0.892
去除决策级	0.928	0.831	0.905
完整模型	0.935	0.869	0.915

[1]	杨雪莹, 李高鹏, 张云. 语义引导的星载多角度SAR三维重建方法[J]. 系统工程与电子技术, 2026, 48(2): 430-446.
[2]	刘梦莉, 舒高峰, 李宁. 基于多干扰机协同调制的多通道SAR-GMTI欺骗干扰方法[J]. 系统工程与电子技术, 2026, 48(2): 490-502.
[3]	胡继军, 韩伟, 金龙, 周希娃, 张国玉, 周钦月. 雷达导引头的多站组网单比特干扰对抗方法[J]. 系统工程与电子技术, 2026, 48(2): 503-514.
[4]	陈海青, 汪刘应, 刘顾, 王龙, 葛超群, 陈孟州. 合成孔径雷达图像去噪算法研究进展[J]. 系统工程与电子技术, 2026, 48(1): 94-105.
[5]	张彬, 许高添, 张廷豪, 李志辉, 何宏强. 分布式合成孔径雷达前视高分辨成像算法[J]. 系统工程与电子技术, 2026, 48(1): 119-131.
[6]	蒋明煜, 张顺生, 肖思瑶. 面向轻量级交叉注意力卷积网络的SAR目标识别[J]. 系统工程与电子技术, 2025, 47(9): 2853-2861.
[7]	李响, 曾顶, 殷君君, 国贤玉, 杨健. 基于梯度融合的极化SAR图像引导滤波[J]. 系统工程与电子技术, 2025, 47(9): 2890-2904.
[8]	朱国辉, 杨琳, 汪洋, 郑陶冶. 一种高速机动平台SAR成像PRF改进设计方法[J]. 系统工程与电子技术, 2025, 47(9): 2905-2912.
[9]	付卫红, 彭文洪, 刘乃安. 混合注意力优化的SAR图像小目标检测方法[J]. 系统工程与电子技术, 2025, 47(8): 2519-2526.
[10]	倪康, 贾文杰, 邹旻瑞, 郑志忠. 基于动态聚合网络的SAR目标检测[J]. 系统工程与电子技术, 2025, 47(8): 2527-2539.
[11]	黄丰卓, 冯东, 华洋晟, 黄晓涛. 一种改进的全息SAR相位梯度自聚焦算法[J]. 系统工程与电子技术, 2025, 47(6): 1786-1795.
[12]	张新征, 闫梦可, 朱晓林. 噪声伪标签容忍的半监督SAR目标识别[J]. 系统工程与电子技术, 2025, 47(6): 1796-1805.
[13]	贾竣皓, 陈学斌, 李璋峰, 叶春茂. 空间群目标ISAR成像与检测方法[J]. 系统工程与电子技术, 2025, 47(6): 1843-1854.
[14]	刘宁, 李芳芳, 李新武, 洪文. 结合足迹和相位信息的SAR高层建筑三维重建[J]. 系统工程与电子技术, 2025, 47(5): 1469-1486.
[15]	贾钰嘉, 张思乾, 唐涛, 匡纲要. 强化散射特征的机载SAR实传图像盲超分辨重建[J]. 系统工程与电子技术, 2025, 47(3): 753-767.

基于多级别融合的可见光和SAR图像联合识别

Joint recognition of visible light and SAR images based on multi-level fusion

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献 31

相关文章 15

编辑推荐

Metrics

本文评价