基于图像结构信息的可见光和SAR图像快速配准

doi:10.12305/j.issn.1001-506X.2025.02.10

摘要/Abstract

摘要：

针对可见光和合成孔径雷达(synthetic aperture radar, SAR)图像配准过程中的几何差异、辐射差异和斑点噪声问题, 提出一种基于图像结构信息的可见光和SAR图像快速配准算法。首先, 建立高斯尺度空间, 利用偏移均值滤波和双线性插值建立图像自相似性方向图; 然后, 在最大和最小自相似性图上进行加速分段特征测试(features from accelerated segment test, FAST)特征点检测, 获取角点和边缘特征; 再者, 基于最小自相似性索引图和等面积策略构建描述子, 并提出描述子多方向转换方法和批量生成方法; 最后, 利用最邻近距离比算法和快速抽样一致性算法识别正确匹配。实验结果表明, 所提算法在可见光和SAR图像配准方面具有明显优势。

关键词: 可见光和合成孔径雷达图像配准, 结构信息, 多方向转换, 批量生成

Abstract:

To address the problem of geometric differences, radiometric differences, and speckle noise in the registration process of the images of visible light and synthetic aperture radar (SAR), a fast registration algorithm for optical and SAR images based on image structure information is proposed. Firstly, a Gaussian scale space is established and an image self-similarity orientation map is constructed by using offset mean filtering and bilinear interpolation. Then, characteristic points detection for features from accelerated segment test (FAST) is performed on the maximum and minimum self-similarity maps to obtain corner and edge features. Furthermore, based on the minimum self-similarity index map and equal area strategy, descriptors are constructed, and multi-orientation conversion method and batch generation method for descriptors are proposed. Finally, the nearest-neighbor distance ratio (NNDR) algorithm and the fast sample consensus (FSC) algorithm are performed to identify correct matches. The experimental results show that the proposed algorithm has significant advantages in visible optical and SAR image registration.

Key words: visible optical and synthetic aperture radar (SAR) image registration, structural information, multi-orientation conversion, batch generation

中图分类号:

TN957.52

贾蕾蕾, 刘利民, 董健. 基于图像结构信息的可见光和SAR图像快速配准[J]. 系统工程与电子技术, 2025, 47(2): 428-441.

Leilei JIA, Limin LIU, Jian DONG. Fast registration of optical and SAR images based on image structural information[J]. Systems Engineering and Electronics, 2025, 47(2): 428-441.

图/表 20

图1

图2

图3

图4

图5

图6

图7

图8

图9

图10

图11

图12

图13

图14

图15

图16

图17

图18

图19

表1

参考文献 43

1	PAUL S , PATI U C . A comprehensive review on remote sensing image registration[J]. International Journal of Remote Sensing, 2021, 42 (14): 5396- 5432. doi: 10.1080/01431161.2021.1906985
2	YE Y Y , ZHANG J C , ZHOU L , et al. Optical and SAR image fusion based on complementary feature decomposition and visual saliency features[J]. IEEE Trans.on Geoscience and Remote Sensing, 2024, 62, 5205315.
3	ZHOU R F , QUAN D , WANG S , et al. A unified deep learning network for remote sensing image registration and change detection[J]. IEEE Trans.on Geoscience and Remote Sensing, 2024, 62, 5101216.
4	MAO Y Q , CHEN K Q , ZHAO L J , et al. Elevation estimation-driven building 3-D reconstruction from single-view remote sensing imagery[J]. IEEE Trans.on Geoscience and Remote Sensing, 2023, 61, 5608718.
5	GAMBRYCH J , GROMEK D , ABRATKIEWICZ K , et al. SAR and orthophoto image registration with simultaneous SAR-based altitude measurement for airborne navigation systems[J]. IEEE Trans.on Geoscience and Remote Sensing, 2023, 61, 5219714.
6	ZITOVA B , FLUSSER J . Image registration methods: a survey[J]. Image and Vision Computing, 2003, 21 (11): 977- 1000. doi: 10.1016/S0262-8856(03)00137-9
7	FENG R T , SHEN H F , BAI J J , et al. Advances and opportunities in remote sensing image geometric registration: a systema-tic review of state-of-the-art approaches and future research directions[J]. IEEE Geoscience and Remote Sensing Magazine, 2021, 9 (4): 120- 142. doi: 10.1109/MGRS.2021.3081763
8	HUGHES L H , SCHMITT M , ZHU X X . Mining hard negative samples for SAR-optical image matching using generative adversarial networks[J]. Remote Sensing, 2018, 10 (10): 1552. doi: 10.3390/rs10101552
9	LIU M , ZHOU G X , MA L F , et al. SIFNet: a self-attention interaction fusion network for multisource satellite imagery template matching[J]. International Journal of Applied Earth Observation and Geoinformation, 2023, 118, 103247. doi: 10.1016/j.jag.2023.103247
10	ZHANG H , LEI L , NI W P , et al. Optical and SAR image matching using pixelwise deep dense features[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19, 6000705.
11	LI H Y , XU F , YANG W , et al. Learning to find the optimal correspondence between SAR and optical image patches[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2023, 16, 9816- 9830. doi: 10.1109/JSTARS.2023.3324768
12	XU W Y , YUAN X H , HU Q W , et al. SAR-optical feature matching: a large-scale patch dataset and a deep local descriptor[J]. International Journal of Applied Earth Observation and Geoinformation, 2023, 122, 103433. doi: 10.1016/j.jag.2023.103433
13	YE Y Y , YANG C , GONG G Q , et al. Robust optical and SAR image matching using attention-enhanced structural features[J]. IEEE Trans.on Geoscience and Remote Sensing, 2024, 62, 5610212.
14	XIANG Y M , JIAO N G , WANG F , et al. A robust two-stage registration algorithm for large optical and SAR images[J]. IEEE Trans.on Geoscience and Remote Sensing, 2022, 60, 5218615.
15	XIONG X , JIN G W , XU Q , et al. Self-similarity features for multimodal remote sensing image matching[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14, 12440- 12454. doi: 10.1109/JSTARS.2021.3131489
16	USS M , VOZEL B , LUKIN V , et al. Efficient discrimination and localization of multimodal remote sensing images using CNN-based prediction of localization uncertainty[J]. Remote Sensing, 2020, 12 (4): 703. doi: 10.3390/rs12040703
17	LIU D, MANSOUR H, BOUFOUNOS P T. Robust mutual information-based multi-image registration[C]//Proc. of the IEEE International Geoscience and Remote Sensing Symposium, 2019: 915-918.
18	PRATT W K . Correlation techniques of image registration[J]. IEEE Trans.on Aerospace and Electronic Systems, 1974, 10 (3): 353- 358.
19	HAST A. Robust and invariant phase based local feature matching[C]//Proc. of the 22nd International Conference on Pattern Recognition, 2014: 809-814.
20	SURI S , REINARTZ P . Mutual-information-based registration of TerraSAR-X and Ikonos imagery in urban areas[J]. IEEE Trans.on Geoscience and Remote Sensing, 2009, 48 (2): 939- 949.
21	叶沅鑫, 单杰, 彭剑威, 等. 利用局部自相似的多光谱遥感图像自动配准[J]. 测绘学报, 2014, 43 (3): 268- 275.
	YE Y X , SHAN J , PENG J W , et al. Automated multispectral remote sensing image registration using local self-similarity[J]. Acta Geodaetica et Cartographica Sinica, 2014, 43 (3): 268- 275.
22	SHECHTMAN E, IRANI M. Matching local self-similarities across images and videos[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2007.
23	YE Y Y , SHEN L , HAO M , et al. Robust optical-to-SAR image matching based on shape properties[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14 (4): 564- 568. doi: 10.1109/LGRS.2017.2660067
24	XIONG X , XU Q , JIN G W , et al. Rank-based local self-similarity descriptor for optical-to-SAR image matching[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 17 (10): 1742- 1746. doi: 10.1109/LGRS.2019.2955153
25	YE Y Y , SHAN J , BRUZZONE L , et al. Robust registration of multimodal remote sensing images based on structural similarity[J]. IEEE Trans.on Geoscience and Remote Sensing, 2017, 55 (5): 2941- 2958. doi: 10.1109/TGRS.2017.2656380
26	YE Y Y , BRUZZONE L , SHAN J , et al. Fast and robust matching for multimodal remote sensing image registration[J]. IEEE Trans.on Geoscience and Remote Sensing, 2019, 57 (11): 9059- 9070. doi: 10.1109/TGRS.2019.2924684
27	XIANG Y M , TAO R S , WAN L , et al. OS-PC: combining feature representation and 3-D phase correlation for subpixel optical and SAR image registration[J]. IEEE Trans.on Geoscience and Remote Sensing, 2020, 58 (9): 6451- 6466. doi: 10.1109/TGRS.2020.2976865
28	FAN J W , WU Y , LI M , et al. SAR and optical image registration using nonlinear diffusion and phase congruency structural descriptor[J]. IEEE Trans.on Geoscience and Remote Sensing, 2018, 56 (9): 5368- 5379. doi: 10.1109/TGRS.2018.2815523
29	XIANG Y M , TAO R S , WANG F , et al. Automatic registration of optical and SAR images via improved phase congruency model[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13, 5847- 5861. doi: 10.1109/JSTARS.2020.3026162
30	XIANG Y M , WANG F , YOU H J . OS-SIFT: a robust SIFT-like algorithm for high-resolution optical-to-SAR image registration in suburban areas[J]. IEEE Trans.on Geoscience and Remote Sensing, 2018, 56 (6): 3078- 3090. doi: 10.1109/TGRS.2018.2790483
31	杨勇, 胡思茹. 基于模板匹配约束下的光学与SAR图像配准[J]. 系统工程与电子技术, 2019, 41 (10): 2235- 2242. doi: 10.3969/j.issn.1001-506X.2019.10.12
	YANG Y , HU S R . Registration of optical and SAR images based on template matching constraints[J]. Systems Engineering and Electronics, 2019, 41 (10): 2235- 2242. doi: 10.3969/j.issn.1001-506X.2019.10.12
32	LOWE D G . Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60 (2): 91- 110. doi: 10.1023/B:VISI.0000029664.99615.94
33	FAN B , HUO C L , PAN C H , et al. Registration of optical and SAR satellite images by exploring the spatial relationship of the improved SIFT[J]. IEEE Geoscience and Remote Sensing Letters, 2013, 10 (4): 657- 661. doi: 10.1109/LGRS.2012.2216500
34	XU C , SUI H G , LI H L , et al. An automatic optical and SAR image registration method with iterative level set segmentation and SIFT[J]. International Journal of Remote Sensing, 2015, 36 (15): 3997- 4017. doi: 10.1080/01431161.2015.1070321
35	MA W P , WEN Z L , WU Y , et al. Remote sensing image regi- stration with modified SIFT and enhanced feature matching[J]. IEEE Geoscience and Remote Sensing Letters, 2016, 14 (1): 3- 7.
36	ZHANG W N . Combination of SIFT and Canny edge detection for registration between SAR and optical images[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19, 4007205.
37	ZHANG X T , WANG Y H , LIU H W . Robust optical and SAR image registration based on OS-SIFT and cascaded sample consensus[J]. IEEE Geoscience and Remote Sensing Letters, 2021, 19, 4011605.
38	叶沅鑫, 慎利, 陈敏, 等. 局部相位特征描述的多源遥感影像自动匹配[J]. 武汉大学学报(信息科学版), 2017, 42 (9): 1278- 1284.
	YE Y X , SHEN L , CHEN M , et al. An automatic matching method based on local phase feature descriptor for multi-source remote sensing images[J]. Geomatics and Information Science of Wuhan University, 2017, 42 (9): 1278- 1284.
39	PAUL S , PATI U C . Automatic optical-to-SAR image registration using a structural descriptor[J]. IET Image Processing, 2020, 14 (1): 62- 73. doi: 10.1049/iet-ipr.2019.0389
40	LI J Y , HU Q W , AI M Y . RIFT: multi-modal image matching based on radiation-variation insensitive feature transform[J]. IEEE Trans.on Image Processing, 2020, 29, 3296- 3310. doi: 10.1109/TIP.2019.2959244
41	ZHANG Y J , YAO Y Y , WAN Y , et al. Histogram of the ori- entation of the weighted phase descriptor for multi-modal remote sensing image matching[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2023, 196, 1- 15. doi: 10.1016/j.isprsjprs.2022.12.018
42	XIONG X , JIN G W , XU Q , et al. Robust registration algorithm for optical and SAR images based on adjacent self-similarity feature[J]. IEEE Trans.on Geoscience and Remote Sensing, 2022, 60, 5233117.
43	LI J Y , XU W Y , SHI P C , et al. LNIFT: locally normalized image for rotation invariant multimodal feature matching[J]. IEEE Trans.on Geoscience and Remote Sensing, 2022, 60, 5621314.

实验组别	评价指标	OS-SIFT	OS-SIFT+CSC	RIFT	OSS	LNIFT	ASS	HOWP	本文算法
1	NCM/TNM	5/19	50/169	34/95	45/135	68/204	78/220	88/190	269/498
	CMR	0.26	0.30	0.36	0.33	0.33	0.35	0.46	0.54
	RMSE	1.52	1.48	1.46	1.42	1.48	1.40	1.38	1.32
	PT	4.95	5.93	3.62	8.55	11.21	7.60	6.09	5.6
2	NCM/TNM	-	5/108	4/10	8/26	40/115	140/296	20/47	189/368
	CMR	-	0.05	0.40	0.31	0.35	0.47	0.43	0.51
	RMSE	-	1.56	1.51	1.45	1.33	1.38	1.34	1.29
	PT	-	25.33	6.93	23.08	16.22	14.52	10.29	9.21
3	NCM/TNM	-	-	-	8/23	64/164	-	-	98/197
	CMR	-	-	-	0.35	0.39	-	-	0.50
	RMSE	-	-	-	1.49	1.42	-	-	1.29
	PT	-	-	-	2.07	4.77	-	-	1.5
4	NCM/TNM	-	14/40	30/72	10/32	97/256	30/76	26/65	197/339
	CMR	-	0.35	0.42	0.31	0.38	0.39	0.40	0.58
	RMSE	-	1.48	1.45	1.39	1.32	1.38	1.31	1.21
	PT	-	1.24	1.26	1.81	4.4	1.36	1.43	1.67
5	NCM/TNM	1/4	5/18	8/23	6/21	62/157	3/21	-	50/110
	CMR	0.25	0.28	0.35	0.29	0.39	0.14	-	0.45
	RMSE	1.56	1.62	1.61	1.58	1.49	1.50	-	1.45
	PT	2.82	1.27	1.33	1.99	4.27	1.19	-	1.15
6	NCM/TNM	-	-	-	-	40/102	-	-	58/133
	CMR	-	-	-	-	0.39	-	-	0.44
	RMSE	-	-	-	-	1.50	-	-	1.36
	PT	-	-	-	-	4.8	-	-	1.60

[1]	肖凯, 肖国尧, 孙宗正, 王铎鹏, 全英汇. 小型化宽带侦察干扰一体化系统的设计与验证[J]. 系统工程与电子技术, 2025, 47(1): 22-33.
[2]	孟洋, 周国如, 李洁, 张冰尘. 基于结构化字典学习的判别稀疏微波成像方法[J]. 系统工程与电子技术, 2025, 47(1): 94-100.
[3]	张法桐, 付耀文, 杨威, 张文鹏, 颜上取. 微小型无人机载FMCW SAR宽波束运动补偿算法[J]. 系统工程与电子技术, 2024, 46(10): 3303-3311.
[4]	王进, 冷祥光, 孙忠镇, 马晓杰, 杨阳, 计科峰. 复杂运动舰船目标SAR成像空/时变散焦特性研究[J]. 系统工程与电子技术, 2024, 46(7): 2237-2255.
[5]	李震, 何华锋, 周涛, 张鑫, 韩晓斐, 王栗沅. 基于UCA-FDA雷达距离补偿的DOA估计方法[J]. 系统工程与电子技术, 2024, 46(6): 1967-1974.
[6]	扈琪, 胡绍海, 刘帅奇. 基于多层显著性模型的SAR图像舰船目标检测[J]. 系统工程与电子技术, 2024, 46(2): 478-487.
[7]	陈洋, 肖国尧, 全英汇, 任爱锋, 别博文, 邢孟道. 基于多核DSP的星载双基FMCW SAR成像算法实现[J]. 系统工程与电子技术, 2024, 46(1): 121-129.
[8]	祝昇翔, 何岷, 贺志毅, 王嘉欣. 联合最小方差与最大似然谱估计的前视成像方法[J]. 系统工程与电子技术, 2023, 45(10): 3108-3115.
[9]	周春花, 魏维伟, 张学成, 郑鑫, 程冕之. 逆合成孔径成像雷达隐身目标零样本识别[J]. 系统工程与电子技术, 2023, 45(10): 3116-3121.
[10]	朱瀚神, 胡文华, 郭宝锋, 焦丽婷, 朱晓秀, 朱常安. 双基地ISAR稀疏孔径机动目标MTRC补偿成像算法[J]. 系统工程与电子技术, 2023, 45(7): 2022-2030.
[11]	朱晶晶, 朱圣棋, 廖桂生, 许京伟, 兰岚, 曾操. 相控阵和频率分集阵双模式雷达联合目标检测[J]. 系统工程与电子技术, 2023, 45(5): 1342-1350.
[12]	朱晓秀, 刘利民, 胡文华, 郭宝锋, 史林, 朱瀚神. 基于GTD模型的多视角多频带ISAR融合成像[J]. 系统工程与电子技术, 2023, 45(3): 726-735.
[13]	李瑞泽, 张双辉, 刘永祥. 基于卷积ADMM网络的高效结构化稀疏ISAR成像方法[J]. 系统工程与电子技术, 2023, 45(1): 56-70.
[14]	吴志鹏, 张平, 李震, 黄磊, 刘畅, 高硕. 基于轻小型无人机雷达的植被高度反演方法[J]. 系统工程与电子技术, 2022, 44(12): 3667-3675.
[15]	李彦君, 刘佳, 徐秋锋. 基于切比雪夫拟合的BP自聚焦算法[J]. 系统工程与电子技术, 2022, 44(10): 3020-3028.