基于图像结构信息的可见光和SAR图像快速配准

doi:10.12305/j.issn.1001-506X.2025.02.10

Abstract

Abstract:

To address the problem of geometric differences, radiometric differences, and speckle noise in the registration process of the images of visible light and synthetic aperture radar (SAR), a fast registration algorithm for optical and SAR images based on image structure information is proposed. Firstly, a Gaussian scale space is established and an image self-similarity orientation map is constructed by using offset mean filtering and bilinear interpolation. Then, characteristic points detection for features from accelerated segment test (FAST) is performed on the maximum and minimum self-similarity maps to obtain corner and edge features. Furthermore, based on the minimum self-similarity index map and equal area strategy, descriptors are constructed, and multi-orientation conversion method and batch generation method for descriptors are proposed. Finally, the nearest-neighbor distance ratio (NNDR) algorithm and the fast sample consensus (FSC) algorithm are performed to identify correct matches. The experimental results show that the proposed algorithm has significant advantages in visible optical and SAR image registration.

Key words: visible optical and synthetic aperture radar (SAR) image registration, structural information, multi-orientation conversion, batch generation

CLC Number:

TN957.52

Leilei JIA, Limin LIU, Jian DONG. Fast registration of optical and SAR images based on image structural information[J]. Systems Engineering and Electronics, 2025, 47(2): 428-441.

Figures/Tables 20

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Fig.9

Fig.10

Fig.11

Fig.12

Fig.13

Fig.14

Fig.15

Fig.16

Fig.17

Fig.18

Fig.19

Table 1

References 43

1	PAUL S , PATI U C . A comprehensive review on remote sensing image registration[J]. International Journal of Remote Sensing, 2021, 42 (14): 5396- 5432. doi: 10.1080/01431161.2021.1906985
2	YE Y Y , ZHANG J C , ZHOU L , et al. Optical and SAR image fusion based on complementary feature decomposition and visual saliency features[J]. IEEE Trans.on Geoscience and Remote Sensing, 2024, 62, 5205315.
3	ZHOU R F , QUAN D , WANG S , et al. A unified deep learning network for remote sensing image registration and change detection[J]. IEEE Trans.on Geoscience and Remote Sensing, 2024, 62, 5101216.
4	MAO Y Q , CHEN K Q , ZHAO L J , et al. Elevation estimation-driven building 3-D reconstruction from single-view remote sensing imagery[J]. IEEE Trans.on Geoscience and Remote Sensing, 2023, 61, 5608718.
5	GAMBRYCH J , GROMEK D , ABRATKIEWICZ K , et al. SAR and orthophoto image registration with simultaneous SAR-based altitude measurement for airborne navigation systems[J]. IEEE Trans.on Geoscience and Remote Sensing, 2023, 61, 5219714.
6	ZITOVA B , FLUSSER J . Image registration methods: a survey[J]. Image and Vision Computing, 2003, 21 (11): 977- 1000. doi: 10.1016/S0262-8856(03)00137-9
7	FENG R T , SHEN H F , BAI J J , et al. Advances and opportunities in remote sensing image geometric registration: a systema-tic review of state-of-the-art approaches and future research directions[J]. IEEE Geoscience and Remote Sensing Magazine, 2021, 9 (4): 120- 142. doi: 10.1109/MGRS.2021.3081763
8	HUGHES L H , SCHMITT M , ZHU X X . Mining hard negative samples for SAR-optical image matching using generative adversarial networks[J]. Remote Sensing, 2018, 10 (10): 1552. doi: 10.3390/rs10101552
9	LIU M , ZHOU G X , MA L F , et al. SIFNet: a self-attention interaction fusion network for multisource satellite imagery template matching[J]. International Journal of Applied Earth Observation and Geoinformation, 2023, 118, 103247. doi: 10.1016/j.jag.2023.103247
10	ZHANG H , LEI L , NI W P , et al. Optical and SAR image matching using pixelwise deep dense features[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19, 6000705.
11	LI H Y , XU F , YANG W , et al. Learning to find the optimal correspondence between SAR and optical image patches[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2023, 16, 9816- 9830. doi: 10.1109/JSTARS.2023.3324768
12	XU W Y , YUAN X H , HU Q W , et al. SAR-optical feature matching: a large-scale patch dataset and a deep local descriptor[J]. International Journal of Applied Earth Observation and Geoinformation, 2023, 122, 103433. doi: 10.1016/j.jag.2023.103433
13	YE Y Y , YANG C , GONG G Q , et al. Robust optical and SAR image matching using attention-enhanced structural features[J]. IEEE Trans.on Geoscience and Remote Sensing, 2024, 62, 5610212.
14	XIANG Y M , JIAO N G , WANG F , et al. A robust two-stage registration algorithm for large optical and SAR images[J]. IEEE Trans.on Geoscience and Remote Sensing, 2022, 60, 5218615.
15	XIONG X , JIN G W , XU Q , et al. Self-similarity features for multimodal remote sensing image matching[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14, 12440- 12454. doi: 10.1109/JSTARS.2021.3131489
16	USS M , VOZEL B , LUKIN V , et al. Efficient discrimination and localization of multimodal remote sensing images using CNN-based prediction of localization uncertainty[J]. Remote Sensing, 2020, 12 (4): 703. doi: 10.3390/rs12040703
17	LIU D, MANSOUR H, BOUFOUNOS P T. Robust mutual information-based multi-image registration[C]//Proc. of the IEEE International Geoscience and Remote Sensing Symposium, 2019: 915-918.
18	PRATT W K . Correlation techniques of image registration[J]. IEEE Trans.on Aerospace and Electronic Systems, 1974, 10 (3): 353- 358.
19	HAST A. Robust and invariant phase based local feature matching[C]//Proc. of the 22nd International Conference on Pattern Recognition, 2014: 809-814.
20	SURI S , REINARTZ P . Mutual-information-based registration of TerraSAR-X and Ikonos imagery in urban areas[J]. IEEE Trans.on Geoscience and Remote Sensing, 2009, 48 (2): 939- 949.
21	叶沅鑫, 单杰, 彭剑威, 等. 利用局部自相似的多光谱遥感图像自动配准[J]. 测绘学报, 2014, 43 (3): 268- 275.
	YE Y X , SHAN J , PENG J W , et al. Automated multispectral remote sensing image registration using local self-similarity[J]. Acta Geodaetica et Cartographica Sinica, 2014, 43 (3): 268- 275.
22	SHECHTMAN E, IRANI M. Matching local self-similarities across images and videos[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2007.
23	YE Y Y , SHEN L , HAO M , et al. Robust optical-to-SAR image matching based on shape properties[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14 (4): 564- 568. doi: 10.1109/LGRS.2017.2660067
24	XIONG X , XU Q , JIN G W , et al. Rank-based local self-similarity descriptor for optical-to-SAR image matching[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 17 (10): 1742- 1746. doi: 10.1109/LGRS.2019.2955153
25	YE Y Y , SHAN J , BRUZZONE L , et al. Robust registration of multimodal remote sensing images based on structural similarity[J]. IEEE Trans.on Geoscience and Remote Sensing, 2017, 55 (5): 2941- 2958. doi: 10.1109/TGRS.2017.2656380
26	YE Y Y , BRUZZONE L , SHAN J , et al. Fast and robust matching for multimodal remote sensing image registration[J]. IEEE Trans.on Geoscience and Remote Sensing, 2019, 57 (11): 9059- 9070. doi: 10.1109/TGRS.2019.2924684
27	XIANG Y M , TAO R S , WAN L , et al. OS-PC: combining feature representation and 3-D phase correlation for subpixel optical and SAR image registration[J]. IEEE Trans.on Geoscience and Remote Sensing, 2020, 58 (9): 6451- 6466. doi: 10.1109/TGRS.2020.2976865
28	FAN J W , WU Y , LI M , et al. SAR and optical image registration using nonlinear diffusion and phase congruency structural descriptor[J]. IEEE Trans.on Geoscience and Remote Sensing, 2018, 56 (9): 5368- 5379. doi: 10.1109/TGRS.2018.2815523
29	XIANG Y M , TAO R S , WANG F , et al. Automatic registration of optical and SAR images via improved phase congruency model[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13, 5847- 5861. doi: 10.1109/JSTARS.2020.3026162
30	XIANG Y M , WANG F , YOU H J . OS-SIFT: a robust SIFT-like algorithm for high-resolution optical-to-SAR image registration in suburban areas[J]. IEEE Trans.on Geoscience and Remote Sensing, 2018, 56 (6): 3078- 3090. doi: 10.1109/TGRS.2018.2790483
31	杨勇, 胡思茹. 基于模板匹配约束下的光学与SAR图像配准[J]. 系统工程与电子技术, 2019, 41 (10): 2235- 2242. doi: 10.3969/j.issn.1001-506X.2019.10.12
	YANG Y , HU S R . Registration of optical and SAR images based on template matching constraints[J]. Systems Engineering and Electronics, 2019, 41 (10): 2235- 2242. doi: 10.3969/j.issn.1001-506X.2019.10.12
32	LOWE D G . Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60 (2): 91- 110. doi: 10.1023/B:VISI.0000029664.99615.94
33	FAN B , HUO C L , PAN C H , et al. Registration of optical and SAR satellite images by exploring the spatial relationship of the improved SIFT[J]. IEEE Geoscience and Remote Sensing Letters, 2013, 10 (4): 657- 661. doi: 10.1109/LGRS.2012.2216500
34	XU C , SUI H G , LI H L , et al. An automatic optical and SAR image registration method with iterative level set segmentation and SIFT[J]. International Journal of Remote Sensing, 2015, 36 (15): 3997- 4017. doi: 10.1080/01431161.2015.1070321
35	MA W P , WEN Z L , WU Y , et al. Remote sensing image regi- stration with modified SIFT and enhanced feature matching[J]. IEEE Geoscience and Remote Sensing Letters, 2016, 14 (1): 3- 7.
36	ZHANG W N . Combination of SIFT and Canny edge detection for registration between SAR and optical images[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19, 4007205.
37	ZHANG X T , WANG Y H , LIU H W . Robust optical and SAR image registration based on OS-SIFT and cascaded sample consensus[J]. IEEE Geoscience and Remote Sensing Letters, 2021, 19, 4011605.
38	叶沅鑫, 慎利, 陈敏, 等. 局部相位特征描述的多源遥感影像自动匹配[J]. 武汉大学学报(信息科学版), 2017, 42 (9): 1278- 1284.
	YE Y X , SHEN L , CHEN M , et al. An automatic matching method based on local phase feature descriptor for multi-source remote sensing images[J]. Geomatics and Information Science of Wuhan University, 2017, 42 (9): 1278- 1284.
39	PAUL S , PATI U C . Automatic optical-to-SAR image registration using a structural descriptor[J]. IET Image Processing, 2020, 14 (1): 62- 73. doi: 10.1049/iet-ipr.2019.0389
40	LI J Y , HU Q W , AI M Y . RIFT: multi-modal image matching based on radiation-variation insensitive feature transform[J]. IEEE Trans.on Image Processing, 2020, 29, 3296- 3310. doi: 10.1109/TIP.2019.2959244
41	ZHANG Y J , YAO Y Y , WAN Y , et al. Histogram of the ori- entation of the weighted phase descriptor for multi-modal remote sensing image matching[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2023, 196, 1- 15. doi: 10.1016/j.isprsjprs.2022.12.018
42	XIONG X , JIN G W , XU Q , et al. Robust registration algorithm for optical and SAR images based on adjacent self-similarity feature[J]. IEEE Trans.on Geoscience and Remote Sensing, 2022, 60, 5233117.
43	LI J Y , XU W Y , SHI P C , et al. LNIFT: locally normalized image for rotation invariant multimodal feature matching[J]. IEEE Trans.on Geoscience and Remote Sensing, 2022, 60, 5621314.

实验组别	评价指标	OS-SIFT	OS-SIFT+CSC	RIFT	OSS	LNIFT	ASS	HOWP	本文算法
1	NCM/TNM	5/19	50/169	34/95	45/135	68/204	78/220	88/190	269/498
	CMR	0.26	0.30	0.36	0.33	0.33	0.35	0.46	0.54
	RMSE	1.52	1.48	1.46	1.42	1.48	1.40	1.38	1.32
	PT	4.95	5.93	3.62	8.55	11.21	7.60	6.09	5.6
2	NCM/TNM	-	5/108	4/10	8/26	40/115	140/296	20/47	189/368
	CMR	-	0.05	0.40	0.31	0.35	0.47	0.43	0.51
	RMSE	-	1.56	1.51	1.45	1.33	1.38	1.34	1.29
	PT	-	25.33	6.93	23.08	16.22	14.52	10.29	9.21
3	NCM/TNM	-	-	-	8/23	64/164	-	-	98/197
	CMR	-	-	-	0.35	0.39	-	-	0.50
	RMSE	-	-	-	1.49	1.42	-	-	1.29
	PT	-	-	-	2.07	4.77	-	-	1.5
4	NCM/TNM	-	14/40	30/72	10/32	97/256	30/76	26/65	197/339
	CMR	-	0.35	0.42	0.31	0.38	0.39	0.40	0.58
	RMSE	-	1.48	1.45	1.39	1.32	1.38	1.31	1.21
	PT	-	1.24	1.26	1.81	4.4	1.36	1.43	1.67
5	NCM/TNM	1/4	5/18	8/23	6/21	62/157	3/21	-	50/110
	CMR	0.25	0.28	0.35	0.29	0.39	0.14	-	0.45
	RMSE	1.56	1.62	1.61	1.58	1.49	1.50	-	1.45
	PT	2.82	1.27	1.33	1.99	4.27	1.19	-	1.15
6	NCM/TNM	-	-	-	-	40/102	-	-	58/133
	CMR	-	-	-	-	0.39	-	-	0.44
	RMSE	-	-	-	-	1.50	-	-	1.36
	PT	-	-	-	-	4.8	-	-	1.60

[1]	Kai XIAO, Guoyao XIAO, Zongzheng SUN, Duopeng WANG, Yinghui QUAN. Design and verification of miniaturized broadband reconnaissance and jamming integrated system [J]. Systems Engineering and Electronics, 2025, 47(1): 22-33.
[2]	Yang MENG, Guoru ZHOU, Jie LI, Bingchen ZHANG. Discriminative sparse microwave imaging method based on structured dictionary learning [J]. Systems Engineering and Electronics, 2025, 47(1): 94-100.
[3]	Fatong ZHANG, Yaowen FU, Wei YANG, Wenpeng ZHANG, Shangqu YAN. Wide-beam motion compensation algorithm for micro-UAV FMCW SAR [J]. Systems Engineering and Electronics, 2024, 46(10): 3303-3311.
[4]	Jin WANG, Xiangguang LENG, Zhongzhen SUN, Xiaojie MA, Yang YANG, Kefeng JI. Study of space/time varying defocus characteristics of complex moving ship targets in SAR imaging [J]. Systems Engineering and Electronics, 2024, 46(7): 2237-2255.
[5]	Zhen LI, Huafeng HE, Tao ZHOU, Xin ZHANG, Xiaofei HAN, Liyuan WANG. DOA estimation method of UCA-FDA radar with distance compensation [J]. Systems Engineering and Electronics, 2024, 46(6): 1967-1974.
[6]	Qi HU, Shaohai HU, Shuaiqi LIU. Ship detection in SAR image based on multi-layer saliency model [J]. Systems Engineering and Electronics, 2024, 46(2): 478-487.
[7]	Yang CHEN, Guoyao XIAO, Yinghui QUAN, Aifeng REN, Bowen BIE, Mengdao XING. Implementation of spaceborne bistatic FMCW SAR imaging algorithm based on multi-core DSP [J]. Systems Engineering and Electronics, 2024, 46(1): 121-129.
[8]	Shengxiang ZHU, Min HE, Zhiyi HE, Jiaxin WANG. Forward-looking imaging method by integrating minimum variance and maximum likelihood spectrum estimation [J]. Systems Engineering and Electronics, 2023, 45(10): 3108-3115.
[9]	Chunhua ZHOU, Weiwei WEI, Xuecheng ZHANG, Xin ZHENG, Mianzhi CHENG. Zero-shot identification for stealth target by inverse synthetic aperture imaging radar [J]. Systems Engineering and Electronics, 2023, 45(10): 3116-3121.
[10]	Hanshen ZHU, Wenhua HU, Baofeng GUO, Liting JIAO, Xiaoxiu ZHU, Chang'an ZHU. Bistatic ISAR sparse aperture maneuvering target MTRC compensation imaging algorithm [J]. Systems Engineering and Electronics, 2023, 45(7): 2022-2030.
[11]	Jingjing ZHU, Shengqi ZHU, Guisheng LIAO, Jingwei XU, Lan LAN, Cao ZENG. Joint target detection based on phased array and frequency diverse array dual-mode radar [J]. Systems Engineering and Electronics, 2023, 45(5): 1342-1350.
[12]	Xiaoxiu ZHU, Limin LIU, Wenhua HU, Baofeng GUO, Lin SHI, Hanshen ZHU. Multi-angle and multi-band ISAR fusion imaging based on GTD model [J]. Systems Engineering and Electronics, 2023, 45(3): 726-735.
[13]	Ruize LI, Shuanghui ZHANG, Yongxiang LIU. Computational efficient structural sparse ISAR imaging method based on convolutional ADMM-net [J]. Systems Engineering and Electronics, 2023, 45(1): 56-70.
[14]	Zhipeng WU, Ping ZHANG, Zhen LI, Lei HUANG, Chang LIU, Shuo GAO. Vegetation height inversion method based on light-weighted and small UAV-radar [J]. Systems Engineering and Electronics, 2022, 44(12): 3667-3675.
[15]	Yanjun LI, Jia LIU, Qiufeng XU. BP autofocus algorithm based on Chebyshev fitting [J]. Systems Engineering and Electronics, 2022, 44(10): 3020-3028.

Fast registration of optical and SAR images based on image structural information

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 20

References 43

Related Articles 15

Recommended Articles

Metrics

Comments