基于风格迁移不变特征的SAR与光学图像配准算法

doi:10.12305/j.issn.1001-506X.2022.05.14

系统工程与电子技术 ›› 2022, Vol. 44 ›› Issue (5): 1536-1542.doi: 10.12305/j.issn.1001-506X.2022.05.14

基于风格迁移不变特征的SAR与光学图像配准算法

陈世伟, 夏海*, 杨小冈, 李小锋

火箭军工程大学导弹工程学院, 陕西西安 710025

收稿日期:2021-03-19 出版日期:2022-05-01 发布日期:2022-05-16
通讯作者: 夏海
作者简介:陈世伟(1979—), 男, 副教授, 博士, 主要研究方向为精确制导技术、遥感图像处理与分析|夏海(1997—), 男, 硕士研究生, 主要研究方向为图像配准技术|杨小冈(1978—), 男, 教授, 博士, 主要研究方向为机器视觉、精确制导|李小锋(1983—), 男, 讲师, 博士, 主要研究方向为精确制导技术、机器视觉
基金资助:
国家自然科学基金(61806209)

SAR and optical image registration algorithm based on style transfer invariable features

Shiwei CHEN, Hai XIA*, Xiaogang YANG, Xiaofeng LI

Missile Engineering Institute, Rocket Force University of Engineering, Xi'an 710025, China

Received:2021-03-19 Online:2022-05-01 Published:2022-05-16
Contact: Hai XIA

摘要/Abstract

摘要：

针对异源遥感图像的匹配难题, 提出一种基于风格迁移不变特征的合成孔径雷达(synthetic aperture radar, SAR)图像与光学图像配准算法。首先, 训练SAR图像转换为光学图像的风格迁移网络。然后, 基于风格迁移网络生成人工光学图像及其与原SAR图像之间的差异图, 并利用小波多尺度特性增强人工光学图像和差异图的边缘区域, 二值分割后提取人工光学图像的边缘不变特征。同时, 提取光学基准图像的边缘特征。最后, 通过互相关性准则进行边缘特征匹配, 进而实现原始SAR图像与光学基准图像的精确配准。实验结果表明, 较同类算法, 即使在训练样本不足的条件下, 生成的人工光学图像也能与光学基准图像实现精确配准, 增强了算法的适应性。

关键词: 异源图像匹配, 图像风格迁移, 差异图, 边缘不变特征

Abstract:

According to the matching difficulties of multi-sensor remote sensing images, a new synthetic aperture radar (SAR) and optical image registration approach based on image style transfer invariable features is proposed. First, the style transfer neural network for SAR image conversion to optical image is trained. Then, the artificial optical image and the difference map between the artificial optical image and the original SAR image are generated based on the style transfer neural network, and the edge regions of the artificial optical image and the difference image are enhanced by using the multi-scale characteristics of wavelet. After binary segmentation, the stable edge features of the artificial optical image. And the edge features of the optical reference image are also extracted. Finally, the edge features are matched by the cross-correlation criterion to achieve the accurate registration between the original SAR image and the optical reference image. The experiments demonstrate that the artificial optical image can be accurately registered with the optical reference image even under the condition of insufficient training samples. This enhances the adaptability of the algorithm.

Key words: multi-sensor images matching, image style transfer, difference image, stable edge features

中图分类号:

TP394.1

陈世伟, 夏海, 杨小冈, 李小锋. 基于风格迁移不变特征的SAR与光学图像配准算法[J]. 系统工程与电子技术, 2022, 44(5): 1536-1542.

Shiwei CHEN, Hai XIA, Xiaogang YANG, Xiaofeng LI. SAR and optical image registration algorithm based on style transfer invariable features[J]. Systems Engineering and Electronics, 2022, 44(5): 1536-1542.

图/表 6

图1

图2

图3

表1

表2

图4

参考文献 30

1	杨勇, 胡思茹. 基于模板匹配约束下的光学与SAR图像配准[J]. 系统工程与电子技术, 2019, 41 (10): 2235- 2242. doi: 10.3969/j.issn.1001-506X.2019.10.12
	YANG Y , HU S R . Registration of optical and SAR images base on template matching constraints[J]. Systems Engineering and Electronics, 2019, 41 (10): 2235- 2242. doi: 10.3969/j.issn.1001-506X.2019.10.12
2	TAO Z , JIAN J P , PEI Y Z , et al. Mapping winter wheat with multi-temporal SAR and optical images in an urban agricultural region[J]. Sensors, 2017, 17 (6): 1210- 1215. doi: 10.3390/s17061210
3	BROWN L G . A survey of image registration techniques[J]. ACM Computing Surveys, 1999, 24 (4): 325- 376.
4	SURI S , REINARTZ P . Mutual-information-based registration of TerraSAR[J]. IEEE Trans.on Geoscience & Remote Sensing, 2010, 48 (2): 939- 949.
5	HONG T D , SCHOWENGERDT R A . A robust technique for precise registration of radar and optical satellite images[J]. Photogrammetric Engineering & Remote Sensing, 2005, 71 (5): 585- 593.
6	AN Y, SUN W. A contour-based approach to multisensor image registration using narrow angle features[C]//Proc. of the 2nd International Conference on Electric Information and Control Engineering, 2012: 325-332.
7	LUO J , OUBONG G . A comparison of SIFT, PCA-SIFT and SURF[J]. International Journal of Image Processing, 2009, 3 (4): 131- 183.
8	MATAS J , CHUM O , URBAN M , et al. Robust wide-baseline stereo from maximally stable external regions[J]. Image Vision Computing, 2004, 22 (10): 761- 769. doi: 10.1016/j.imavis.2004.02.006
9	MERKLE N , AUER S , MULLER R , et al. Exploring the potential of conditional adversarial networks for optical and SAR image matching[J]. IEEE Journal of Selected Topics in Applied Earth Observations & Remote Sensing, 2018, 11 (6): 1811- 1820.
10	GATYS L A, ECKER A S. Image style transfer using convolutional neural networks[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 309-312.
11	陈淑環, 韦玉科, 徐乐, 等. 基于深度学习的图像风格迁移研究综述[J]. 计算机应用研究, 2019, 36 (8): 2250- 2255.
	CHEN S H , WEI Y K , XU L , et al. Survey of image style transfer based on deep learning[J]. Application Research of Computers, 2019, 36 (8): 2250- 2255.
12	GOODFELLOW I J , POUGHT A J , MIRZA M , et al. Gene-rative adversarial networks[J]. Advances in Neural Information Processing Systems, 2014, 3, 2672- 2680.
13	ISOLA P, ZHU J Y, ZHOU T, et al. Image to image translation with conditional adversarial networks[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017: 167-173.
14	ZHU J Y, PARK T, ISOLA P, et al. Unpaired image to image translation using cycle-consistent adversarial networks[C]//Proc. of the IEEE International Conference on Computer Vision, 2017: 33-43.
15	CHOI Y, CHOI M, KIM M, et al. StarGAN: unified generative adversarial networks for multi-domain image-to-image translation[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018: 122-130.
16	MIRZA M, OSINDERO S. Conditional generative adversarial nets[EB/OL]. [2021-03-10]. https://doi.org/10.48550/arXiv.1411.1784.
17	LI J S , Y LI H , LI J , et al. Single exposure optical image watermarking using a cGAN network[J]. IEEE Photonics Journal, 2021, 13 (2): 6900111.
18	SON M J , JUNG S W , JUNG S M , et al. BCGAN: a CGAN-based over-sampling model using the boundary class for data balancing[J]. The Journal of Supercomputing, 2021, 77 (9): 1- 25.
19	杜泽星, 殷进勇, 杨建. 基于半监督学习的遥感飞机图像检测方法[J]. 激光与光电子学进展, 2020, 57 (6): 123- 131.
	DU Z X , YIN J Y , YANG J . Remote sensing aircraft image detection based on semi-supervised learning[J]. Laser & Optoelectronics Progress, 2020, 57 (6): 123- 131.
20	JAMAL A T , ISHAK A B , ABDEL S . Tumor edge detection in mammography images using quantum and machine learning approaches[J]. Neural Computing and Applications, 2021, 33 (13): 1- 12.
21	ZHOU S P , WANG J J , WANG L , et al. Hierarchical and interactive refinement network for edge-preserving salient object detection[J]. IEEE Trans.on Image Processing, 2021, 30, 1- 14. doi: 10.1109/TIP.2020.3027992
22	李德仁. 利用遥感影像进行变化检测[J]. 武汉大学学报·信息科学版, 2003, 28 (s1): 7- 12.
	LI D R . Change detection from remote sensing images[J]. Geomatics and Information Science of Wuhan University, 2003, 28 (s1): 7- 12.
23	XIA H Y , LI X R , ZhANG H G , et al. A bathymetry mapping approach combining log-ratio and semianalytical models using four-band multispectral imagery without ground data[J]. IEEE Trans.on Geoscience and Remote Sensing, 2020, 58 (4): 2695- 2709. doi: 10.1109/TGRS.2019.2953381
24	毛天祺, 刘伟, 黄洁, 等. 二进小波增强与边缘局部信息FCM的SAR图像变化检测[J]. 信号处理, 2018, 34 (1): 54- 61.
	MAO T Q , LIU W , HANG J , et al. Change detection of SAR images using dyadic wavelet enhancement and edge local information FCM[J]. Journal of Signal Processing, 2018, 34 (1): 54- 61.
25	XU J L , WEN X P , ZHANG H N , et al. Multi-scale characteristics of remote sensing lineaments[J]. Earth Science Informatics, 2020, 13 (12): 1- 11.
26	CAI M X , WANG S S , WU C . Research on real-time data transmission and multi-scale video image decomposition of embedded optical sensor array based on machine learning[J]. Multimedia Tools and Applications, 2020, (6): 1- 21.
27	王志社, 杨风暴, 纪利娥, 等. 基于聚类分割和形态学的可见光与SAR图像配准[J]. 光学学报, 2014, 34 (2): 176- 182.
	WANG Z S , YANG F B , JI L E , et al. Optical and SAR image registration based on cluster segmentation and mathematical morph-plogy[J]. Acta Optica Sinica, 2014, 34 (2): 176- 182.
28	PALLOTTA L , GIUNTA G , CLEMENTE C . Subpixel SAR image registration through parabolic interpolation of the 2D cross-correlation[J]. IEEE Trans.on Geoscience and Remote Sensing, 2020, 58 (6): 4132- 4144. doi: 10.1109/TGRS.2019.2961245
29	MILLS P H , WU Y J , HO C , et al. Sensitive and automated detection of iron-oxide-labeled cells using phase image cross-correlation analysis[J]. Magnetic Resonance Imaging, 2008, 26 (5): 618- 628. doi: 10.1016/j.mri.2008.01.007
30	陈世伟, 张胜修, 杨小冈, 等. 基于椭圆对称方向矩的可见光与红外图像配准算法[J]. 工程科学学报, 2017, (7): 138- 144.
	CHEN S W , ZHANG S X , YANG X G , et al. Visual-infrared images registration based on ellipse symmetrical orientation moment[J]. Chinese Journal of Engineering, 2017, (7): 138- 144.

序号	场景类型	数据集大小	训练时间/h	模型名称
1	城市遥感图像	30	6	City1
2		100	20	City2
3		300	65	City3
4	港口遥感图像	100	18	Harbor
5	山区遥感图像	50	8	Mountain

序号	方法(算法+模型)	场景类型	MSR/%	MAP
1	边缘+City1	城市	4	2.75
2	边缘+City2		8	1.84
3	边缘+City3		6	1.65
4	NCC+City1		12	4.68
5	NCC+City2		22	3.24
6	NCC+City3		38	2.95
7	SIFT+City1		20	3.23
8	SIFT+City2		42	2.47
9	SIFT+City3		52	1.18
10	本文算法+City1		30	2.78
11	本文算法+City2		58	2.34
12	本文算法+City3		66	1.92
13	边缘+Harbor	港口	6.7	2.24
14	NCC+Harbor		26.7	4.53
15	SIFT+Harbor		43.3	3.76
16	本文算法+Harbor		80	2.68
17	边缘+Mountain	山区	5	4.06
18	NCC+Mountain		5	4.12
19	SIFT+Mountain		15	3.09
20	本文算法+Mountain		35	2.83

基于风格迁移不变特征的SAR与光学图像配准算法

SAR and optical image registration algorithm based on style transfer invariable features

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 6

参考文献 30

相关文章 0

编辑推荐

Metrics

本文评价