基于风格迁移不变特征的SAR与光学图像配准算法

doi:10.12305/j.issn.1001-506X.2022.05.14

Abstract

Abstract:

According to the matching difficulties of multi-sensor remote sensing images, a new synthetic aperture radar (SAR) and optical image registration approach based on image style transfer invariable features is proposed. First, the style transfer neural network for SAR image conversion to optical image is trained. Then, the artificial optical image and the difference map between the artificial optical image and the original SAR image are generated based on the style transfer neural network, and the edge regions of the artificial optical image and the difference image are enhanced by using the multi-scale characteristics of wavelet. After binary segmentation, the stable edge features of the artificial optical image. And the edge features of the optical reference image are also extracted. Finally, the edge features are matched by the cross-correlation criterion to achieve the accurate registration between the original SAR image and the optical reference image. The experiments demonstrate that the artificial optical image can be accurately registered with the optical reference image even under the condition of insufficient training samples. This enhances the adaptability of the algorithm.

Key words: multi-sensor images matching, image style transfer, difference image, stable edge features

CLC Number:

TP394.1

Shiwei CHEN, Hai XIA, Xiaogang YANG, Xiaofeng LI. SAR and optical image registration algorithm based on style transfer invariable features[J]. Systems Engineering and Electronics, 2022, 44(5): 1536-1542.

Figures/Tables 6

Fig.1

Fig.2

Fig.3

Table 1

Table 2

Fig.4

References 30

1	杨勇, 胡思茹. 基于模板匹配约束下的光学与SAR图像配准[J]. 系统工程与电子技术, 2019, 41 (10): 2235- 2242. doi: 10.3969/j.issn.1001-506X.2019.10.12
	YANG Y , HU S R . Registration of optical and SAR images base on template matching constraints[J]. Systems Engineering and Electronics, 2019, 41 (10): 2235- 2242. doi: 10.3969/j.issn.1001-506X.2019.10.12
2	TAO Z , JIAN J P , PEI Y Z , et al. Mapping winter wheat with multi-temporal SAR and optical images in an urban agricultural region[J]. Sensors, 2017, 17 (6): 1210- 1215. doi: 10.3390/s17061210
3	BROWN L G . A survey of image registration techniques[J]. ACM Computing Surveys, 1999, 24 (4): 325- 376.
4	SURI S , REINARTZ P . Mutual-information-based registration of TerraSAR[J]. IEEE Trans.on Geoscience & Remote Sensing, 2010, 48 (2): 939- 949.
5	HONG T D , SCHOWENGERDT R A . A robust technique for precise registration of radar and optical satellite images[J]. Photogrammetric Engineering & Remote Sensing, 2005, 71 (5): 585- 593.
6	AN Y, SUN W. A contour-based approach to multisensor image registration using narrow angle features[C]//Proc. of the 2nd International Conference on Electric Information and Control Engineering, 2012: 325-332.
7	LUO J , OUBONG G . A comparison of SIFT, PCA-SIFT and SURF[J]. International Journal of Image Processing, 2009, 3 (4): 131- 183.
8	MATAS J , CHUM O , URBAN M , et al. Robust wide-baseline stereo from maximally stable external regions[J]. Image Vision Computing, 2004, 22 (10): 761- 769. doi: 10.1016/j.imavis.2004.02.006
9	MERKLE N , AUER S , MULLER R , et al. Exploring the potential of conditional adversarial networks for optical and SAR image matching[J]. IEEE Journal of Selected Topics in Applied Earth Observations & Remote Sensing, 2018, 11 (6): 1811- 1820.
10	GATYS L A, ECKER A S. Image style transfer using convolutional neural networks[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 309-312.
11	陈淑環, 韦玉科, 徐乐, 等. 基于深度学习的图像风格迁移研究综述[J]. 计算机应用研究, 2019, 36 (8): 2250- 2255.
	CHEN S H , WEI Y K , XU L , et al. Survey of image style transfer based on deep learning[J]. Application Research of Computers, 2019, 36 (8): 2250- 2255.
12	GOODFELLOW I J , POUGHT A J , MIRZA M , et al. Gene-rative adversarial networks[J]. Advances in Neural Information Processing Systems, 2014, 3, 2672- 2680.
13	ISOLA P, ZHU J Y, ZHOU T, et al. Image to image translation with conditional adversarial networks[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017: 167-173.
14	ZHU J Y, PARK T, ISOLA P, et al. Unpaired image to image translation using cycle-consistent adversarial networks[C]//Proc. of the IEEE International Conference on Computer Vision, 2017: 33-43.
15	CHOI Y, CHOI M, KIM M, et al. StarGAN: unified generative adversarial networks for multi-domain image-to-image translation[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018: 122-130.
16	MIRZA M, OSINDERO S. Conditional generative adversarial nets[EB/OL]. [2021-03-10]. https://doi.org/10.48550/arXiv.1411.1784.
17	LI J S , Y LI H , LI J , et al. Single exposure optical image watermarking using a cGAN network[J]. IEEE Photonics Journal, 2021, 13 (2): 6900111.
18	SON M J , JUNG S W , JUNG S M , et al. BCGAN: a CGAN-based over-sampling model using the boundary class for data balancing[J]. The Journal of Supercomputing, 2021, 77 (9): 1- 25.
19	杜泽星, 殷进勇, 杨建. 基于半监督学习的遥感飞机图像检测方法[J]. 激光与光电子学进展, 2020, 57 (6): 123- 131.
	DU Z X , YIN J Y , YANG J . Remote sensing aircraft image detection based on semi-supervised learning[J]. Laser & Optoelectronics Progress, 2020, 57 (6): 123- 131.
20	JAMAL A T , ISHAK A B , ABDEL S . Tumor edge detection in mammography images using quantum and machine learning approaches[J]. Neural Computing and Applications, 2021, 33 (13): 1- 12.
21	ZHOU S P , WANG J J , WANG L , et al. Hierarchical and interactive refinement network for edge-preserving salient object detection[J]. IEEE Trans.on Image Processing, 2021, 30, 1- 14. doi: 10.1109/TIP.2020.3027992
22	李德仁. 利用遥感影像进行变化检测[J]. 武汉大学学报·信息科学版, 2003, 28 (s1): 7- 12.
	LI D R . Change detection from remote sensing images[J]. Geomatics and Information Science of Wuhan University, 2003, 28 (s1): 7- 12.
23	XIA H Y , LI X R , ZhANG H G , et al. A bathymetry mapping approach combining log-ratio and semianalytical models using four-band multispectral imagery without ground data[J]. IEEE Trans.on Geoscience and Remote Sensing, 2020, 58 (4): 2695- 2709. doi: 10.1109/TGRS.2019.2953381
24	毛天祺, 刘伟, 黄洁, 等. 二进小波增强与边缘局部信息FCM的SAR图像变化检测[J]. 信号处理, 2018, 34 (1): 54- 61.
	MAO T Q , LIU W , HANG J , et al. Change detection of SAR images using dyadic wavelet enhancement and edge local information FCM[J]. Journal of Signal Processing, 2018, 34 (1): 54- 61.
25	XU J L , WEN X P , ZHANG H N , et al. Multi-scale characteristics of remote sensing lineaments[J]. Earth Science Informatics, 2020, 13 (12): 1- 11.
26	CAI M X , WANG S S , WU C . Research on real-time data transmission and multi-scale video image decomposition of embedded optical sensor array based on machine learning[J]. Multimedia Tools and Applications, 2020, (6): 1- 21.
27	王志社, 杨风暴, 纪利娥, 等. 基于聚类分割和形态学的可见光与SAR图像配准[J]. 光学学报, 2014, 34 (2): 176- 182.
	WANG Z S , YANG F B , JI L E , et al. Optical and SAR image registration based on cluster segmentation and mathematical morph-plogy[J]. Acta Optica Sinica, 2014, 34 (2): 176- 182.
28	PALLOTTA L , GIUNTA G , CLEMENTE C . Subpixel SAR image registration through parabolic interpolation of the 2D cross-correlation[J]. IEEE Trans.on Geoscience and Remote Sensing, 2020, 58 (6): 4132- 4144. doi: 10.1109/TGRS.2019.2961245
29	MILLS P H , WU Y J , HO C , et al. Sensitive and automated detection of iron-oxide-labeled cells using phase image cross-correlation analysis[J]. Magnetic Resonance Imaging, 2008, 26 (5): 618- 628. doi: 10.1016/j.mri.2008.01.007
30	陈世伟, 张胜修, 杨小冈, 等. 基于椭圆对称方向矩的可见光与红外图像配准算法[J]. 工程科学学报, 2017, (7): 138- 144.
	CHEN S W , ZHANG S X , YANG X G , et al. Visual-infrared images registration based on ellipse symmetrical orientation moment[J]. Chinese Journal of Engineering, 2017, (7): 138- 144.

序号	场景类型	数据集大小	训练时间/h	模型名称
1	城市遥感图像	30	6	City1
2		100	20	City2
3		300	65	City3
4	港口遥感图像	100	18	Harbor
5	山区遥感图像	50	8	Mountain

序号	方法(算法+模型)	场景类型	MSR/%	MAP
1	边缘+City1	城市	4	2.75
2	边缘+City2		8	1.84
3	边缘+City3		6	1.65
4	NCC+City1		12	4.68
5	NCC+City2		22	3.24
6	NCC+City3		38	2.95
7	SIFT+City1		20	3.23
8	SIFT+City2		42	2.47
9	SIFT+City3		52	1.18
10	本文算法+City1		30	2.78
11	本文算法+City2		58	2.34
12	本文算法+City3		66	1.92
13	边缘+Harbor	港口	6.7	2.24
14	NCC+Harbor		26.7	4.53
15	SIFT+Harbor		43.3	3.76
16	本文算法+Harbor		80	2.68
17	边缘+Mountain	山区	5	4.06
18	NCC+Mountain		5	4.12
19	SIFT+Mountain		15	3.09
20	本文算法+Mountain		35	2.83

SAR and optical image registration algorithm based on style transfer invariable features

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 6

References 30

Related Articles 0

Recommended Articles

Metrics

Comments