基于Faster R-CNN的电子海图和雷达图像的数据融合

doi:10.3969/j.issn.1001-506X.2020.06.09

Abstract

Abstract:

Marine narigation radar and electronic navigational chart (ENC) are important navigation instruments. The fusion of radar images and electronic navigational chart data can give more abundant navigation and collision avoidance information. Therefore, this paper proposes a data fusion algorithm based on deep learning to extract robust features from radar images. At first the ability of deep learning is exploited to perform target detection for the identification of marine radar targets. Then, the image processing is performed on the identified targets to determine the reference points for consistent data fusion of ENC and marine radar information. Finally, an affine transformation-fusion algorithm is built to merge the marine radar and electronic chart data according to the determined reference points. The proposed fusion algorithn is verified through simulations using ENC data and marine radar images from real ships in narrow waters in a continuous period. The results show a suitable edge matching performance of the shoreline and real-time applicability. The algorithm is more robust than the simple pixel-level image fusion, and it realizes the feature level fusion of ENC and radar images.

Key words: deep learning, marine radar image, electronic navigational chart (ENC), data fusion

CLC Number:

TN957.51

Daheng ZHANG, Yingjun ZHANG, Chuang ZHANG. Data fusion of electronic navigational chart and radar images based on Faster R-CNN[J]. Systems Engineering and Electronics, 2020, 42(6): 1267-1273.

Figures/Tables 11

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Table 1

Fig.6

Fig.7

Fig.8

Table 2

Fig.9

References 29

1	LLINAS J, HALL D L.An introduction to multi-sensor data fusion[C]//Proc.of the IEEE International Symposium on Circuits & Systems, 1998: 537-540.
2	AL-SHARMAN M , EMRAN B J , JARADAT M A , et al. Precision landing using an adaptive fuzzy multi-sensor data fusion architecture[J]. Applied Soft Computing, 2018, 69, 149- 164. doi: 10.1016/j.asoc.2018.04.025
3	阎海峰, 胡高歌, 高社生, 等. 基于极大似然准则与滚动时域估计的自适应UKF算法[J]. 系统工程与电子技术, 2016, 38 (7): 1629- 1637.
	YAN H F , HU G G , GAO S S , et al. Adaptive UKF based on maximum likelihood principle and receding horizon estimation[J]. Systems Engineering and Electronics, 2016, 38 (7): 1629- 1637.
4	孟阳, 高社生, 王维. 基于高斯过程回归的平方根UPF算法[J]. 系统工程与电子技术, 2015, 37 (12): 2817- 2822. doi: 10.3969/j.issn.1001-506X.2015.12.23
	MENG Y , GAO S S , WANG W . Square-root unscented particle filter based on Gaussian process regression[J]. Systems Engineering and Electronics, 2015, 37 (12): 2817- 2822. doi: 10.3969/j.issn.1001-506X.2015.12.23
5	赵蓝飞, 王爱丽, 王博, 等. 基于子图融合技术的图像增强算法[J]. 系统工程与电子技术, 2017, 39 (12): 2840- 2848. doi: 10.3969/j.issn.1001-506X.2017.12.30
	ZHAO L F , WANG A L , WANG B , et al. Image enhancement algorithm based on sub-image fusion[J]. Systems Engineering and Electronics, 2017, 39 (12): 2840- 2848. doi: 10.3969/j.issn.1001-506X.2017.12.30
6	MA J Y , JIANG J J , ZHOU H B , et al. Guided locality preserving feature matching for remote sensing image registration[J]. IEEE Trans.on Geoscience and Remote Sensing, 2018, PP (99): 1- 13.
7	李广儒, 张闯, 胡景峰. 基于Harris特征点检测法的雷达图像与电子海图的数据融合[J]. 大连海事大学学报, 2009, 35 (2): 55- 58.
	LI G R , ZHANG C , HU J F . Data fusion of radar image and ECDIS based on matching of Harris feature points[J]. Journal of Dalian Maritime University, 2009, 35 (2): 55- 58.
8	操文芷.基于电子海图和航海雷达的无人水面艇路径规划研究[D].辽宁:大连海事大学, 2017.
	CAO W Z.Research on path planning of USV based on electronic chart and radar image[D].Liaoning: Dalian Maritime University, 2017.
9	ZHAO Y , GAO S S , ZHANG J , et al. Robust predictive augmented unscented Kalman filter[J]. International Journal of Control, Automation and Systems, 2014, 12 (5): 996- 1004. doi: 10.1007/s12555-013-0048-2
10	GOYAL M, REEVES N, RAJBHANDARI S, et al.Fully convolutional networks for diabetic foot ulcer segmentation[C]//Proc.of the IEEE International Conference on Systems, 2017: 618-623.
11	KAZIMIERSKI W , STATECZNY A . Radar and automatic identification system track fusion in an electronic chart display and information system[J]. Journal of Navigation, 2015, 68 (6): 1141- 1154. doi: 10.1017/S0373463315000405
12	彭书娟, 曲长文, 李健伟. 基于ROEWA算子局部活动轮廓的SAR图像分割算法[J]. 系统工程与电子技术, 2019, 41 (2): 280- 291.
	PENG S J , QU C W , LI J W . Local motion contour segmentation of SAR image based on ROEWA operator[J]. Systems Engineering and Electronics, 2019, 41 (2): 280- 291.
13	刘赢, 田润澜, 王晓峰. 基于深层卷积神经网络和双谱特征的雷达信号识别方法[J]. 系统工程与电子技术, 2019, 41 (9): 1998- 2005.
	LIU Y , TIAN R L , WANG X F . Radar signal recognition method based on deep convolutional neural network and bispectrum feature[J]. Systems Engineering and Electronics, 2019, 41 (9): 1998- 2005.
14	才志.雷达图像与电子海图叠加的研究[D].辽宁:大连海事大学, 2011.
	CAI Z.Research of superposition between radar image and electronic chart[D].Liaoning: Dalian Maritime University, 2011.
15	IEC 62388.Maritime navigation and radiocommunication equipment and systems-shipborne radar-performance requirements, methods of testing and required test results[S].Geneva: International Electrotechnology Commission, 2013.
16	DAI J F, LI Y, HE K M, et al.R-FCN: object detection via region-based fully convolutional networks[C]//Proc.of the 30th International Conference on Neural Information Processing Systems, 2016: 379-387.
17	SUN X , WU P , HOI S C H . Face detection using deep learning:an improved faster RCNN approach[J]. Neurocomputing, 2018, 299, 42- 50. doi: 10.1016/j.neucom.2018.03.030
18	HOANG NGAN LE T, ZHENG Y, ZHU C, et al.Multiple scale faster-rcnn approach to driver's cell-phone usage and hands on steering wheel detection[C]//Proc.of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2016: 46-53.
19	REN S, HE K, GIRSHICK R, et al.Faster R-CNN: towards real-time object detection with region proposal networks[C]//Proc.of the Advances in Neural Information Processing Systems, 2015: 91-99.
20	XU Y Z , YU G Z , WANG Y P , et al. Car detection from low-altitude UAV imagery with the faster R-CNN[J]. Journal of Advanced Transportation, 2017, 2823619.
21	ZHAO X T, LI W, ZHANG Y F, et al.A faster RCNN-based pedestrian detection system[C]//Proc.of the IEEE 84th Vehicular Technology Conference (VTC-Fall), 2016: 1-5.
22	REN Y , ZHU C R , XIAO S P . Deformable faster R-CNN with aggregating multi-layer features for partially occluded object detection in optical remote sensing images[J]. Remote Sensing, 2018, 10 (9): 1470. doi: 10.3390/rs10091470
23	SALVADOR A, GIRó-I-NIETO X, MARQUéS F, et al.Faster R-CNN features for instance search[C]//Proc.of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2016: 9-16.
24	WANG Q , ZHANG X D , CHEN G Z , et al. Change detection based on faster R-CNN for high-resolution remote sensing images[J]. Remote Sensing Letters, 2018, 9 (10): 923- 932. doi: 10.1080/2150704X.2018.1492172
25	谢兴澜, 李源惠. 电子海图中不同基准纬度海图墨卡托坐标转换[J]. 大连海事大学学报, 2003, 29 (2): 19- 21.
	XIE X L , LI Y H . Mercator coordinate transformation of charts with different datum latitude in electronic chart[J]. Journal of Dalian Maritime University, 2003, 29 (2): 19- 21.
26	郑尧. 电子海图与雷达图像的融合显示[J]. 舰船科学技术, 2015, 37 (3): 181- 184. doi: 10.3404/j.issn.1672-7649.2015.03.040
	ZHENG Y . Research of integration display on electronic charts and radar images[J]. Ship Science and Technology, 2015, 37 (3): 181- 184. doi: 10.3404/j.issn.1672-7649.2015.03.040
27	OTSU N . A threshold selection method from gray-level histograms[J]. IEEE Trans.on Systems Man & Cybernetics, 1979, 9 (1): 62- 66.
28	杨功流, 窦玉宝, 郑荣才. 一种雷达和电子海图图像叠加方法[J]. 中国惯性技术学报, 2010, 18 (2): 181- 184.
	YANG G L , DOU Y B , ZHENG R C . Method of image overlay on radar and electronic chart[J]. Journal of Chinese Inertial Technology, 2010, 18 (2): 181- 184.
29	熊恒昌.基于深度学习的实时目标检测[D].湖南:湖南大学, 2018.
	Xiong H C.Real-time object detection using deep learning[D].Changsha: Hunan University of Computer Science and Technology, 2018.

检测状态	MAP	岬角	防波堤	船舶	岸线	桥梁	雷康
类别减少前	0.297 3	0.385 6	0.393 1	0.226 0	0.498 6	0.435 6	0.065 0
类别减少后	0.364 9	0.417 4	NaN	NaN	0.513 2	0.504 2	NaN

物标种类	MAP	桥梁	陆地	岬角
精确度/%	0.517 8	0.534 8	0.584 7	0.433 8

[1]	Xiao HAN, Shiwen CHEN, Meng CHEN, Jincheng YANG. Open-set recognition of LPI radar signal based on reciprocal point learning [J]. Systems Engineering and Electronics, 2022, 44(9): 2752-2759.
[2]	Taiyang HU, Jinyu ZHANG, Hailiang LU, Pengfei LI, Yinan LI, Rongchuan LYU. Distributed synthetic aperture microwave radiometric high-resolution imaging algorithm based on data fusion [J]. Systems Engineering and Electronics, 2022, 44(8): 2403-2409.
[3]	Limin ZHANG, Kaiwen TAN, Wenjun YAN, Yuyuan ZHANG. Radar emitter recognition based on multi-level jumper residual network [J]. Systems Engineering and Electronics, 2022, 44(7): 2148-2156.
[4]	Guodong JIN, Yuanliang XUE, Lining TAN, Jiankun XU. Advances in object tracking algorithm based on siamese network [J]. Systems Engineering and Electronics, 2022, 44(6): 1805-1822.
[5]	Xiaofeng ZHAO, Yebin XU, Fei WU, Jiahui NIU, Wei CAI, Zhili ZHANG. Ground infrared target detection method based on global sensing mechanism [J]. Systems Engineering and Electronics, 2022, 44(5): 1461-1467.
[6]	Hong ZOU, Chenyang BAI, Peng HE, Yaping CUI, Ruyan WANG, Dapeng WU. Edge service placement strategy based on distributed deep learning [J]. Systems Engineering and Electronics, 2022, 44(5): 1728-1737.
[7]	Dong CHEN, Yanwei JU. Ship object detection SAR images based on semantic segmentation [J]. Systems Engineering and Electronics, 2022, 44(4): 1195-1201.
[8]	Jingming SUN, Shengkang YU, Jun SUN. Pose sensitivity analysis of HRRP recognition based on deep learning [J]. Systems Engineering and Electronics, 2022, 44(3): 802-807.
[9]	Yunxiang YAO, Ying CHEN. Target tracking network based on dual-modal interactive fusion under attention mechanism [J]. Systems Engineering and Electronics, 2022, 44(2): 410-419.
[10]	Yongxing GAO, Xudong WANG, Ling WANG, Daiyin ZHU, Jun GUO, Fanwang MENG. Weather signal detection for dual polarization weather radar based on RCNN [J]. Systems Engineering and Electronics, 2022, 44(11): 3380-3387.
[11]	Yiheng ZHOU, Jun YANG, Saiqiang XIA, Mingjiu LYU. Estimation method of micro-motion parameters for rotor targets under flashing [J]. Systems Engineering and Electronics, 2022, 44(1): 54-63.
[12]	Rong FU, Tianyao HUANG, Yimin LIU. DNN based 1-bit block sparse recovery in frequency agile coherent radar [J]. Systems Engineering and Electronics, 2022, 44(1): 70-75.
[13]	Yuyuan ZHANG, Limin ZHANG, Wenjun YAN. SFBC-OFDM recognition method based on cross-correlation feature map and dilated dense convolutional neural networks [J]. Systems Engineering and Electronics, 2021, 43(9): 2657-2664.
[14]	Lulu ZHANG, Siya CHEN, Guang JIN. Evaluation modeling of spacecraft radiation resistance equivalent test [J]. Systems Engineering and Electronics, 2021, 43(9): 2673-2677.
[15]	Caiyun WANG, Yangyu LI, Xiaofei LI, Jianing WANG, Wenyi WEI. Aerial image super-resolution restruction based on sparsity and deep learning [J]. Systems Engineering and Electronics, 2021, 43(8): 2045-2050.

Data fusion of electronic navigational chart and radar images based on Faster R-CNN

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 29

Related Articles 15

Recommended Articles

Metrics

Comments