基于决策融合的多无人机协同目标检测识别算法

doi:10.12305/j.issn.1001-506X.2022.03.05

Abstract

Abstract:

Since a single unmanned aerial vehicle (UAV) cannot patrol a large area efficiently, an object detection and recognition algorithm based on multiple UAVs decision fusion is proposed. Firstly, Retinanet algorithm is improved for object detection of a single UAV. Anchor parameters and training strategies are adjusted according to object characteristics of aerial images. Meanwhile, the feature extraction operator is used to register the aerial images of multiple UAVs such that the coordinates of the images of multiple UAVs are consistent and the images are mosaic. Then, the multiple UAVs images are correlated by combining information of the object position and the attribute. Finally, a dynamic switching strategy based on collision measurement is proposed, which adaptively selects the Dempster-Shafer theory (DST) or desert-smarandache theory (DSmT) to fuse the associated object information. Experimental results on multiple UAVs cooperative object identification data set show that the proposed algorithm can not only increase the single patrol range, but also improve the detection accuracy of the UAV patrol system.

Key words: machine vision, multiple unmanned aerial vehicles (UAVs) cooperation, object detection, multi-objective association, dynamic switching strategy

CLC Number:

TP391.4

Hongyao LI, Xiaoqiang LI, Xinzhong HAN, Xueli XIE, Jianxiang XI. Cooperative object detection and recognition algorithm for multiple UAVs based on decision fusion[J]. Systems Engineering and Electronics, 2022, 44(3): 746-754.

Figures/Tables 18

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Table 1

Fig.7

Table 2

Table 3

Fig.8

Fig.9

Table 4

Table 5

Table 6

Table 7

Fig.10

Fig.11

References 25

1	KANISTRAS K , MARTINS G , RUTHERFORD M J , et al. Survey of unmanned aerial vehicles (UAVs) for traffic monitoring[J]. Handbook of UAVs, 2015, 34 (2): 2643- 2666.
2	KORKI M, SHANKAR N D, SHAH R N, et al. Automatic fault detection of power lines using unmanned aerial vehicle (UAV)[C]//Proc. of the 1st International Conference on Unmanned Vehicle Systems-Oman (UVS), 2019.
3	HAN P F, ZHAO H W, CHEN C Z. UAV intelligent system for patrol missions[C]//Proc. of the 8th International Conference on Manufacturing Science and Engineering, 2018: 79-83.
4	LIU Z G , LYU Y , HAN Z W . Detection approach based on an improved faster RCNN for brace sleeve screws in high-speed railways[J]. IEEE Trans.on Instrumentation and Measurement, 2020, 69 (7): 4395- 4403. doi: 10.1109/TIM.2019.2941292
5	HOU B , REN Z L , ZHAO W , et al. Object detection in high-resolution panchromatic images using deep models and spatial template matching[J]. IEEE Trans.on Geoscience and Remote Sensing, 2020, 58 (2): 956- 970. doi: 10.1109/TGRS.2019.2942103
6	CHENG X , YU J B . RetinaNet with difference channel attention and adaptively spatial feature fusion for steel surface defect detection[J]. IEEE Trans.on Instrumentation and Measurement, 2021, 70, 391- 401.
7	TANG X , THARMARASA R , MCDONALD M , et al. Multiple detection-aided low-observable track initialization using ML-PDA[J]. IEEE Trans.on Aerospace and Electronic Systems, 2017, 53 (2): 722- 735. doi: 10.1109/TAES.2017.2664598
8	ROECKER J A . A class of near optimal JPDA algorithms[J]. IEEE Trans.on Aerospace and Electronic Systems, 1994, 30 (2): 504- 510. doi: 10.1109/7.272272
9	SINHA A , DING Z , KIRUBARAJAN T , et al. Track quality based multitarget tracking approach for global nearest-neighbor association[J]. IEEE Trans.on Aerospace & Electronic Systems, 2012, 48 (2): 1179- 1191.
10	LEE S J , MCBRIDE J . Extended object tracking via positive and negative information fusion[J]. IEEE Trans.on Signal Processing, 2016, 67 (7): 1812- 1823.
11	KOCH S , KAUFMAN H , BIEMOND J . Restoration of spatially varying blurred images using multiple model-based extended Kalman filters[J]. IEEE Trans.on Image Processing, 1995, 4 (4): 520- 523. doi: 10.1109/83.370684
12	MIAO Z X, CHENG Y, PAN Q F, et al. An intelligent fusion method of sequential images based on improved dsmt for target recognition[C]//Proc. of the International Conference on Computational Aspects of Social Networks, 2010: 369-373.
13	王琪龙, 李建勇, 沈海阔. 双目视觉-激光测距传感器目标跟踪系统[J]. 光学学报, 2016, 36 (9): 520- 523.
	WANG Q L , LI J Y , SHEN H K . Target tracking system of binocular vision and laser range sensor[J]. Acta Optica Sinica, 2016, 36 (9): 520- 523.
14	陆峰, 徐友春, 李永乐, 等. 基于DSmT理论的多视角融合目标检测识别[J]. 机器人, 2018, 40 (5): 723- 733.
	LU F , XU Y C , LI Y L , et al. Multi-perspective fusion for object detection and recognition based on DSmT[J]. Robot, 2018, 40 (5): 723- 733.
15	CHAVEZ-GARCIA R O , AYCARD O . Multiple sensor fusion and classification for moving object detection and tracking[J]. IEEE Trans.on Intelligent Transportation Systems, 2015, 17 (2): 525- 534.
16	屈耀红, 张峰, 谷任能, 等. 基于距离测量的多无人机协同目标定位方法[J]. 西北工业大学学报, 2019, 37 (2): 266- 272. doi: 10.3969/j.issn.1000-2758.2019.02.008
	QU Y H , ZHANG F , GU R N , et al. Target cooperative location method of multi-UAV based on pseudo range measurement[J]. Journal of Northwestern Polytechnical University, 2019, 37 (2): 266- 272. doi: 10.3969/j.issn.1000-2758.2019.02.008
17	MAZA I, KONDAK K, BERNARD M, et al. Multi-UAV coope- ration and control for load transportation and deployment[C]//Proc. of the 2nd International Symposium on UAVs, 2009: 417-449.
18	KOOHIFAR F , KUMBHAR A , GUVENC I . Receding horizon multi-UAV cooperative tracking of moving RF source[J]. IEEE Communications Letters, 2016, 21 (6): 1433- 1436.
19	ZHAO Y Y , WANG X , WANG C C , et al. Systemic design of distributed multi-UAV cooperative decision-making for multi-target tracking[J]. Autonomous Agents and Multi-Agent Systems, 2019, 33 (2): 132- 158.
20	王轶, 方英武. 基于DST-PCR5多目标自适应视觉跟踪方法[J]. 计算机应用研究, 2018, 35 (1): 632- 636.
	WANG Y , FANG Y W . Adaptive visual tracking method of multi-targets based on DST-PCR5[J]. Application Research of Computers, 2018, 35 (1): 632- 636.
21	JOUSSELME A L , GRENIER D , BOSSÉ É . A new distance between two bodies of evidence[J]. Information Fusion, 2001, 2 (2): 91- 101. doi: 10.1016/S1566-2535(01)00026-4
22	KONG W Z , HONG J C , JIA M Y , et al. YOLOv3-DPFIN: a dual-path feature fusion neural network for robust real-time sonar target detection[J]. IEEE Sensors Journal, 2019, 20 (7): 3745- 3756.
23	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[C]// Proc. of the Advances in Neural Information Processing Systems, 2015: 91-99.
24	LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot multibox detector[C]//Proc. of the European Conference on Computer Vision, 2016: 21-37.
25	RUBLEE E, RABAUD V, KONOLIGE K, et al. ORB: an efficient alternative to SIFT or SURF[C]//Proc. of the International Conference on Computer Vision, 2012: 2564-2571.

检测框	可能情况
检测框	Ⅰ	Ⅱ	Ⅲ	Ⅳ	Ⅴ	Ⅵ	Ⅶ
L_b	1	0	0	0	1	1	1
M_b	0	1	0	1	1	0	1
R_b	0	0	1	1	0	1	1

类别	分类0	分类1	分类2	分类3	分类4	分类5	分类6
UAV-L-score	0.507	0.180	0.050	0.010	0.081	0.001	0.170
UAV-M-score	0.619	0.107	0.074	0.008	0.081	0.001	0.108
UAV-R-score	0.701	0.088	0.065	0.004	0.043	0.001	0.097
D-score	-	-	-	0.163, 弱冲突条件	-	-	-
DST-score	0.982	0.008	0.011	1.4×10^-6	0.001	4.5×10^-9	0.008
PCR5-score	0.837	0.062	0.018	0.003	0.021	5.2×10^-6	0.061

类别	分类0	分类1	分类2	分类3	分类4	分类5	分类6
UAV-L-score	0.507	0.180	0.050	0.010	0.081	0.001	0.170
UAV-M-score	0.019	0.707	0.074	0.008	0.081	0.001	0.108
UAV-R-score	0.701	0.088	0.065	0.004	0.043	0.001	0.097
D-score	-	-	-	0.652, 强冲突条件	-	-	-
DST-score	0.333	0.553	0.012	1.5×10^-5	0.014	4.9×10^-8	0.088
PCR5-score	0.520	0.386	0.017	2.7×10^-4	0.020	4.8×10^-6	0.057

算法	输入	基础网络	mAP/%	时间/ms
Faster R-CNN	600	Resnet-50	68.32	137.4
SSD	512	Vgg-16	63.29	57.8
YOLO v3	416	Darknet-53	66.86	62.4
Retinanet	608	Resnet-50	69.78	84.6
本文算法	608	Resnet-50	72.65	86.7

方案	匹配点对数目	均方根误差	计算时间/s
SIFT+BF	132	1.32	0.351
SURF+BF	125	2.25	0.196
ORB^[21]+BF	98	3.72	0.105
SIFT+KNN	126	1.83	0.311
SURF+KNN	120	2.56	0.153
ORB+KNN	45	5.47	0.052

Cooperative object detection and recognition algorithm for multiple UAVs based on decision fusion

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 18

References 25

Related Articles 12

Recommended Articles

Metrics

Comments

算法	关联正确率/%	计算时间/ms
本文算法	88.52	5.62
PDA^[22]	88.31	8.32
JPDA^[23]	90.70	14.73
MHT^[24]	91.48	17.29

算法	融合前mAP /%	融合后mAP /%	计算时间/ms
Vote	79.65	83.24	5.32
DST	79.65	87.59	12.72
PCR5	79.65	91.82	21.13
动态策略	79.65	92.48	18.25

[1]	Shuang SONG, Yue ZHANG, Linna ZHANG, Yigang CEN, Yidong LI. Lightweight target detection algorithm based on deep learning [J]. Systems Engineering and Electronics, 2022, 44(9): 2716-2725.
[2]	Xiaoling ZHOU, Zhaoxia ZHANG, Ya LU, Qian WANG, Kunkun WANG. SAR image recognition based on improved R-FCN [J]. Systems Engineering and Electronics, 2022, 44(4): 1202-1209.
[3]	Jiali FAN, Shaobing TIAN, Kui HUANG, Xingdong ZHU. Multi-scale object detection algorithm for aircraft carrier surface based on Faster R-CNN [J]. Systems Engineering and Electronics, 2022, 44(1): 40-46.
[4]	Yiming ZHANG, Jianliang AI. Positioning of aerial refueling drogue and docking control based on binocular vision [J]. Systems Engineering and Electronics, 2021, 43(10): 2940-2953.
[5]	Zhenzhen YANG, Jun LE, Yongpeng YANG, Lu FAN. Object detection algorithm of nonconvex motion-assisted low rank and sparse decomposition [J]. Systems Engineering and Electronics, 2020, 42(6): 1218-1225.
[6]	Juan SU, Long YANG, Hua HUANG, Guodong JIN. Improved SSD algorithm for small-sized SAR ship detection [J]. Systems Engineering and Electronics, 2020, 42(5): 1026-1034.
[7]	ZHOU Long, WEI Suyuan, CUI Zhongma, FANG Jiaqi, YANG Xiaoting, YANG Long. Multiobjective detection of complex background radar imagebased on deep learning [J]. Systems Engineering and Electronics, 2019, 41(6): 1258-1264.
[8]	GUO Shao-jun, SHEN Tong-sheng, XU Jian, MA Xin-xing. Detection of multi-ship targets at sea based on ObjectNess BING [J]. Systems Engineering and Electronics, 2016, 38(1): 14-20.
[9]	ZHOU Jun-jing, DUAN Jian-min. Moving object detection for intelligent vehicles based on occupancy grid map [J]. Systems Engineering and Electronics, 2015, 37(2): 436-442.
[10]	DI Xiao-guang, FENG Xiao-dan, Fu Shao-wen. Moving object detection in non-planar scenes based on improved gradient suppression [J]. Systems Engineering and Electronics, 2014, 36(6): 1021-1026.
[11]	LIU Ying-xia, CHANG Fa-liang. Detection algorithm for moving objects based on Bayesian rules and optimized level projection [J]. Journal of Systems Engineering and Electronics, 2011, 33(4): 933-937.
[12]	ZHAO Wen-zhe, QIN Shi-yin. Chromatic image classification and recognition based on interest point features [J]. Journal of Systems Engineering and Electronics, 2011, 33(2): 438-442.