基于决策融合的多无人机协同目标检测识别算法

doi:10.12305/j.issn.1001-506X.2022.03.05

摘要/Abstract

摘要：

针对现阶段单无人机不能高效完成大区域巡视的问题, 提出一种多无人机决策融合的目标检测识别算法。首先改进Retinanet算法进行单无人机的目标检测, 根据航拍图像目标特性, 调整anchor参数和训练策略。同时利用特征提取算子配准多无人机航拍图像, 实现多机图像坐标一致, 并进行图像拼接。然后综合目标的位置信息和属性信息对多机图像进行目标关联。最后提出一种基于冲突度量的动态切换策略, 自适应选择DST(dempster-shafer theory)或DSmT(desert-smarandache theory)融合关联目标信息。在多无人机协同目标识别数据集上进行实验, 结果表明所提算法能在增大单次巡视范围的同时, 提高无人机巡视系统的检测精度。

关键词: 机器视觉, 多无人机协同, 目标检测, 多目标关联, 动态切换策略

Abstract:

Since a single unmanned aerial vehicle (UAV) cannot patrol a large area efficiently, an object detection and recognition algorithm based on multiple UAVs decision fusion is proposed. Firstly, Retinanet algorithm is improved for object detection of a single UAV. Anchor parameters and training strategies are adjusted according to object characteristics of aerial images. Meanwhile, the feature extraction operator is used to register the aerial images of multiple UAVs such that the coordinates of the images of multiple UAVs are consistent and the images are mosaic. Then, the multiple UAVs images are correlated by combining information of the object position and the attribute. Finally, a dynamic switching strategy based on collision measurement is proposed, which adaptively selects the Dempster-Shafer theory (DST) or desert-smarandache theory (DSmT) to fuse the associated object information. Experimental results on multiple UAVs cooperative object identification data set show that the proposed algorithm can not only increase the single patrol range, but also improve the detection accuracy of the UAV patrol system.

Key words: machine vision, multiple unmanned aerial vehicles (UAVs) cooperation, object detection, multi-objective association, dynamic switching strategy

中图分类号:

TP391.4

李洪瑶, 李小强, 韩心中, 谢学立, 席建祥. 基于决策融合的多无人机协同目标检测识别算法[J]. 系统工程与电子技术, 2022, 44(3): 746-754.

Hongyao LI, Xiaoqiang LI, Xinzhong HAN, Xueli XIE, Jianxiang XI. Cooperative object detection and recognition algorithm for multiple UAVs based on decision fusion[J]. Systems Engineering and Electronics, 2022, 44(3): 746-754.

图/表 18

图1

图2

图3

图4

图5

图6

表1

图7

表2

表3

图8

图9

表4

表5

表6

表7

图10

图11

参考文献 25

1	KANISTRAS K , MARTINS G , RUTHERFORD M J , et al. Survey of unmanned aerial vehicles (UAVs) for traffic monitoring[J]. Handbook of UAVs, 2015, 34 (2): 2643- 2666.
2	KORKI M, SHANKAR N D, SHAH R N, et al. Automatic fault detection of power lines using unmanned aerial vehicle (UAV)[C]//Proc. of the 1st International Conference on Unmanned Vehicle Systems-Oman (UVS), 2019.
3	HAN P F, ZHAO H W, CHEN C Z. UAV intelligent system for patrol missions[C]//Proc. of the 8th International Conference on Manufacturing Science and Engineering, 2018: 79-83.
4	LIU Z G , LYU Y , HAN Z W . Detection approach based on an improved faster RCNN for brace sleeve screws in high-speed railways[J]. IEEE Trans.on Instrumentation and Measurement, 2020, 69 (7): 4395- 4403. doi: 10.1109/TIM.2019.2941292
5	HOU B , REN Z L , ZHAO W , et al. Object detection in high-resolution panchromatic images using deep models and spatial template matching[J]. IEEE Trans.on Geoscience and Remote Sensing, 2020, 58 (2): 956- 970. doi: 10.1109/TGRS.2019.2942103
6	CHENG X , YU J B . RetinaNet with difference channel attention and adaptively spatial feature fusion for steel surface defect detection[J]. IEEE Trans.on Instrumentation and Measurement, 2021, 70, 391- 401.
7	TANG X , THARMARASA R , MCDONALD M , et al. Multiple detection-aided low-observable track initialization using ML-PDA[J]. IEEE Trans.on Aerospace and Electronic Systems, 2017, 53 (2): 722- 735. doi: 10.1109/TAES.2017.2664598
8	ROECKER J A . A class of near optimal JPDA algorithms[J]. IEEE Trans.on Aerospace and Electronic Systems, 1994, 30 (2): 504- 510. doi: 10.1109/7.272272
9	SINHA A , DING Z , KIRUBARAJAN T , et al. Track quality based multitarget tracking approach for global nearest-neighbor association[J]. IEEE Trans.on Aerospace & Electronic Systems, 2012, 48 (2): 1179- 1191.
10	LEE S J , MCBRIDE J . Extended object tracking via positive and negative information fusion[J]. IEEE Trans.on Signal Processing, 2016, 67 (7): 1812- 1823.
11	KOCH S , KAUFMAN H , BIEMOND J . Restoration of spatially varying blurred images using multiple model-based extended Kalman filters[J]. IEEE Trans.on Image Processing, 1995, 4 (4): 520- 523. doi: 10.1109/83.370684
12	MIAO Z X, CHENG Y, PAN Q F, et al. An intelligent fusion method of sequential images based on improved dsmt for target recognition[C]//Proc. of the International Conference on Computational Aspects of Social Networks, 2010: 369-373.
13	王琪龙, 李建勇, 沈海阔. 双目视觉-激光测距传感器目标跟踪系统[J]. 光学学报, 2016, 36 (9): 520- 523.
	WANG Q L , LI J Y , SHEN H K . Target tracking system of binocular vision and laser range sensor[J]. Acta Optica Sinica, 2016, 36 (9): 520- 523.
14	陆峰, 徐友春, 李永乐, 等. 基于DSmT理论的多视角融合目标检测识别[J]. 机器人, 2018, 40 (5): 723- 733.
	LU F , XU Y C , LI Y L , et al. Multi-perspective fusion for object detection and recognition based on DSmT[J]. Robot, 2018, 40 (5): 723- 733.
15	CHAVEZ-GARCIA R O , AYCARD O . Multiple sensor fusion and classification for moving object detection and tracking[J]. IEEE Trans.on Intelligent Transportation Systems, 2015, 17 (2): 525- 534.
16	屈耀红, 张峰, 谷任能, 等. 基于距离测量的多无人机协同目标定位方法[J]. 西北工业大学学报, 2019, 37 (2): 266- 272. doi: 10.3969/j.issn.1000-2758.2019.02.008
	QU Y H , ZHANG F , GU R N , et al. Target cooperative location method of multi-UAV based on pseudo range measurement[J]. Journal of Northwestern Polytechnical University, 2019, 37 (2): 266- 272. doi: 10.3969/j.issn.1000-2758.2019.02.008
17	MAZA I, KONDAK K, BERNARD M, et al. Multi-UAV coope- ration and control for load transportation and deployment[C]//Proc. of the 2nd International Symposium on UAVs, 2009: 417-449.
18	KOOHIFAR F , KUMBHAR A , GUVENC I . Receding horizon multi-UAV cooperative tracking of moving RF source[J]. IEEE Communications Letters, 2016, 21 (6): 1433- 1436.
19	ZHAO Y Y , WANG X , WANG C C , et al. Systemic design of distributed multi-UAV cooperative decision-making for multi-target tracking[J]. Autonomous Agents and Multi-Agent Systems, 2019, 33 (2): 132- 158.
20	王轶, 方英武. 基于DST-PCR5多目标自适应视觉跟踪方法[J]. 计算机应用研究, 2018, 35 (1): 632- 636.
	WANG Y , FANG Y W . Adaptive visual tracking method of multi-targets based on DST-PCR5[J]. Application Research of Computers, 2018, 35 (1): 632- 636.
21	JOUSSELME A L , GRENIER D , BOSSÉ É . A new distance between two bodies of evidence[J]. Information Fusion, 2001, 2 (2): 91- 101. doi: 10.1016/S1566-2535(01)00026-4
22	KONG W Z , HONG J C , JIA M Y , et al. YOLOv3-DPFIN: a dual-path feature fusion neural network for robust real-time sonar target detection[J]. IEEE Sensors Journal, 2019, 20 (7): 3745- 3756.
23	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[C]// Proc. of the Advances in Neural Information Processing Systems, 2015: 91-99.
24	LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot multibox detector[C]//Proc. of the European Conference on Computer Vision, 2016: 21-37.
25	RUBLEE E, RABAUD V, KONOLIGE K, et al. ORB: an efficient alternative to SIFT or SURF[C]//Proc. of the International Conference on Computer Vision, 2012: 2564-2571.

检测框	可能情况
检测框	Ⅰ	Ⅱ	Ⅲ	Ⅳ	Ⅴ	Ⅵ	Ⅶ
L_b	1	0	0	0	1	1	1
M_b	0	1	0	1	1	0	1
R_b	0	0	1	1	0	1	1

类别	分类0	分类1	分类2	分类3	分类4	分类5	分类6
UAV-L-score	0.507	0.180	0.050	0.010	0.081	0.001	0.170
UAV-M-score	0.619	0.107	0.074	0.008	0.081	0.001	0.108
UAV-R-score	0.701	0.088	0.065	0.004	0.043	0.001	0.097
D-score	-	-	-	0.163, 弱冲突条件	-	-	-
DST-score	0.982	0.008	0.011	1.4×10^-6	0.001	4.5×10^-9	0.008
PCR5-score	0.837	0.062	0.018	0.003	0.021	5.2×10^-6	0.061

类别	分类0	分类1	分类2	分类3	分类4	分类5	分类6
UAV-L-score	0.507	0.180	0.050	0.010	0.081	0.001	0.170
UAV-M-score	0.019	0.707	0.074	0.008	0.081	0.001	0.108
UAV-R-score	0.701	0.088	0.065	0.004	0.043	0.001	0.097
D-score	-	-	-	0.652, 强冲突条件	-	-	-
DST-score	0.333	0.553	0.012	1.5×10^-5	0.014	4.9×10^-8	0.088
PCR5-score	0.520	0.386	0.017	2.7×10^-4	0.020	4.8×10^-6	0.057

算法	输入	基础网络	mAP/%	时间/ms
Faster R-CNN	600	Resnet-50	68.32	137.4
SSD	512	Vgg-16	63.29	57.8
YOLO v3	416	Darknet-53	66.86	62.4
Retinanet	608	Resnet-50	69.78	84.6
本文算法	608	Resnet-50	72.65	86.7

方案	匹配点对数目	均方根误差	计算时间/s
SIFT+BF	132	1.32	0.351
SURF+BF	125	2.25	0.196
ORB^[21]+BF	98	3.72	0.105
SIFT+KNN	126	1.83	0.311
SURF+KNN	120	2.56	0.153
ORB+KNN	45	5.47	0.052