基于无锚框的红外多类别多目标实时跟踪网络

doi:10.12305/j.issn.1001-506X.2022.02.06

Abstract

Abstract:

In this paper, an improved real-time infrared multi-class multi-target tracking network is proposed, which not only ensures the tracking accuracy, but also redesigns the anchor-free network structure to further reduce the number of network parameters and the network inference latency. By optimizing the target feature vector, the recognition accuracy is further improved, and the tracking process is simplified and developed. In addition, through detailed analysis of the execution time of related processes, GPU and CPU are selected to perform the optimal operation respectively to improve the overall tracking speed. The above method is applied to the low altitude sea surface infrared target tracking dataset. The results show that under the comprehensive evaluation metric proposed in this paper, the score is increased by 1.78 and the running speed of the algorithm reaches 52.37 FPS in NVIDIA Jetson Xavier NX compared with other lightweight network, which meets the real-time operation requirements of edge devices.

Key words: multi-target tracking, anchor-free, infrared target, multi-class, real time in edge devices, re-identification

CLC Number:

TP391.4

Zizhuang SONG, Jiawei YANG, Dongfang ZHANG, Shiqiang WANG, Shuo ZHANG. Real-time infrared multi-class multi-target anchor-free tracking network[J]. Systems Engineering and Electronics, 2022, 44(2): 401-409.

Figures/Tables 14

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Table 1

Fig.6

Table 2

Table 3

Table 4

Table 5

Table 6

Table 7

Fig.7

References 31

1	王法胜, 鲁明羽, 赵清杰, 等. 粒子滤波算法[J]. 计算机学报, 2014, 37 (8): 1679- 1694.
	WANG F S , LU M Y , ZHAO Q J , et al. Particle filtering algorithm[J]. Chinese Journal of Computers, 2014, 37 (8): 1679- 1694.
2	COLLINS R T. Mean-shift blob tracking through scale space[C]// Proc. of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. DOI: 10.1109/CVPR.2003.1211475.
3	WELCH G, BISHOP G. An introduction to the Kalman filter[C]// Proc. of the ACM Special Interest Group on Computer Graphics and Interactive Techniques, 2001.
4	BOLME D S, BEVERIDGE J R, DRAPER B A, et al. Visual object tracking using adaptive correlation filters[C]//Proc. of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2010: 2544-2550.
5	HENRIQUES J F, CASEIRO R, MARTINS P, et al. Exploiting the circulant structure of tracking-by-detection with kernels[C]// Proc. of the European Conference on Computer Vision, 2012: 702-715.
6	HENRIQUES J F , CASEIRO R , MARTINS P , et al. High-speed tracking with kernelized correlation filters[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2014, 37 (3): 583- 596.
7	杜若鹏, 张磊, 卢杨. 上下文感知相关滤波的红外目标跟踪改进算法[J]. 激光与红外, 2020, 50 (7): 839- 845.
	DU R P , ZHANG L , LU Y . Improved infrared target tracking algorithm based on context-aware correlation filter[J]. Laser & Infrared, 2020, 50 (7): 839- 845.
8	DANELLJAN M, ROBINSON A, SHAHBAZ K F, et al. Beyond correlation filters: learning continuous convolution operators for visual tracking[C]//Proc. of the European Conference on Computer Vision, 2016: 472-488.
9	王承赟, 张龙杰, 李相民, 等. 基于改进的核相关滤波算法的红外目标跟踪[J]. 电光与控制, 2021, 28 (7): 9- 13.
	WANG C Y , ZHANG L J , LI X M , et al. Infrared target tracking based on improved kernel correlation filter algorithm[J]. Electronics Optics & Control, 2021, 28 (7): 9- 13.
10	宋国鹏. 基于孪生网络的单目标跟踪算法研究及其应用[D]. 北京: 北京交通大学, 2020.
	SONG G P. Research and application of single target tracking algorithm based on siamese network[D]. Beijing: Beijing Jiaotong University, 2020.
11	柳赟, 孙淑艳. 基于自适应模板更新的改进孪生卷积网络目标跟踪算法[J]. 计算机应用与软件, 2021, 38 (4): 145- 151, 230. doi: 10.3969/j.issn.1000-386x.2021.04.024
	LIU Y , SUN S Y . Object tracking algorithm based on improved siamese convolutional networks combined with adaptive template updating[J]. Computer Applications and Software, 2021, 38 (4): 145- 151, 230. doi: 10.3969/j.issn.1000-386x.2021.04.024
12	DANELLJAN M, BHAT G, SHAHBAZ K F, et al. Eco: efficient convolution operators for tracking[C]//Proc. of the Conference on Computer Vision and Pattern Recognition, 2017: 6638-6646.
13	BERTINETTO L, VALMADRE J, HENRIQUES J F, et al. Fully-convolutional siamese networks for object tracking[C]//Proc. of the European Conference on Computer Vision, 2016: 850-865.
14	LI B, YAN J, WU W, et al. High performance visual tracking with siamese region proposal network[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 8971-8980.
15	ZHU Z, WANG Q, LI B, et al. Distractor-aware siamese networks for visual object tracking[C]//Proc. of the European Conference on Computer Vision, 2018: 101-117.
16	LI B, WU W, WANG Q, et al. SiamRPN++: evolution of siamese visual tracking with very deep networks[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 4282-4291.
17	BEWLEY A, GE Z, OTT L, et al. Simple online and realtime tracking[C]//Proc. of the IEEE International Conference on Image Processing, 2016: 3464-3468.
18	WOJKE N, BEWLEY A, PAULUS D. Simple online and realtime tracking with a deep association metric[C]//Proc. of the IEEE International Conference on Image Processing, 2017: 3645-3649.
19	WANG Z D, ZHENG L, LIU Y X, et al. Towards real-time multi-object tracking[C]// Proc. of the European Conference on Computer Vision, 2020: 107-122.
20	ZHANG Y F , WANG C Y , WANG X G , et al. FairMOT: on the fairness of detection and re-identification in multiple object tracking[J]. arXiv preprint, 2020,
21	ZHOU X , WANG D . KRÄHENBVHL P. Objects as points[J]. arXiv preprint, 2019, arXiv, 1904.07850.
22	李震霄, 孙伟, 刘明明, 等. 交通监控场景中的车辆检测与跟踪算法研究[J]. 计算机工程与应用, 2021, 57 (8): 103- 111.
	LI Z X , SUN W , LIU M M , et al. Research on vehicle detection and tracking algorithms in traffic monitoring scenes[J]. Computer Engineering and Applications, 2021, 57 (8): 103- 111.
23	赵朵朵, 章坚武, 傅剑峰. 基于深度学习的实时人流统计方法研究[J]. 传感技术学报, 2020, 33 (8): 1161- 1168. doi: 10.3969/j.issn.1004-1699.2020.08.013
	ZHAO D D , ZHANG J W , FU J F . Research on real-time statistics of people flow based on deep learning[J]. Chinese Journal of Sensors and Actuators, 2020, 33 (8): 1161- 1168. doi: 10.3969/j.issn.1004-1699.2020.08.013
24	张宏鸣, 汪润, 董佩杰, 等. 基于DeepSORT算法的肉牛多目标跟踪方法[J]. 农业机械学报, 2021, 52 (4): 248- 256.
	ZHANG H M , WANG R , DONG P J , et al. Beef cattle multi-target tracking based on DeepSORT algorithm[J]. Transactions of the Chinese Society for Agricultural Machinery, 2021, 52 (4): 248- 256.
25	MILAN A , LEAL-TAIXÉ L , REID I , et al. MOT16: a benchmark for multi-object tracking[J]. arXiv preprint, 2016, arXiv, 1603.00831.
26	DING X, ZHANG X, MA N, et al. RepVGG: making VGG-style ConvNets great again[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 13733-13742.
27	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.
28	SIMONYAN K , ZISSERMAN A . Very deep convolutional networks for large-scale image recognition[J]. arXiv preprint, 2014, arXiv, 1409.1556.
29	LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 2117-2125.
30	LUO H, GU Y, LIAO X, et al. Bag of tricks and a strong baseline for deep person re-identification[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2019.
31	KENDALL A, GAL Y, CIPOLLA R. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 7482-7491.

处理单元	FP	RD	TT	总时间
GPU+CPU	28.26	81.39	7.09	116.74
GPU	28.26	48.93	17.56	94.75

网络结构	MOTA	IDs	IDF1	MT	ML	Score
ResNet18+DCN	59.52	361	63.15	38/71	10/71	65.53
ResNet18+DFPN	63.47	306	65.64	40/71	9/71	68.19
HRNet18	66.51	227	68.81	41/71	7/71	70.80
DLA34	67.18	221	69.26	42/71	7/71	71.43
RepVGG+DCN(Deploy)	62.03	284	64.04	38/71	9/71	66.73
RepVGG+FFPN(Train)	66.24	267	66.72	40/71	8/71	69.51
RepVGG+FFPN(Deploy)	66.24	267	66.72	40/71	8/71	69.51

网络结构	参数量 (×10⁶)	计算复杂度/ GFLOPs	网络推理时间/ms
ResNet18+DCN	15.46	58.74	28.68
ResNet18+DFPN	16.72	60.24	35.08
DLA34	19.84	72.12	63.89
HRNet18	11.62	127.54	184.12
RepVGG+DCN(Deploy)	13.51	63.99	32.62
RepVGG+FFPN(Train)	13.02	61.06	47.32
RepVGG+FFPN(Deploy)	11.91	57.64	28.26

改进方法	MOTA	IDs	IDF1	MT	ML	Score
CE	66.24	267	66.72	40/71	8/71	69.51
CE+Triplet	67.01	232	68.44	41/71	6/71	71.19
LSCE+Triplet	67.17	226	69.54	42/71	6/71	71.85
LSCE+Triplet+Center	67.24	190	70.18	42/71	6/71	72.03
LSCE+Triplet+Center+BNNeck	68.23	176	71.56	42/71	6/71	72.62

跟踪方法	MOTA	IDs	IDF1	MT	ML	Score	TT/ms
特征向量+IoU+卡尔曼滤波	62.19	6 952	17.08	35/71	8/71	54.32	25.65
特征向量+IoU	68.23	176	71.56	42/71	6/71	72.62	7.09
特征向量	69.18	266	66.51	44/71	5/71	72.65	5.83
改进特征向量(C++)	69.73	253	67.45	46/71	5/71	73.73	1.29

Real-time infrared multi-class multi-target anchor-free tracking network

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 14

References 31

Related Articles 15

Recommended Articles

Metrics

Comments

部署方法	LI/ms	IP/ms	FP/ms	RD/ms	TT/ms	帧率/FPS
Train	1.56	2.85	47.32	83.29	5.83	7.10
Deploy	1.56	2.85	28.26	83.29	5.83	8.21
Deploy+TensorRT(FP16)	1.56	2.85	10.66	32.26	5.83	18.81
Deploy+TensorRT(FP16)+C++	1.56	2.85	10.66	2.73	1.29	52.37

[1]	Xiaofeng ZHAO, Yebin XU, Fei WU, Jiahui NIU, Wei CAI, Zhili ZHANG. Ground infrared target detection method based on global sensing mechanism [J]. Systems Engineering and Electronics, 2022, 44(5): 1461-1467.
[2]	Lifan YIN, Yiqun ZHANG, Shuo WANG, Chenggang SUN. A survey on histogram probabilistic multi-hypothesis tracker technique [J]. Systems Engineering and Electronics, 2021, 43(11): 3118-3125.
[3]	Zhengjie LI, Junwei XIE, Haowei ZHANG, Zhaojian ZHANG. Joint power and bandwidth allocation algorithm based on collocated MIMO radar [J]. Systems Engineering and Electronics, 2020, 42(5): 1041-1049.
[4]	Xiaodong LU, Tao CUI, Wei WANG, Cheng CHENG. Multi-target labeled multi-Bernoulli filter with time-delay and registration errors [J]. Systems Engineering and Electronics, 2020, 42(4): 904-911.
[5]	PENG Huafu, HUANG Gaoming, TIAN Wei, QIU Hao. Labeled multi-Bernoulli filter based on amplitude information [J]. Systems Engineering and Electronics, 2018, 40(12): 2636-2641.
[6]	LIU Jianye, WANG Hua, ZHOU Wanmeng. LEO constellation sensor resources scheduling algorithm based on Genetic and Simulated annealing [J]. Systems Engineering and Electronics, 2018, 40(11): 2476-.
[7]	GU Xiaolin, ZHOU Shilin, LEI Lin. Multi-target tracking based on m-best data association and tracklet associatio [J]. Systems Engineering and Electronics, 2017, 39(7): 1640-1646.
[8]	YUAN Changshun, WANG Jun, XIANG Hong, SUN Jinping. Adaptive δ-GLMB filtering algorithm based on VB approximation [J]. Systems Engineering and Electronics, 2017, 39(2): 237-243.
[9]	LEI Lei, WANG Xiaodan, QUAN Wen, LUO Xi . Non-competence reliability in multi-classification based on error correcting output codes [J]. Systems Engineering and Electronics, 2017, 39(12): 2637-2645.
[10]	ZHANG Jinli, LI Min. Infrared target tracking based on matrix S_1/2 norm [J]. Systems Engineering and Electronics, 2017, 39(10): 2177-2183.
[11]	CHEN Hang, ZHANG Bo-yan, CHEN Ying. Multiple hypothesis tracking with adaptive association depth [J]. Systems Engineering and Electronics, 2016, 38(9): 2000-2007.
[12]	YUAN Changshun, WANG Jun, LEI Peng, SUN Jinping, BI Yanxian. Multi-target tracking based on PHD filter for phased array radar [J]. Systems Engineering and Electronics, 2016, 38(3): 539-544.
[13]	LU Hong, LI Hong-sheng, FEI Shu-min, CHENG Yong. Block level saliency centroid representation and multi-level association based multi-target tracking [J]. Systems Engineering and Electronics, 2015, 37(9): 2182-2190.
[14]	LEI Lei, WANG Xiao-dan, LUO Xi, SONG Ya-fei. Hierarchical error correcting output codes based on SVDD [J]. Systems Engineering and Electronics, 2015, 37(8): 1916-1921.
[15]	ZHU You-qing, ZHOU Shi-lin. Classification-aided GM-PHD filter based on signal feature of radar emitter [J]. Systems Engineering and Electronics, 2015, 37(6): 1273-1279.