基于相关滤波的视觉目标跟踪算法新进展

doi:10.12305/j.issn.1001-506X.2021.08.05

Abstract

Abstract:

With excellent comprehensive performance, correlation filter-based tracking algorithms have become a hotspot of the theoretical research and practical application in the field of visual object tracking. Despite many studies, there is still a lack of systematic analyses on the existing correlation filter-based tracking algorithms from the level of tracking framework. Therefore, starting from the basic framework of object tracking algorithms, the characteristics of correlation filter-based tracking algorithms are deeply analyzed, and their basic problems in each working stage are presented in this paper. On this basis, the main technological progress of correlation filter-based tracking algorithms and characteristics of corresponding algorithms in recent ten years are summarized, and 20 typical correlation filter-based tracking algorithms are evaluated and analyzed. Finally, the outstanding issues to be urgently solved and the future research directions of correlation filter-based tracking algorithms are discussed.

Key words: computer vision, object tracking, correlation filter, deep learning

CLC Number:

TP391

Yueping HUANG, Xiaofeng LI, Xiaogang YANG, Naixin QI, Ruitao LU, Shengxiu ZHANG. Advances in visual object tracking algorithm based on correlation filter[J]. Systems Engineering and Electronics, 2021, 43(8): 2051-2065.

Figures/Tables 9

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Table 1

Fig.8

References 72

1	ZHU Y, MOTTAGHI R, KOLVE E, et al. Target-driven visual navigation in indoor scenes using deep reinforcement learning[C]//Proc. of the International Conference on Robotics and Automation, 2017: 3357-3364.
2	DUAN X, XIE S Y, MENG Y, et al. Brain computer integration controlled unmanned vehicle for target reconnaissance[C]//Proc. of the IEEE International Conference on Unmanned Systems, 2019: 9-35.
3	BUCH N , VELASTIN S A , ORWELL J . A review of computer vision techniques for the analysis of urban traffic[J]. IEEE Trans.on Intelligent Transportation Systems, 2011, 12 (3): 920- 939. doi: 10.1109/TITS.2011.2119372
4	RAUTARAY S , AGRAWAL A . Vision based hand gesture re-cognition for human computer interaction: a survey[J]. Artificial Intelligence Review, 2015, 43 (1): 1- 54. doi: 10.1007/s10462-012-9356-9
5	ALLGIECK T, MAGNOR M, XU W, et al. Video based reconstruction of 3D people models[C]//Proc. of the IEEE International Conference on Computer Vision and Pattern Recognition, 2018: 8387-8397.
6	卢湖川, 李佩霞, 王栋. 目标跟踪算法综述[J]. 模式识别与人工智能, 2018, 31 (1): 61- 76.
	LU H C , LU P X , WANG D . Visual object tracking: a survey[J]. Pattern Recognition and Artificial Intelligence, 2018, 31 (1): 61- 76.
7	WANG N Y, SHI J P, YEUNG D, et al. Understanding and diagnosing visual tracking systems[C]//Proc. of the International Conference on Computer Vision, 2015: 3101-3109.
8	WU Y, LIM J, YANG M H. Online object tracking: a benchmark[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2013: 2411-2418.
9	LI Y , AI H , YAMASHITA T , et al. Tracking in low frame rate video: a cascade particle filter with discriminative observers of different life spans[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2017, 30 (10): 1728- 1740.
10	NOOR K, SIDDQUEE E, SARMA D, et al. Performance analysis of a surveillance system to detect and track vehicles using Haar cascaded classifiers and optical flow method[C]//Proc. of the IEEE Conference on Industrial Electronics and Applications, 2017: 258-263.
11	COMANICIU D , MEER P , INTELLIGENCE M . Mean shift: a robust approach toward feature space analysis[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2002, 24 (5): 603- 619. doi: 10.1109/34.1000236
12	PARAGIOS N , DERICHE R . Geodesic active regions and level set methods for supervised texture segmentation[J]. International Journal of Computer Vision, 2002, 46 (3): 223- 247. doi: 10.1023/A:1014080923068
13	MIN W D , FAN M D , GUO X G , et al. A new approach to track multiple vehicles with the combination of robust detection and two classifiers[J]. IEEE Trans.on Intelligent Transportation Systems, 2017, 19 (1): 174- 186.
14	BABENKO B, YANG M H, BELONGIE S. Visual tracking with online multiple instance learning[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2009: 983-990.
15	SAFFARI A, LEISTNER C, SANTNER J, et al. On-line random forests[C]//Proc. of the IEEE International Conference on Computer Vision Workshops, 2009: 1393-1400.
16	LIU Q S , YANG J , ZHANG K , et al. Adaptive compressive tracking via online vector boosting feature selection[J]. IEEE Trans.on Cybernetics, 2015, 47 (12): 4289- 4301.
17	BOLME D S, BEVERIDGE J R, DRAPER B A, et al. Visual object tracking using adaptive correlation filters[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2010: 2544-2550.
18	HENRIQUES J F , CASEIRO R , MARTINS P , et al. High-speed tracking with kernelized correlation filters[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2015, 37 (3): 583- 596. doi: 10.1109/TPAMI.2014.2345390
19	HENRIQUES J F, CASEIRO R, MARTINS P, et al. Exploiting the circulant structure of tracking-by-detection with kernels[C]//Proc. of the IEEE European Conference on Computer Vision, 2012: 702-715.
20	HARE S , GOLODETZ S , SAFFARI A , et al. Struck: structured output tracking with kernels[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2016, 38 (10): 2096- 2109. doi: 10.1109/TPAMI.2015.2509974
21	KALAL Z , MIKOLAJCZYK K , MATAS J . Tracking-learning-detection[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2012, 34 (7): 1409- 1422. doi: 10.1109/TPAMI.2011.239
22	FAN H, XIANG J H. Robust visual tracking via local-global correlation filter[C]//Proc. of the National Conference on Artificial Intelligence, 2017: 4025-4031.
23	DANELLJAN M, HAGER G, KHAN F S, et al. Adaptive decontamination of the training set: a unified formulation for discriminative visual tracking[C]//Proc. of the Computer Vision and Pattern Recognition, 2016: 1430-1438.
24	ZHANG T, XU C, YANG M H. Multi-task correlation particle filter for robust object tracking[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 725-732.
25	WU Y , LIM J , YANG M H . Object tracking benchmark[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2015, 37 (9): 1834- 1848. doi: 10.1109/TPAMI.2014.2388226
26	孟琭, 杨旭. 目标跟踪算法综述[J]. 自动化学报, 2019, 45 (7): 1244- 1260.
	MENG L , YANG X . A survey of object tracking algorithms[J]. Acta Automatica Sinica, 2019, 45 (7): 1244- 1260.
27	GRAY R M . Toeplitz and circulant matrices[J]. Foundations and Trends in Communications and Information Theory, 2005, 2 (3): 155- 239. doi: 10.1561/0100000006
28	XUE H , CHEN S C , YANG Q . Discriminatively regularized least-squares classification[J]. Pattern Recognition, 2009, 42 (1): 93- 104. doi: 10.1016/j.patcog.2008.07.010
29	GALOOGAHI H K, SIM T, LUCEY S. Correlation filters with limited boundaries[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2015: 4630-4638.
30	BIBI A, MUELLER M, GHANEM B. Target response adaptation for correlation filter tracking[C]//Proc. of the IEEE European Conference on Computer Vision, 2016: 419-433.
31	DANELLJAN M, HAGER G, KHAN F S, et al. Learning spatially regularized correlation filters for visual tracking[C]//Proc. of the IEEE International Conference on Computer Vision, 2015: 4310-4318.
32	DANELLJAN M, ROBINSON A, KHAN F S, et al. Beyond correlation filters: learning continuous convolution operators for visual tracking[C]//Proc. of the IEEE European Conference on Computer Vision, 2016: 472-488.
33	DANELLJAN M, BHAT G, KHAN F S, et al. ECO: efficient convolution operators for tracking[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 6931-6939.
34	LI F, TIAN C, ZUO W M, et al. Learning spatial-temporal regularized correlation filters for visual tracking[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 4904-1913.
35	DAI K, WANG D, LU H C, et al. Visual tracking via adaptive spatially-regularized correlation filters[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2019: 4670-4679.
36	LI Y M, FU C H, DING F Q, et al. Autotrack: towards high-performance visual tracking for uav with automatic spatio-temporal regularization[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2020: 11920-11929.
37	GALOOGAHI H K, FAGG A, LUCEY S. Learning background-aware correlation filters for visual tracking[C]//Proc. of the IEEE International Conference on Computer Vision, 2017: 1144-1152.
38	LUKEZIC A, VOJIR T, ZAJC L C, et al. Discriminative correlation filter with channel and spatial reliability[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 4847-4856.
39	VAN D W J , SCHMID C , VERBEEK J , et al. Learning color names for real-world applications[J]. IEEE Trans.on Image Processing, 2009, 18 (7): 1512- 1523. doi: 10.1109/TIP.2009.2019809
40	DANELLJAN M, SHAHBAZ K F, FELSBERG M, et al. Adaptive color attributes for real-time visual tracking[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2014: 1090-1097.
41	MA C, YANG X K, ZHANG C Y, et al. Long-term correlation tracking[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2015: 5388-5396.
42	MA C, HUANG J B, YANG X, et al. Hierarchical convolutional features for visual tracking[C]//Proc. of the IEEE International Conference on Computer Vision, 2015: 3074-3082.
43	ZHANG Y, CHAN W, JAITLY N, et al. Very deep convolutional networks for end-to-end speech recognition[C]//Proc. of the International Conference on Acoustics, Speech, and Signal Processing, 2017: 4845-4849.
44	BHAT G, JOHNANDER J, DANELLJAN M, et al. Unveiling the power of deep tracking[C]//Proc. of the IEEE European Conference on Computer Vision, 2018: 493-509.
45	GUNDOGDU E , ALATAN A A . Good features to correlate for visual tracking[J]. IEEE Trans.on Image Processing, 2018, 27 (5): 2526- 2540. doi: 10.1109/TIP.2018.2806280
46	WANG N, SONG Y B, MA C, et al. Unsupervised deep tracking[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2019: 1308-1317.
47	XU T, FENG Z H, WU X J, et al. Joint group feature selection and discriminative filter learning for robust visual object tracking[C]//Proc. of the IEEE International Conference on Computer Vision, 2019: 7950-7960.
48	WANG N , ZHOU W G , SONG Y B , et al. Real-time correlation tracking via joint model compression and transfer[J]. IEEE Trans.on Image Processing, 2020, 29, 6123- 6135. doi: 10.1109/TIP.2020.2989544
49	DANELLJAN M, HAGER G, KHAN F, et al. Accurate scale estimation for robust visual tracking[C]//Proc. of the British Machine Vision Conference, 2014: 123-135.
50	LI Y, ZHU J K. A scale adaptive kernel correlation filter tracker with feature integration[C]//Proc. of the IEEE European Conference on Computer Vision, 2014: 254-265.
51	ZHANG S L , LU W , XING W W , et al. Learning scale-adaptive tight correlation filter for object tracking[J]. IEEE Trans.on Cybernetics, 2020, 50 (1): 270- 283. doi: 10.1109/TCYB.2018.2868782
52	LIU T, GANG W, YANG Q X. Real-time part-based visual tracking via adaptive correlation filters[C]//Proc. of the IEEE International Conference on Computer Vision Workshop, 2015: 587-594.
53	MONTERO A S, LANG J, LAGANIERE R. Scalable kernel correlation filter with sparse feature integration[C]//Proc. of the IEEE International Conference on Computer Vision Workshop, 2015: 587-594.
54	TANG M, YU B, ZHANG F, et al. High-speed tracking with multi-kernel correlation filters[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 4874-4883.
55	WANG M M, LIU Y, HUANG Z. Large margin object trac-king with circulant feature maps[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 4800-4808.
56	WANG N, ZHOU W G, TIAN Q, et al. Multi-cue correlation filters for robust visual tracking[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 4844-4853.
57	MA H , ACTON S T , LIN Z . SITUP: scale invariant tracking using average peak-to-correlation energy[J]. IEEE Trans.on Image Processing, 2020, 29, 3546- 3557. doi: 10.1109/TIP.2019.2962694
58	谢瑜, 陈莹. 通道裁剪下的多特征组合目标跟踪算法[J]. 系统工程与电子技术, 2020, 42 (4): 764- 772.
	XIE Y , CHEN Y . Multi-feature combined target tracking algorithm based on channel clipping[J]. Systems Engineering and Electronics, 2020, 42 (4): 764- 772.
59	BERTINETTO L, VALMADRE J, GOLODETZ S, et al. Staple: complementary learners for real-time tracking[C]//Proc. of the IEEE International Conference on Computer Vision and Pattern Recognition, 2016: 1401-1409.
60	ZUO W M , WU X H , LIN L , et al. Learning support correlation filters for visual tracking[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2019, 41 (5): 1158- 1171. doi: 10.1109/TPAMI.2018.2829180
61	XUAN S Y , LI S Y , HAN M F , et al. Object tracking in satellite videos by improved correlation filters with motion estimations[J]. IEEE Trans.on Geoscience and Remote Sensing, 2020, 58 (2): 1074- 1086. doi: 10.1109/TGRS.2019.2943366
62	SONG Y B, MA C, GONG L J, et al. CREST: convolutional residual learning for visual tracking[C]//Proc. of the IEEE International Conference on Computer Vision, 2017: 2555-2564.
63	WANG Q, GAO J, XING J L, et al. Dcfnet: discriminant correlation filters network for visual tracking[EB/OL]. [2020-06-16]. http://arxiv.org/abs/1704.04057vl.
64	BERTINETTO L, VALMADRE J, HENRIQUES J F, et al. Fully-convolutional siamese networks for object tracking[C]//Proc. of the IEEE European Conference on Computer Vision, 2016: 850-865.
65	CHOI J W, CHANG H J, YUN S D, et al. Attentional correlation filter network for adaptive visual tracking[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 4828-4837.
66	EUNBYUNG P, ALEXANDER C B. Meta-tracker: fast and robust online adaptation for visual object trackers[C]//Proc. of the IEEE European Conference on Computer Vision, 2018: 534-550.
67	ZHANG M D, WANG Q, XING J L, et al. Visual tracking via spatially aligned correlation filters network[C]//Proc. of the IEEE European Conference on Computer Vision, 2018: 469-485.
68	ZHANG H L , CHEN J , NIE G H , et al. Uncertain motion tracking based on convolutional net with semantics estimation and region proposals[J]. Pattern Recognition, 2020, 102, 107- 232.
69	REN S Q , HE K M , GIRSHICK R , et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2017, 39 (6): 1137- 1149. doi: 10.1109/TPAMI.2016.2577031
70	JAIN M , SUBRAMANYAM A V , DENMAN S , et al. LSTM guided ensemble correlation filter tracking with appearance model pool[J]. Computer Vision and Image Understanding, 2020, 195, 102935. doi: 10.1016/j.cviu.2020.102935
71	SAINATH T N, VINYALS O, SENIOR A W, et al. Convolutional, long short-term memory, fully connected deep neural networks[C]//Proc. of the International Conference on Acoustics, Speech, and Signal Processing, 2015: 4580-4584.
72	VALMADRE J, BERTINETTO L, HENRIQUES J F, et al. Long-term tracking in the wild: a benchmark[C]//Proc. of the IEEE European Conference on Computer Vision, 2018: 692-707.

算法	精确度	AUC	运行速度/fps	特征类型
CSK	0.518	0.382	304.815	Gray
CN	0.594	0.422	57.422	Gray+CN
DSST	0.693	0.470	23.268	HOG
KCF	0.696	0.477	140.465	HOG
SAMF	0.753	0.553	14.334	Gray+HOG+CN
CF2	0.845	0.603	10.624	CNN
CFLB	0.457	0.333	265.977	Gray
SRDCF	0.788	0.597	5.637	HOG
CCOT	0.896	0.667	1.302	CNN+HOG+CN
Staple	0.776	0.576	56.876	HOG+CH
CREST	0.839	0.624	9.064	CNN
BACF	0.816	0.615	25.339	HOG
CSR_DCF	0.909	0.675	13.029	HOG+CN
DCFNet	0.818	0.626	34.118	CNN
ECO	0.909	0.687	12.758	CNN+HOG+CN
ECO_HC	0.840	0.631	52.496	HOG+CN
MKCFup	0.760	0.586	49.835	HOG+CN
MCCT	0.917	0.682	7.712	CNN+HOG
MCCT_H	0.848	0.636	36.302	Gray+HOG+CN
STRCF	0.864	0.654	15.391	CNN+HOG+CN
ASRCF	0.919	0.689	16.516	CNN+HOG
GFS_DCF	0.931	0.693	1.377	CNN+HOG+ CN+IC

[1]	Xiao HAN, Shiwen CHEN, Meng CHEN, Jincheng YANG. Open-set recognition of LPI radar signal based on reciprocal point learning [J]. Systems Engineering and Electronics, 2022, 44(9): 2752-2759.
[2]	Zhuling QIU, Yufei ZHA, Zhenyu LI, Yuming LI, Peng ZHANG, Chuan ZHU. Temporal regularized correlation filter tracking algorithm based on multi-model distillation [J]. Systems Engineering and Electronics, 2022, 44(8): 2448-2456.
[3]	Limin ZHANG, Kaiwen TAN, Wenjun YAN, Yuyuan ZHANG. Radar emitter recognition based on multi-level jumper residual network [J]. Systems Engineering and Electronics, 2022, 44(7): 2148-2156.
[4]	Guodong JIN, Yuanliang XUE, Lining TAN, Jiankun XU. Advances in object tracking algorithm based on siamese network [J]. Systems Engineering and Electronics, 2022, 44(6): 1805-1822.
[5]	Xiaofeng ZHAO, Yebin XU, Fei WU, Jiahui NIU, Wei CAI, Zhili ZHANG. Ground infrared target detection method based on global sensing mechanism [J]. Systems Engineering and Electronics, 2022, 44(5): 1461-1467.
[6]	Hong ZOU, Chenyang BAI, Peng HE, Yaping CUI, Ruyan WANG, Dapeng WU. Edge service placement strategy based on distributed deep learning [J]. Systems Engineering and Electronics, 2022, 44(5): 1728-1737.
[7]	Dong CHEN, Yanwei JU. Ship object detection SAR images based on semantic segmentation [J]. Systems Engineering and Electronics, 2022, 44(4): 1195-1201.
[8]	Jingming SUN, Shengkang YU, Jun SUN. Pose sensitivity analysis of HRRP recognition based on deep learning [J]. Systems Engineering and Electronics, 2022, 44(3): 802-807.
[9]	Yunxiang YAO, Ying CHEN. Target tracking network based on dual-modal interactive fusion under attention mechanism [J]. Systems Engineering and Electronics, 2022, 44(2): 410-419.
[10]	Yongxing GAO, Xudong WANG, Ling WANG, Daiyin ZHU, Jun GUO, Fanwang MENG. Weather signal detection for dual polarization weather radar based on RCNN [J]. Systems Engineering and Electronics, 2022, 44(11): 3380-3387.
[11]	Yiheng ZHOU, Jun YANG, Saiqiang XIA, Mingjiu LYU. Estimation method of micro-motion parameters for rotor targets under flashing [J]. Systems Engineering and Electronics, 2022, 44(1): 54-63.
[12]	Rong FU, Tianyao HUANG, Yimin LIU. DNN based 1-bit block sparse recovery in frequency agile coherent radar [J]. Systems Engineering and Electronics, 2022, 44(1): 70-75.
[13]	Yuyuan ZHANG, Limin ZHANG, Wenjun YAN. SFBC-OFDM recognition method based on cross-correlation feature map and dilated dense convolutional neural networks [J]. Systems Engineering and Electronics, 2021, 43(9): 2657-2664.
[14]	Caiyun WANG, Yangyu LI, Xiaofei LI, Jianing WANG, Wenyi WEI. Aerial image super-resolution restruction based on sparsity and deep learning [J]. Systems Engineering and Electronics, 2021, 43(8): 2045-2050.
[15]	Bin WANG, Guoyu WANG. Instantaneous coastline automatic extraction algorithm for SAR images based on improved deep learning network [J]. Systems Engineering and Electronics, 2021, 43(8): 2108-2115.

Advances in visual object tracking algorithm based on correlation filter

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 72

Related Articles 15

Recommended Articles

Metrics

Comments