基于相关滤波的视觉目标跟踪算法新进展

doi:10.12305/j.issn.1001-506X.2021.08.05

摘要/Abstract

摘要：

相关滤波跟踪算法凭借其优良的综合性能, 已成为视觉目标跟踪领域中理论研究和实践应用的热点。尽管目前已有大量研究, 但仍缺乏从跟踪框架层面对现有相关滤波跟踪算法的系统分析。因此, 文中从视频目标跟踪算法基本框架出发, 深入剖析了相关滤波跟踪算法的特性以及各工作阶段存在的基本问题。以此为依据, 归纳总结了近十年来其主要技术新进展及相应算法特点, 并对20种典型相关滤波跟踪算法进行了测试与分析。最后, 给出了相关滤波跟踪算法亟待解决的重点问题以及未来可研究方向。

关键词: 计算机视觉, 目标跟踪, 相关滤波, 深度学习

Abstract:

With excellent comprehensive performance, correlation filter-based tracking algorithms have become a hotspot of the theoretical research and practical application in the field of visual object tracking. Despite many studies, there is still a lack of systematic analyses on the existing correlation filter-based tracking algorithms from the level of tracking framework. Therefore, starting from the basic framework of object tracking algorithms, the characteristics of correlation filter-based tracking algorithms are deeply analyzed, and their basic problems in each working stage are presented in this paper. On this basis, the main technological progress of correlation filter-based tracking algorithms and characteristics of corresponding algorithms in recent ten years are summarized, and 20 typical correlation filter-based tracking algorithms are evaluated and analyzed. Finally, the outstanding issues to be urgently solved and the future research directions of correlation filter-based tracking algorithms are discussed.

Key words: computer vision, object tracking, correlation filter, deep learning

中图分类号:

TP391

黄月平, 李小锋, 杨小冈, 齐乃新, 卢瑞涛, 张胜修. 基于相关滤波的视觉目标跟踪算法新进展[J]. 系统工程与电子技术, 2021, 43(8): 2051-2065.

Yueping HUANG, Xiaofeng LI, Xiaogang YANG, Naixin QI, Ruitao LU, Shengxiu ZHANG. Advances in visual object tracking algorithm based on correlation filter[J]. Systems Engineering and Electronics, 2021, 43(8): 2051-2065.

图/表 9

图1

图2

图3

图4

图5

图6

图7

表1

图8

参考文献 72

1	ZHU Y, MOTTAGHI R, KOLVE E, et al. Target-driven visual navigation in indoor scenes using deep reinforcement learning[C]//Proc. of the International Conference on Robotics and Automation, 2017: 3357-3364.
2	DUAN X, XIE S Y, MENG Y, et al. Brain computer integration controlled unmanned vehicle for target reconnaissance[C]//Proc. of the IEEE International Conference on Unmanned Systems, 2019: 9-35.
3	BUCH N , VELASTIN S A , ORWELL J . A review of computer vision techniques for the analysis of urban traffic[J]. IEEE Trans.on Intelligent Transportation Systems, 2011, 12 (3): 920- 939. doi: 10.1109/TITS.2011.2119372
4	RAUTARAY S , AGRAWAL A . Vision based hand gesture re-cognition for human computer interaction: a survey[J]. Artificial Intelligence Review, 2015, 43 (1): 1- 54. doi: 10.1007/s10462-012-9356-9
5	ALLGIECK T, MAGNOR M, XU W, et al. Video based reconstruction of 3D people models[C]//Proc. of the IEEE International Conference on Computer Vision and Pattern Recognition, 2018: 8387-8397.
6	卢湖川, 李佩霞, 王栋. 目标跟踪算法综述[J]. 模式识别与人工智能, 2018, 31 (1): 61- 76.
	LU H C , LU P X , WANG D . Visual object tracking: a survey[J]. Pattern Recognition and Artificial Intelligence, 2018, 31 (1): 61- 76.
7	WANG N Y, SHI J P, YEUNG D, et al. Understanding and diagnosing visual tracking systems[C]//Proc. of the International Conference on Computer Vision, 2015: 3101-3109.
8	WU Y, LIM J, YANG M H. Online object tracking: a benchmark[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2013: 2411-2418.
9	LI Y , AI H , YAMASHITA T , et al. Tracking in low frame rate video: a cascade particle filter with discriminative observers of different life spans[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2017, 30 (10): 1728- 1740.
10	NOOR K, SIDDQUEE E, SARMA D, et al. Performance analysis of a surveillance system to detect and track vehicles using Haar cascaded classifiers and optical flow method[C]//Proc. of the IEEE Conference on Industrial Electronics and Applications, 2017: 258-263.
11	COMANICIU D , MEER P , INTELLIGENCE M . Mean shift: a robust approach toward feature space analysis[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2002, 24 (5): 603- 619. doi: 10.1109/34.1000236
12	PARAGIOS N , DERICHE R . Geodesic active regions and level set methods for supervised texture segmentation[J]. International Journal of Computer Vision, 2002, 46 (3): 223- 247. doi: 10.1023/A:1014080923068
13	MIN W D , FAN M D , GUO X G , et al. A new approach to track multiple vehicles with the combination of robust detection and two classifiers[J]. IEEE Trans.on Intelligent Transportation Systems, 2017, 19 (1): 174- 186.
14	BABENKO B, YANG M H, BELONGIE S. Visual tracking with online multiple instance learning[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2009: 983-990.
15	SAFFARI A, LEISTNER C, SANTNER J, et al. On-line random forests[C]//Proc. of the IEEE International Conference on Computer Vision Workshops, 2009: 1393-1400.
16	LIU Q S , YANG J , ZHANG K , et al. Adaptive compressive tracking via online vector boosting feature selection[J]. IEEE Trans.on Cybernetics, 2015, 47 (12): 4289- 4301.
17	BOLME D S, BEVERIDGE J R, DRAPER B A, et al. Visual object tracking using adaptive correlation filters[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2010: 2544-2550.
18	HENRIQUES J F , CASEIRO R , MARTINS P , et al. High-speed tracking with kernelized correlation filters[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2015, 37 (3): 583- 596. doi: 10.1109/TPAMI.2014.2345390
19	HENRIQUES J F, CASEIRO R, MARTINS P, et al. Exploiting the circulant structure of tracking-by-detection with kernels[C]//Proc. of the IEEE European Conference on Computer Vision, 2012: 702-715.
20	HARE S , GOLODETZ S , SAFFARI A , et al. Struck: structured output tracking with kernels[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2016, 38 (10): 2096- 2109. doi: 10.1109/TPAMI.2015.2509974
21	KALAL Z , MIKOLAJCZYK K , MATAS J . Tracking-learning-detection[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2012, 34 (7): 1409- 1422. doi: 10.1109/TPAMI.2011.239
22	FAN H, XIANG J H. Robust visual tracking via local-global correlation filter[C]//Proc. of the National Conference on Artificial Intelligence, 2017: 4025-4031.
23	DANELLJAN M, HAGER G, KHAN F S, et al. Adaptive decontamination of the training set: a unified formulation for discriminative visual tracking[C]//Proc. of the Computer Vision and Pattern Recognition, 2016: 1430-1438.
24	ZHANG T, XU C, YANG M H. Multi-task correlation particle filter for robust object tracking[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 725-732.
25	WU Y , LIM J , YANG M H . Object tracking benchmark[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2015, 37 (9): 1834- 1848. doi: 10.1109/TPAMI.2014.2388226
26	孟琭, 杨旭. 目标跟踪算法综述[J]. 自动化学报, 2019, 45 (7): 1244- 1260.
	MENG L , YANG X . A survey of object tracking algorithms[J]. Acta Automatica Sinica, 2019, 45 (7): 1244- 1260.
27	GRAY R M . Toeplitz and circulant matrices[J]. Foundations and Trends in Communications and Information Theory, 2005, 2 (3): 155- 239. doi: 10.1561/0100000006
28	XUE H , CHEN S C , YANG Q . Discriminatively regularized least-squares classification[J]. Pattern Recognition, 2009, 42 (1): 93- 104. doi: 10.1016/j.patcog.2008.07.010
29	GALOOGAHI H K, SIM T, LUCEY S. Correlation filters with limited boundaries[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2015: 4630-4638.
30	BIBI A, MUELLER M, GHANEM B. Target response adaptation for correlation filter tracking[C]//Proc. of the IEEE European Conference on Computer Vision, 2016: 419-433.
31	DANELLJAN M, HAGER G, KHAN F S, et al. Learning spatially regularized correlation filters for visual tracking[C]//Proc. of the IEEE International Conference on Computer Vision, 2015: 4310-4318.
32	DANELLJAN M, ROBINSON A, KHAN F S, et al. Beyond correlation filters: learning continuous convolution operators for visual tracking[C]//Proc. of the IEEE European Conference on Computer Vision, 2016: 472-488.
33	DANELLJAN M, BHAT G, KHAN F S, et al. ECO: efficient convolution operators for tracking[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 6931-6939.
34	LI F, TIAN C, ZUO W M, et al. Learning spatial-temporal regularized correlation filters for visual tracking[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 4904-1913.
35	DAI K, WANG D, LU H C, et al. Visual tracking via adaptive spatially-regularized correlation filters[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2019: 4670-4679.
36	LI Y M, FU C H, DING F Q, et al. Autotrack: towards high-performance visual tracking for uav with automatic spatio-temporal regularization[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2020: 11920-11929.
37	GALOOGAHI H K, FAGG A, LUCEY S. Learning background-aware correlation filters for visual tracking[C]//Proc. of the IEEE International Conference on Computer Vision, 2017: 1144-1152.
38	LUKEZIC A, VOJIR T, ZAJC L C, et al. Discriminative correlation filter with channel and spatial reliability[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 4847-4856.
39	VAN D W J , SCHMID C , VERBEEK J , et al. Learning color names for real-world applications[J]. IEEE Trans.on Image Processing, 2009, 18 (7): 1512- 1523. doi: 10.1109/TIP.2009.2019809
40	DANELLJAN M, SHAHBAZ K F, FELSBERG M, et al. Adaptive color attributes for real-time visual tracking[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2014: 1090-1097.
41	MA C, YANG X K, ZHANG C Y, et al. Long-term correlation tracking[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2015: 5388-5396.
42	MA C, HUANG J B, YANG X, et al. Hierarchical convolutional features for visual tracking[C]//Proc. of the IEEE International Conference on Computer Vision, 2015: 3074-3082.
43	ZHANG Y, CHAN W, JAITLY N, et al. Very deep convolutional networks for end-to-end speech recognition[C]//Proc. of the International Conference on Acoustics, Speech, and Signal Processing, 2017: 4845-4849.
44	BHAT G, JOHNANDER J, DANELLJAN M, et al. Unveiling the power of deep tracking[C]//Proc. of the IEEE European Conference on Computer Vision, 2018: 493-509.
45	GUNDOGDU E , ALATAN A A . Good features to correlate for visual tracking[J]. IEEE Trans.on Image Processing, 2018, 27 (5): 2526- 2540. doi: 10.1109/TIP.2018.2806280
46	WANG N, SONG Y B, MA C, et al. Unsupervised deep tracking[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2019: 1308-1317.
47	XU T, FENG Z H, WU X J, et al. Joint group feature selection and discriminative filter learning for robust visual object tracking[C]//Proc. of the IEEE International Conference on Computer Vision, 2019: 7950-7960.
48	WANG N , ZHOU W G , SONG Y B , et al. Real-time correlation tracking via joint model compression and transfer[J]. IEEE Trans.on Image Processing, 2020, 29, 6123- 6135. doi: 10.1109/TIP.2020.2989544
49	DANELLJAN M, HAGER G, KHAN F, et al. Accurate scale estimation for robust visual tracking[C]//Proc. of the British Machine Vision Conference, 2014: 123-135.
50	LI Y, ZHU J K. A scale adaptive kernel correlation filter tracker with feature integration[C]//Proc. of the IEEE European Conference on Computer Vision, 2014: 254-265.
51	ZHANG S L , LU W , XING W W , et al. Learning scale-adaptive tight correlation filter for object tracking[J]. IEEE Trans.on Cybernetics, 2020, 50 (1): 270- 283. doi: 10.1109/TCYB.2018.2868782
52	LIU T, GANG W, YANG Q X. Real-time part-based visual tracking via adaptive correlation filters[C]//Proc. of the IEEE International Conference on Computer Vision Workshop, 2015: 587-594.
53	MONTERO A S, LANG J, LAGANIERE R. Scalable kernel correlation filter with sparse feature integration[C]//Proc. of the IEEE International Conference on Computer Vision Workshop, 2015: 587-594.
54	TANG M, YU B, ZHANG F, et al. High-speed tracking with multi-kernel correlation filters[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 4874-4883.
55	WANG M M, LIU Y, HUANG Z. Large margin object trac-king with circulant feature maps[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 4800-4808.
56	WANG N, ZHOU W G, TIAN Q, et al. Multi-cue correlation filters for robust visual tracking[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 4844-4853.
57	MA H , ACTON S T , LIN Z . SITUP: scale invariant tracking using average peak-to-correlation energy[J]. IEEE Trans.on Image Processing, 2020, 29, 3546- 3557. doi: 10.1109/TIP.2019.2962694
58	谢瑜, 陈莹. 通道裁剪下的多特征组合目标跟踪算法[J]. 系统工程与电子技术, 2020, 42 (4): 764- 772.
	XIE Y , CHEN Y . Multi-feature combined target tracking algorithm based on channel clipping[J]. Systems Engineering and Electronics, 2020, 42 (4): 764- 772.
59	BERTINETTO L, VALMADRE J, GOLODETZ S, et al. Staple: complementary learners for real-time tracking[C]//Proc. of the IEEE International Conference on Computer Vision and Pattern Recognition, 2016: 1401-1409.
60	ZUO W M , WU X H , LIN L , et al. Learning support correlation filters for visual tracking[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2019, 41 (5): 1158- 1171. doi: 10.1109/TPAMI.2018.2829180
61	XUAN S Y , LI S Y , HAN M F , et al. Object tracking in satellite videos by improved correlation filters with motion estimations[J]. IEEE Trans.on Geoscience and Remote Sensing, 2020, 58 (2): 1074- 1086. doi: 10.1109/TGRS.2019.2943366
62	SONG Y B, MA C, GONG L J, et al. CREST: convolutional residual learning for visual tracking[C]//Proc. of the IEEE International Conference on Computer Vision, 2017: 2555-2564.
63	WANG Q, GAO J, XING J L, et al. Dcfnet: discriminant correlation filters network for visual tracking[EB/OL]. [2020-06-16]. http://arxiv.org/abs/1704.04057vl.
64	BERTINETTO L, VALMADRE J, HENRIQUES J F, et al. Fully-convolutional siamese networks for object tracking[C]//Proc. of the IEEE European Conference on Computer Vision, 2016: 850-865.
65	CHOI J W, CHANG H J, YUN S D, et al. Attentional correlation filter network for adaptive visual tracking[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 4828-4837.
66	EUNBYUNG P, ALEXANDER C B. Meta-tracker: fast and robust online adaptation for visual object trackers[C]//Proc. of the IEEE European Conference on Computer Vision, 2018: 534-550.
67	ZHANG M D, WANG Q, XING J L, et al. Visual tracking via spatially aligned correlation filters network[C]//Proc. of the IEEE European Conference on Computer Vision, 2018: 469-485.
68	ZHANG H L , CHEN J , NIE G H , et al. Uncertain motion tracking based on convolutional net with semantics estimation and region proposals[J]. Pattern Recognition, 2020, 102, 107- 232.
69	REN S Q , HE K M , GIRSHICK R , et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2017, 39 (6): 1137- 1149. doi: 10.1109/TPAMI.2016.2577031
70	JAIN M , SUBRAMANYAM A V , DENMAN S , et al. LSTM guided ensemble correlation filter tracking with appearance model pool[J]. Computer Vision and Image Understanding, 2020, 195, 102935. doi: 10.1016/j.cviu.2020.102935
71	SAINATH T N, VINYALS O, SENIOR A W, et al. Convolutional, long short-term memory, fully connected deep neural networks[C]//Proc. of the International Conference on Acoustics, Speech, and Signal Processing, 2015: 4580-4584.
72	VALMADRE J, BERTINETTO L, HENRIQUES J F, et al. Long-term tracking in the wild: a benchmark[C]//Proc. of the IEEE European Conference on Computer Vision, 2018: 692-707.

算法	精确度	AUC	运行速度/fps	特征类型
CSK	0.518	0.382	304.815	Gray
CN	0.594	0.422	57.422	Gray+CN
DSST	0.693	0.470	23.268	HOG
KCF	0.696	0.477	140.465	HOG
SAMF	0.753	0.553	14.334	Gray+HOG+CN
CF2	0.845	0.603	10.624	CNN
CFLB	0.457	0.333	265.977	Gray
SRDCF	0.788	0.597	5.637	HOG
CCOT	0.896	0.667	1.302	CNN+HOG+CN
Staple	0.776	0.576	56.876	HOG+CH
CREST	0.839	0.624	9.064	CNN
BACF	0.816	0.615	25.339	HOG
CSR_DCF	0.909	0.675	13.029	HOG+CN
DCFNet	0.818	0.626	34.118	CNN
ECO	0.909	0.687	12.758	CNN+HOG+CN
ECO_HC	0.840	0.631	52.496	HOG+CN
MKCFup	0.760	0.586	49.835	HOG+CN
MCCT	0.917	0.682	7.712	CNN+HOG
MCCT_H	0.848	0.636	36.302	Gray+HOG+CN
STRCF	0.864	0.654	15.391	CNN+HOG+CN
ASRCF	0.919	0.689	16.516	CNN+HOG
GFS_DCF	0.931	0.693	1.377	CNN+HOG+ CN+IC

[1]	韩啸, 陈世文, 陈蒙, 杨锦程. 基于互易点学习的LPI信号开集识别[J]. 系统工程与电子技术, 2022, 44(9): 2752-2759.
[2]	仇祝令, 查宇飞, 李振宇, 李禹铭, 张鹏, 朱川. 基于多模型蒸馏的时间正则化相关滤波跟踪算法[J]. 系统工程与电子技术, 2022, 44(8): 2448-2456.
[3]	侯子林, 程婷, 彭瀚. 基于量测转换序贯滤波的GMPHD机动目标跟踪[J]. 系统工程与电子技术, 2022, 44(8): 2474-2482.
[4]	张立民, 谭凯文, 闫文君, 张聿远. 基于多级跳线残差网络的雷达辐射源识别[J]. 系统工程与电子技术, 2022, 44(7): 2148-2156.
[5]	史浩然, 卢发兴, 祁江鑫, 杨光. 基于辅助信标的无人机协同目标跟踪[J]. 系统工程与电子技术, 2022, 44(7): 2302-2310.
[6]	金国栋, 薛远亮, 谭力宁, 许剑锟. 基于孪生神经网络的目标跟踪算法进展研究[J]. 系统工程与电子技术, 2022, 44(6): 1805-1822.
[7]	翟光, 王妍欣, 孙一勇. 基于低轨星网的多目标协同跟踪滤波技术[J]. 系统工程与电子技术, 2022, 44(6): 1957-1967.
[8]	赵晓枫, 徐叶斌, 吴飞, 牛家辉, 蔡伟, 张志利. 基于全局感知机制的地面红外目标检测方法[J]. 系统工程与电子技术, 2022, 44(5): 1461-1467.
[9]	王帅, 向建军, 彭芳, 唐书娟. 基于新最速下降法的目标跟踪算法[J]. 系统工程与电子技术, 2022, 44(5): 1512-1519.
[10]	邹虹, 白陈阳, 何鹏, 崔亚平, 王汝言, 吴大鹏. 基于分布式深度学习的边缘服务放置策略[J]. 系统工程与电子技术, 2022, 44(5): 1728-1737.
[11]	辛怀声, 曹晨. 基于交互多模型的分组δ-广义标签多伯努利算法[J]. 系统工程与电子技术, 2022, 44(4): 1128-1138.
[12]	陈冬, 句彦伟. 基于语义分割实现的SAR图像舰船目标检测[J]. 系统工程与电子技术, 2022, 44(4): 1195-1201.
[13]	孙晶明, 虞盛康, 孙俊. 基于深度学习的HRRP识别姿态敏感性分析[J]. 系统工程与电子技术, 2022, 44(3): 802-807.
[14]	谢家豪, 黄树彩, 韦道知, 张曌宇, 王文豪. 基于P_EV准则的不确定混合多传感器联盟求解[J]. 系统工程与电子技术, 2022, 44(3): 819-826.
[15]	宋子壮, 杨嘉伟, 张东方, 王诗强, 张硕. 基于无锚框的红外多类别多目标实时跟踪网络[J]. 系统工程与电子技术, 2022, 44(2): 401-409.