深度视觉目标跟踪进展综述

doi:10.52396/JUST-2021-0037

摘要/Abstract

摘要： 视频目标跟踪是计算机视觉领域的一个重要研究课题.近年来,随着深度学习在视觉目标跟踪领域获得了巨大的成功,一系列优秀的深度跟踪算法涌现出来.本文回顾了近年来深度目标跟踪领域的进展.首先,我们详细讨论了近十年来跟踪领域数据集的发展趋势,这些数据集不仅全面地评估了算法性能同时为模型训练提供了极大的便利.其次,我们分类讨论了几大类经典的深度学习跟踪框架,包括深度相关滤波器跟踪、分类式网络跟踪、双路网络跟踪、基于梯度的深度跟踪算法以及基于Transformer的跟踪算法.最后,我们对全文内容进行总结,并指出未来的发展趋势.

关键词: 深度目标跟踪, 跟踪数据集, 相关滤波器, 分类式跟踪网络, 双路跟踪网络, 梯度跟踪网络

Abstract: Visual object tracking is an important branch in computer visions. In recent years, with the remarkable success of deep learning techniques, a series of deep tracking algorithms have emerged with impressive performances. In this paper, we review the recent development of deep learning based trackers. First, we revisit the development of tracking benchmarks in the last decade. These tracking datasets not only comprehensively help evaluate the tracking algorithms but also largely support the model training of deep trackers. Next, we discuss several representative tracking frameworks including deep correlation filter tracking, classification-based tracking networks, Siamese tracking networks, gradient-based tracking networks and Transformer based deep trackers. Finally, we conclude the paper and discuss the potential future research directions of the visual tracking.

Key words: deep visual tracking, benchmark datasets, correlation filter, classification-based tracking networks, Siamese tracking networks, gradient-based tracking networks

中图分类号:

P595

王宁, 席茂, 周文罡, 李礼, 李厚强. 深度视觉目标跟踪进展综述[J]. 中国科学技术大学学报, 2021, 51(4): 335-344.

WANG Ning, XI Mao, ZHOU Wengang, LI Li, LI Houqiang. Recent advance in deep visual object tracking[J]. Journal of University of Science and Technology of China, 2021, 51(4): 335-344.

参考文献

[1] Babenko B, Yang M H, Belongie S. Robust object tracking with online multiple instance learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(8):1619-1632.
[2] Kalal Z, Mikolajczyk K, Matas J. Tracking-learning-detection. IEEE Transactions on Software Engineering, 2011, 34(7):1409-1422.
[3] Zhong W, Lu H, Yang M H. Robust object tracking via sparsity-based collaborative model. Proceedings of the IEEE Conference of Computer Vision and Pattern Recognition, 2012.
[4] Hare S, Saffari A, Torr P H S. Struck:Structured output tracking with kernels. Proceedings of the International Conference on Computer Vision, 2011.
[5] Henriques J F, Caseiro R, Martins P, et al. High-speed tracking with kernelized correlation filters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(3):583-596.
[6] Kristan M, Leonardis A, Matas J, et al. The sixth visual object tracking VOT2018 challenge results. Proceedings of the European Conference on Computer Vision Workshops, 2018.
[7] Mueller M, Bibi A, Giancola S,et al. TrackingNet: A large-scale dataset and benchmark for object tracking in the wild. Proceedings of the European Conference on Computer Vision, 2018.
[8] Fan H, Ling H, Lin L, et al. Lasot: A high-quality benchmark for large-scale single object tracking. Proceedings of the IEEE Conference of Computer Vision and Pattern Recognition, 2019.
[9] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. Proceedings of the Conference on Advances in Neural Information Processing Systems, 2012.
[10] Ma C, Huang J B, Yang X, et al. Hierarchical convolutional features for visual tracking. Proceedings of the IEEE International Conference on Computer Vision, 2015.
[11] Nam H, Han B. Learning multi-domain convolutional neural networks for visual tracking. Proceedings of the IEEE Conference of Computer Vision and Pattern Recognition, 2016.
[12] Bertinetto L, Valmadre J, Henriques J F, et al. Fully-convolutional Siamese networks for object tracking. Proceedings of the European Conference on Computer Vision Workshops, 2016.
[13] Danelljan M, Bhat G, Khan F S, et al. ECO: Efficient convolution operators for tracking. Proceedings of the IEEE Conference of Computer Vision and Pattern Recognition, 2017.
[14] Li B, Yan J, Wu W, et al. High performance visual tracking with siamese region proposal network. Proceedings of the IEEE Conference of Computer Vision and Pattern Recognition, 2018.
[15] Danelljan M, Bhat G, Khan F S, et al. ATOM: Accurate tracking by overlap maximization. Proceedings ofthe IEEE Conference of Computer Vision and Pattern Recognition, 2019.
[16] Bhat G, Danelljan M, Gool L V, et al. Learning discriminative model prediction for tracking. Proceedings of the International Conference on Computer Vision, 2019.
[17] Wu Y, Lim J, Yang M H. Object tracking benchmark. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9):1834-1848.
[18] Danelljan M, Robinson A, Khan F S, et al. Beyond correlation filters: Learning continuous convolution operators for visual tracking.Proceedings of the European Conference on Computer Vision, 2016.
[19] Valmadre J, Bertinetto L, Henriques J, et al. End-to-end representation learning for correlation filter based tracking. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[20] Song Y, Ma C, Wu X, et al. VITAL: Visual tracking via adversarial learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
[21] Jung I, Son J, Baek M, et al. Real-time MDNet. Proceedings of the European Conference on Computer Vision, 2018.
[22] Li B, Wu W, Wang Q, et al. SiamRPN++: Evolution of Siamese visual tracking with very deep networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019.
[23] Song Y, Ma C, Gong L, et al. CREST: Convolutional residual learning for visual tracking. Proceedings of the IEEE International Conference on Computer Vision, 2017.
[24] Wang N, Zhou W, Wang J, et al. Transformer meets tracker: Exploiting temporal context for robust visual tracking.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2021.
[25] Chen X, Yan B, Zhu J, et al. Transformer Tracking.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2021.
[26] Yan B, Peng H, Fu J, et al.Learning spatio-temporal transformer for visual tracking. 2013,arXiv: 17154, 2021.
[27] Wu Y, Lim J, Yang M-H. Online object tracking: A benchmark. Proceedings of the IEEE Conference of Computer Vision and Pattern Recognition, 2013.
[28] Liang P, Blasch E, Ling H. Encoding color information for visual tracking: Algorithms and benchmark. IEEE Transactions on Image Processing, 2015, 24(12):5630-5644.
[29] Kiani Galoogahi H, Fagg A, Huang C, et al. Need for speed: A benchmark for higher frame rate object tracking. Proceedings of the IEEE International Conference on Computer Vision, 2017.
[30] Mueller M, Smith N, Ghanem B. A benchmark and simulator for UAV tracking. Proceedings of the European Conference on Computer Vision, 2016.
[31] Huang L, Zhao X, HuangK. Got-10k: a large high-diversity benchmark for generic object tracking in the wild. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019.
[32] Valmadre J, Bertinetto L, Henriques J F, et al. Long-term tracking in the wild: A benchmark. Proceedings of the European Conference on Computer Vision, 2018.
[33] Danelljan M, Hager G, Shahbaz Khan F, et al. Learning spatially regularized correlation filters for visual tracking. Proceedings of the International Conference on Computer Vision, 2015.
[34] Kiani Galoogahi H, Fagg A, Lucey S. Learning background-aware correlation filters for visual tracking. Proceedings of the International Conference on Computer Vision,2017.
[35] Dai K, Wang D, Lu H, et al. Visual tracking via adaptive spatially-regularized correlation filters. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019: 4670-4679.
[36] Danelljan M, Hager G, Shahbaz Khan F, et al. Adaptive decontamination of the training set: A unified formulation for discriminative visual tracking. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
[37] Mueller M, Smith N, Ghanem B. Context-aware correlation filter tracking. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[38] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. 2014, arXiv: 1409.1556.
[39] Qi Y, Zhang S, Qin L, et al. Hedged deep tracking. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
[40] Wang N, Zhou W, Tian Q, et al. Multi-cue correlation filters for robust visual tracking. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
[41] Bhat G, Johnander J, Danelljan M, et al. Unveiling the power of deep tracking. Proceedings of the European Conference on Computer Vision, 2018.
[42] He K, Zhang X, Ren S, et al. Deep residual learning for image recognition.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
[43] Wang Q, Gao J, Xing J, et al. DCFNet: Discriminant correlation filters network for visual tracking. 2017, arXiv:1704.04057.
[44] Yao Y, Wu X, Zhang L, et al. Joint representation and truncated inference learning for correlation filter based tracking. Proceedings of the European Conference on Computer Vision, 2018.
[45] Girshick R, Donahue J, Darrell T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014.
[46] Han B, Sim J, Adam H. BranchOut: Regularization for online ensemble tracking with convolutional neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[47] Girshick R. Fast R-CNN.Proceedings of the IEEE International Conference on Computer Vision, 2015.
[48] Zhu Z, Wang Q, Li B, et al. Distractor-aware Siamese networks for visual object tracking. Proceedings of the European Conference on Computer Vision, 2018.
[49] He A, Luo C, Tian X, et al. A twofold Siamese network for real-time object tracking. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
[50] Wang Q, Teng Z, Xing J, et al. Learning attentions: Residual attentional Siamese network for high performance online visual tracking. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
[51] Yu Y, Xiong Y, Huang W, et al. Deformable Siamese attention networks for visual object tracking.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020.
[52] Wang N, Song Y, Ma C, et al. Unsupervised deep tracking. Proceedings of the IEEE conference on computer vision and pattern recognition. 2019.
[53] Wang N, Zhou W, Song Y, et al. Unsupervised deep representation learning for real-time tracking. International Journal of Computer Vision, 2021, 129(2): 400-418.
[54] Gao J, Zhang T, Xu C. Graph convolutional tracking. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019.
[55] Dong X, Shen J, Wang W, et al. Hyperparameter optimization for tracking with continuous deep Q-learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
[56] Du F, Liu P, Zhao W, et al. Correlation-guided attention for corner detection based visual tracking. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020.
[57] Voigtlaender P, Luiten J, Torr P H S, et al. Siam R-CNN: Visual tracking by re-detection.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020.
[58] Guo D, Wang J, Cui Y, et al. SiamCar: Siamese fully convolutional classification and regression for visual tracking. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020.
[59] Zhang Z, Peng H, Fu J, et al. Ocean: Object-aware anchor-free tracking. Proceedings of the European Conference on Computer Vision, 2020.
[60] Yang T, Chan A B. Learning dynamic memory networks for object tracking. Proceedings of the European Conference on Computer Vision, 2018.
[61] Park E, Berg A C. Meta-tracker: Fast and robust online adaptation for visual object trackers. Proceedings of the European Conference on Computer Vision, 2018.
[62] Huang J, Zhou W. Re2EMA: Regularized and reinitialized exponential moving average for target model update in object tracking. Proceedings of the AAAI Conference on Artificial Intelligence, 2019.
[63] Zhang L, Gonzalez-Garcia A, Weijer J, et al. Learning the model update for Siamese trackers. Proceedings of the IEEE International Conference on Computer Vision, 2019.
[64] Li P, Chen B, Ouyang W, et al. GradNet: Gradient-guided network for visual object tracking. Proceedings of the IEEE International Conference on Computer Vision, 2019.
[65] Lu X, Ma C, Ni B, et al. Deep regression tracking with shrinkage loss. Proceedings of the European Conference on Computer Vision, 2018.
[66] Lukezic A, Matas J, Kristan M. D3S-A discriminative single shot segmentation tracker. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020.
[67] Danelljan M, Gool L V, Timofte R. Probabilistic regression for visual tracking. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020.
[68] Bhat G, Danelljan M, Van Gool L, et al. Know your surroundings: Exploiting scene information for object tracking. Proceedings of the European Conference on Computer Vision, 2020.
[69] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need. 2017, arXiv:1706.03762.
[70] Wang X, Girshick R, Gupta A, et al. Non-local neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
[71] Carion N, Massa F, Synnaeve G, et al. End-to-end object detection with transformers. Proceedings of the European Conference on Computer Vision, 2020.