CLASSIFICATION OF SPORTS VIDEOS BY COMBINING RESNET50 MODEL AND FINE TUNING

Pinku Ranjan, Jayant Kumar Rai, Vaibhav Singh, Anand Sharma, Somesh Kumar

DOI Number
https://doi.org/10.2298/FUEE2501039R
First page
039
Last page
052

Abstract


This article represents a classification of sports videos by integrating the ResNet50 model and fine-tuning methods. Broadcasting companies give significant importance to the classification of sports videos. That is a subclass of recognition of human action, which will clarify the context of videos. This work uses a deep neural network-based ResNet50 model with a fine-tuning technique for classifying popular sports types in India into their corresponding classes. This paper considers 14 main sports - badminton, basketball, boxing, cricket, football, hockey, kabaddi, swimming, shooting, table tennis, tennis, volleyball, weight lifting, and wrestling. The dataset is created to focus on sports action-based classification. Fine-tuning is nothing but networking surgery. First, a pre-trained Convolutional Neural Network model will be loaded, and then fine-tuning (network surgery) will be applied. The base model (ResNet50) will be frozen, so it will not be trained via backpropagation. After fine-tuning, the classifier will be ready to correctly classify a sports video into its category. The training accuracy of the proposed classifier is 91.73%, and testing is done on sports videos. The classifier classifies each sports video into its class correctly. A descent confusion matrix has been pertained.

Keywords

Deep Learning, ResNet50, fine-tuning, Computer Vision, Sports Videos Classification

Full Text:

PDF

References


D. Brezeale and D. J. Cook, "Automatic video classification: A survey of the literature", IEEE Trans. Syst. Man. Cybern. Part C (Applications and Reviews), vol. 38, no. 3, pp. 416-430, 2008.

K. Messer, W. Christmas and J. Kittler, "Automatic sports classification", In Proceedings of the IEEE International Conference on Pattern Recognition, Quebec City, QC, Canada, 2002, vol. 2, pp. 1005-1008.

F. Cricri, M. J. Roininen, J. Leppanen, S. Mate, I. D. Curcio, S. Uhlmann and M. Gabbouj, "Sport type classification of mobile videos", IEEE Trans. Multimedia, vol. 16, no. 4, pp. 917-932, 2014.

P. Campr, M. Herbig, J. Vanek and J. Psutka, "Sports video classification in continuous TV broadcasts", In Proceedings of the 12th IEEE International Conference on Signal Processing (ICSP), 2014, pp. 648-652.

M. A. Russo, A. Filonenko and K.-H. Jo, "Sports classification in sequential frames using CNN and RNN", In Proceedings of the IEEE International Conference on Information and Communication Technology Robotics (ICT-ROBOT), 2018, pp. 1-3.

A. J. Eronen, V. T. Peltonen, J. T. Tuomi, A. P. Klapuri, S. Fagerlund, T. Sorsa, G. Lorho and J. Huopaniemi, "Audio-based context recognition", IEEE Trans. Audio, Speech Lang. Process., vol. 14, no. 1, pp. 321-329, 2005.

R. Gade, M. Abou-Zleikha, M. Græsbøll Christensen and T. B. Moeslund, "Audio-visual classification of sports types", In Proceedings of the IEEE International Conference on Computer Vision Workshops, 2015, pp. 51-56.

V. Ellappan and R. Rajasekaran, "Event recognition and classification in sports video",” In Proceedings of the 2017 Second IEEE International Conference on Recent Trends and Challenges in Computational Models (ICRTCCM), 2017, pp. 182-187.

K. He, X. Zhang, S. Ren and J. Sun, "Deep residual learning for image recognition", In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770-778.

Y. Hong, C. Ling and Z. Ye, "End-to-end soccer video scene and event classification with deep transfer learning", In Proceedings of the IEEE International Conference on Intelligent Systems and Computer Vision (ISCV), 2018, pp. 1-4.

M. H. Sigari, S. A. Sureshjani and H. Soltanian-Zadeh, "Sports video classification using an ensemble classifier", In Proceedings of the 7th Iranian IEEE Conference on Machine Vision and Image Processing, 2011, pp. 1-4.

Y. Xing, C. Lv, H. Wang, D. Cao, E. Velenis and F.-Y. Wang, "Driver activity recognition for intelligent vehicles: A deep learning approach", IEEE Trans. Veh. Technol., vol. 68, no. 6, pp. 5379-5390, 2019.

K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition", arXiv preprint, arXiv:1409.1556, 2014.

R. K. Srivastava, K. Greff and J. Schmidhuber, "Highway networks", arXiv preprint, arXiv:1505.00387, 2015.

A. Ekin, A. M. Tekalp and R. Mehrotra, "Automatic soccer video analysis and summarization", IEEE Trans. Image Process., vol. 12, no. 7, pp. 796-807, 2003.

H. Jiang, Y. Lu and J. Xue, "Automatic soccer video event detection based on a deep neural network combined CNN and RNN",” In Proceedings 28th IEEE International Conference on Tools with Artificial Intelligence (ICTAI), 2016, pp. 490-494.

S. J. Pan and Q. Yang, "A survey on transfer learning", IEEE Trans. Knowl. Data Eng., vol. 22, no. 10, pp. 1345- 1359, 2009.

A. Krizhevsky, I. Sutskever and G. E. Hinton, "Imagenet classification with deep convolutional neural networks", in Advances in neural information processing systems, 2012, pp. 1097-1105.

C. K. Mohan and B. Yegnanarayana, "Classification of sport videos using edge-based features and autoassociative neural network models", Signal Image Video Process., vol. 4, no. 1, pp. 61-73, 2010.

C. Xu, J. Cheng, Y. Zhang, Y. Zhang and H. Lu, "Sports video analysis: Semantics extraction, editorial content creation and adaptation", J. Multimedia, vol. 4, no. 2, pp. 69-79, 2009.

J. Wang, C. Xu and E. Chng, "Automatic sports video genre classification using pseudo-2D-HMM", Jinjun Wang, Changsheng Xu and E. Chng, "Automatic Sports Video Genre Classification using Pseudo-2D-HMM," In Proceedings of the IEEE 18th International Conference on Pattern Recognition (ICPR'06), Hong Kong, China, 2006, pp. 778-781.

L. Li, N. Zhang, L.-Y. Duan, Q. Huang, J. Du and L. Guan, "Automatic sports genre categorization and view-type classification over large-scale dataset",” in Proceedings of the 17th ACM International Conference on Multimedia, 2009, pp. 653-656.

V. Chaudhary, Rashmi and V. Uniyal, "An effective video noise removal algorithm", Int. Res. J. Eng. Technol. (IRJET), vol. 3, no. 8, pp. 2031-2034, 2016.

X. Yunjun, "A sports training video classification model based on deep learning",” Scientific Programming, vol. 2021, p. 7252896, 2021.

S. M. Daudpota, A. Muhammad and J. Baber. "Video genre identification using clustering-based shot detection algorithm", Signal, Image and Video Process., vol. 13, pp. 1413-1420, 2019.

S. Zhang, "Detection of aerobics action based on convolutional neural network", Comput. Intell. Neurosci., vol. 2022, p. 1857406, 2022.

Y. I. Mohamad, S. S. Baraheem and T. V. Nguyen, "Olympic games event recognition via transfer learning with photobombing guided data augmentation", J. Imaging, vol. 7, no. 2, p. 12, 2021.

M. Ramesh and K. Mahesh, "Sports Video Classification Framework Using Enhanced Threshold Based Keyframe Selection Algorithm and Customized CNN on UCF101 and Sports1-M Dataset", Comput. Intell. Neurosci., vol. 2022, p. 218431, 2022.

N. Feng et al., "SSET: a dataset for shot segmentation, event detection, player tracking in soccer videos." Multim. Tools Appl., vol. 79, pp. 28971-28992, 2020.

M. Tabish, ZuR. Tanooli and M. Shaheen, "Activity recognition framework in sports videos." Multim. Tools Appl., vol. 83, pp. 15101-15123, 2021.

M. Rafiq et al, "Scene classification for sports video summarization using transfer learning", Sensors, vol. 20, no. 6, p.1702, 2020.

F. Wu, Q. Wang, J. Bian, N. Ding, F. Lu, J. Cheng, D. Dou and H. Xiong, "A Survey on Video Action Recognition in Sports: Datasets, Methods and Applications", IEEE Trans. Multim., vol. 25, pp. 7943-7966, 2022.

D. Xiao, F. Zhu, J. Jiang and X. Niu, "Leveraging Natural Cognitive Systems in Conjunction with ResNet50-BiGRU Model and Attention Mechanism for Enhanced Medical Image Analysis and Sports Injury Prediction", Front. Neurosci., vol. 17, p. 1273931, 2023.


Refbacks

  • There are currently no refbacks.


ISSN: 0353-3670 (Print)

ISSN: 2217-5997 (Online)

COBISS.SR-ID 12826626