Using 3D Convolutional Neural Networks for Real-time Detection of Soccer Events

Abstract

Developing systems for the automatic detection of events in video is a task which has gained attention in many areas including sports. However, there are still a number of shortcomings with current systems, such as high latency and determining proper timing boundaries for events detected, making it challenging to operate at the live edge. In this paper, we present an algorithm to detect events in soccer videos in real time, using 3D convolutional neural networks. We run and evaluate our algorithm based on on three different real-world soccer data sets from SoccerNet, the Swedish elite series Allsvenskan, and the Norwegian elite series Eliteserien. Overall, the results show that we can detect highly relevant events with high recall, low latency, and accurate time estimation. Rapid response matters most for us, but we compare our results with current state-of-the-art that has less strict timing requirements. We conclude that our algorithm can detect most events in real-times, but still can be improved with slightly better precision. In addition to the presented algorithm, we perform an extensive ablation study on how the different parts of the training pipeline affect the final results.