Towards long-term player tracking with graph hierarchies and domain-specific features

Maria Koshkina, James H. Elder · 2025

Read Towards long-term player tracking with graph hierarchies and domain-specific features by Maria Koshkina, James H. Elder (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-21242.txt.

Introduction

Maria Koshkina and James Elder present SportsSUSHI, a 2025 method for tracking team sports players over a whole game. Long-term tracking is hard because teammates look alike, players block each other, and players leave the field of view and return. Their method builds a hierarchical graph that links short sightings, or tracklets, into full paths over time.

The key advantage is domain knowledge. The graph uses jersey numbers, team identifiers, and field coordinates to tell look-alike players apart. Tests on the SoccerNet dataset and on a newly proposed hockey dataset went well. The hockey data were recorded with one stationary camera that captures the whole playing surface. Its long sequences and full annotations suit tests of long-term tracking. Adding the domain-specific features significantly improved association accuracy. The dataset and code are shared online.

Catalog record

CREATORMaria Koshkina, James H. Elder
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-21242

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Towards long-term player tracking with graph hierarchies and domain-specific features Maria Koshkina, James H. Elder Abstract In team sports analytics, long-term player tracking remains a challenging task due to player appearance similarity, occlusion, and dynamic motion patterns. Accurately re-identifying players and reconnecting tracklets after extended absences from the field of view or prolonged occlusions is crucial for robust analysis. We introduce SportsSUSHI, a hierarchical graph-based approach that leverages domain-specific features, including jersey numbers, team IDs, and field coordinates, to enhance tracking accuracy. SportsSUSHI achieves high performance on the SoccerNet dataset and a newly proposed hockey tracking dataset. Our hockey dataset, recorded using a stationary camera capturing the entire playing surface, contains long sequences and annotations for team IDs and jersey numbers, making it well-suited for evaluating long-term tracking capabilities. The inclusion of domain-specific features in our approach significantly improves association accuracy, as demonstrated in our experiments. The dataset and code are available at https://github.com/mkoshkina/sports-SUSHI 1 Introduction Figure 1 : Team uniform and protective gear make players on the same team hard to distinguish. Jersey number is a key feature for player re-identification and long-term player tracking. Sports video understanding and automatic statistics are important applications of computer vision. They help coaches improve tactics and run better training, enable efficient game analysis, and can enrich the sports viewing experience. The ability to automatically track players during a team game is essential for game analysis. It involves detecting and following players during a game. Player tracking is a specific case of a well-studied topic in computer vision - Multi-Object Tracking (MOT). Typically, MOT is studied in the context of pedestrian tracking (for example, for surveillance purposes) and vehicle tracking (for self-driving cars and traffic analysis). Tracking players during a team sports event shares the same problem setup and challenges as tracking pedestrians. Generic MOT tracking algorithms can be applied to tracking players and will produce reasonable results. However, there are several important differences between generic MOT and team sport MOT: • Players on the same team often look similar, especially in winter sports like hockey, where gear obscures distinctive features, making traditional re-identification methods less effective. • Pedestrian tracking focuses on short-term tracking, while player tracking in sports requires recovering tracks after long occlusions or absences. • Players move quickly and often follow non-linear motion patterns. • Sports settings involve fewer individuals in view compared to pedestrian scenes. • Players can be uniquely identified by their team and jersey numbers. Most recent approaches to player tracking utilize online short-term tracking [ 24 , 51 , 40 ] . Some apply the post-processing steps to reconnect the resulting tracklets into longer tracks [ 42 , 41 ] . In our work, we follow a unified, hierarchical approach proposed by Cetintas et al. [ 10 ] that addresses both short-term and long-term tracking. We argue that we need to incorporate domain-specific features such as jersey numbers, team ID, and field position to achieve reliable long-term tracking of players. Experiments on SoccerNet [ 15 ] player tracking dataset show that our approach is effective. To further evaluate the performance of our method we propose a new hockey player tracking dataset. Recorded with a stationary camera, the video clips capture the whole playing surface and have longer video sequences (the average length is 1311 frames). The dataset is particularly challenging for tracking because individual player’s features are hidden by protective gear, and motion blur and frequent occlusion make jersey numbers hard to recognize. The dataset is available at https://github.com/mkoshkina/sports-SUSHI . Our main contributions are as follows: • We present a new player tracking method, SportsSUSHI built on top of the hierarchical graph-based method SUSHI [ 10 ] that allows for the long-term association between tracklets. We show how incorporating domain-specific features such as jersey numbers improves the performance of player tracking. • We propose a novel hockey tracking dataset that is unique in that it captures the whole playing surface with a stationary camera. • To showcase its high performance we evaluate SportsSUSHI our novel hockey dataset and on publicly-available SoccerNet [ 15 ] . 2 Related Work 2.1 MOT In recent years, convolutional networks have driven the success of tracking-by-detection for multi-object tracking (MOT). This approach uses object detectors to identify objects in each frame, followed by data association to link detections to tracks. Challenges such as occlusions, missed detections, and similar object appearances are addressed by tracking ”lost” tracks and employing re-identification. MOT methods are classified as online or offline [ 14 ] . Online tracking processes frames sequentially, enabling real-time applications but limiting its ability to correct past errors. Offline tracking, in contrast, processes entire sequences, leveraging both past and future frames for globally optimal trajectories, making it more accurate but unsuitable for real-time use. Online tracking performs well for short-term tracking and pedestrian datasets, where occlusions and absences are minimal. However, it struggles with player tracking, which involves more complex dynamics and prolonged occlusions. The majority of the proposed generic MOT methods are online such as Tracktor++ [ 5 ] , CenterTrack [ 57 ] , FairMOT [ 55 ] , ByteTrack [ 54 ] , SORT [ 6 ] and DeepSORT [ 50 ] . Some of these piggyback on object detectors repurposing them to perform detection and association between a pair of frames in a single step (Tracktor [ 5 ] , CenterTrack [ 57 ] ). The data association approach is often performed using bi-partite matching (with Hungarian algorithm), where matching is sometimes done in stages [ 50 , 54 ] . The offline MOT problem is often modelled as a graph where nodes represent detections or tracklets and edges between two nodes are potential associations. One trajectory corresponds to one flow path in the graph. Several early methods were proposed to use this model and solve for an optimal global solution using min-cost network flow algorithms [ 52 , 13 ] . The min-cost flow formulation is guaranteed to find an optimal solution in polynomial time. MPNTrack [ 9 ] built on this approach but used a graph neural network (GNN) and treated the problem as edge classification. Building the graph based on all frames and all detections is, of course, too expensive. Therefore, they use a batch approach considering 15 frames at a time. That is a limited window to consider for long-term tracking. Cetintas et al. [ 10 ] use the MPNTrack [ 9 ] GNN approach to develop a unified model they call SUSHI which is scalable, generalizable, and suitable for both short-term and long-term tracking. Instead of having a different approach and manual feature architecture for short-term and long-term tracking, they break up an image sequence into a hierarchy. They construct a graph with detections as nodes and employ a GNN to classify the edges. The resulting tracklets are then utilized to create a new graph, connecting them into longer tracklets, and so forth. While the multi-stage approach has been employed in other works, SUSHI [ 10 ] distinguishes itself by reusing the same architecture and weights across all levels. SUSHI breaks down the image sequence into multiple levels, resulting in smaller graphs at each level, which enhances the system’s speed and scalability. To

Read the complete text