Towards Automatic Soccer Commentary Generation with Knowledge-Enhanced Visual Reasoning
Zeyu Jin 1,2 , Xiaoyu Qin 1,* , Songtao Zhou 1 , Kaifeng Yun 1 , Jia Jia 1,2,* · 2026
Read Towards Automatic Soccer Commentary Generation with Knowledge-Enhanced Visual Reasoning by Zeyu Jin 1,2 , Xiaoyu Qin 1,* , Songtao Zhou 1 , Kaifeng Yun 1 , Jia Jia 1,2,* (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-00057.txt.
Introduction
Zeyu Jin and four colleagues present GameSight, a two-stage system that writes soccer commentary in live-television style rather than anonymous play-by-play text. It first ties unnamed figures in video to specific players and teams through visual reasoning, then enriches the draft with historical statistics and the evolving game state. On the SN-Caption-test-align set, player alignment accuracy rises 18.5 percent above Gemini 2.5-pro, alongside better segment accuracy and game-level relevance.
Catalog record
| CREATOR | Zeyu Jin 1,2 , Xiaoyu Qin 1,* , Songtao Zhou 1 , Kaifeng Yun 1 , Jia Jia 1,2,* |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 72987 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.00057
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Towards Automatic Soccer Commentary Generation with Knowledge-Enhanced Visual Reasoning Zeyu Jin 1,2 , Xiaoyu Qin 1,* , Songtao Zhou 1 , Kaifeng Yun 1 , Jia Jia 1,2,* Abstract Soccer commentary plays a crucial role in enhancing the soccer game viewing experience for audiences. Previous studies in automatic soccer commentary generation typically adopt an end-to-end method to generate anonymous live text commentary . Such generated commentary is insufficient in the context of real-world live televised commentary , as it contains anonymous entities, context-dependent errors and lacks statistical insights of the game events. To bridge the gap, we propose GameSight , a two-stage model to address soccer commentary generation as a knowledge-enhanced visual reasoning task , enabling live-televised-like knowledgeable commentary with accurate reference to entities (players and teams). GameSight starts by performing visual reasoning to align anonymous entities with fine-grained visual and contextual analysis. Subsequently, the entity-aligned commentary is refined with knowledge by incorporating external historical statistics and iteratively updated internal game state information. Consequently, GameSight improves the player alignment accuracy by 18.5% on SN-Caption-test-align dataset compared to Gemini 2.5-pro. Combined with further knowledge enhancement, GameSight outperforms in segment-level accuracy and commentary quality, as well as game-level contextual relevance and structural composition. We believe that our work paves the way for a more informative and engaging human-centric experience with the AI sports application. Demo page: https://gamesight2025.github.io/gamesight2025 . I Introduction The soccer industry holds a significant position in the global sports market, with a fan base of over five billion people [ 7 ] . As one of the key elements of the viewing experience, soccer commentary is of vital importance in enhancing fan engagement and delivering captivating information. This has sparked considerable interest in automatic soccer commentary generation [ 25 , 24 ] . Given a segment of soccer game video, end-to-end soccer commentary generation models, e.g., MatchVision [ 37 ] and SoccerComment [ 20 ] , primarily focus on producing descriptions in the style of anonymous live text commentary . However, they fall short when it comes to replicating the real-world live televised commentary . Fig. 1: Overview of the proposed GameSight . It incorporates visual and contextual information to generate soccer commentary that is entity-aligned and knowledge-enhanced. Fig. 2: Unlike previous end-to-end models, GameSight is a two-stage system that addresses soccer commentary generation as a knowledge-enhanced visual reasoning task. It revises and enriches the anonymized commentary with Stage I : visual reasoning for entity alignment, and Stage II : commentary refinement with knowledge. Professional live televised commentary is characterized by several key elements to captivate the audience: (1) Accurate reference to entities [ 31 ] . By aligning names of players and teams in the sportscasting footage and the ongoing events, game commentators help the audience to understand the unfolding action. (2) Awareness of the internal game context. Commentators also integrate the dynamic match context to offer a comprehensive interpretation of current game state. The commentary paragraph should flow continuously, rather than being fragmented, to maintain a cohesive narrative. (3) Utilization of external statistics . Commentary should consist of not only description, but also explanation and comment [ 51 ] . Combining historical statistical knowledge serves as the foundation of these broader perspective and deeper insights [ 10 ] . Previous soccer commentary models fail to meet the above from the following aspects. Prob. 1: Absence of entity grounding. Previous models generate lines with anonymized player and team labels as placeholders [ 25 ] , incompetent for aligning the specific players involved in the events. Prob. 2: Context-dependent errors. These models are unaware of broader context as they aim solely at increasing the text similarity in video captioning. This leads to context-dependent errors in critical game information, such as incorrect score announcements following goal events. Prob. 3: Lack of statistical insight. Live text commentary inherently focus on descriptive commentary due to the absence of video, which differs from live televised commentary. As a result, previous models lack the incorporation of statistical knowledge. To tackle the three key discrepancies between end-to-end soccer commentary models and human-centric televised commentary, we propose GameSight , a two-stage model that addresses soccer commentary generation as a knowledge-enhanced visual reasoning task. We move beyond the limited end-to-end captioning by decoupling the task into visual entity grounding and knowledge-driven refinement. This design mimics the cognitive process of human commentators: first identifying the participants through visual and contextual cues, and then enriching the narrative with deep domain knowledge, as illustrated in Fig. 1 . By integrating both internal game context and external historical statistics, GameSight produces commentary that is not only visually accurate but also contextually profound. In summary, our contributions are three-fold: • We introduce soccer commentary generation as a knowledge-enhanced visual reasoning task, bridging the gap between automated live-text like commentary and human-centric live televised like commentary. • We propose GameSight , a two-stage system that leverages fine-grained visual and contextual information to conduct entity alignment and knowledge enhancement for commentary. • Experiments show that GameSight outperforms in both segment-level and game-level to generate accurate, informative and engaging commentary, showcasing the capability of AI-based method to enrich the viewing experience in sports analysis. II Related Works II-A Automatic Soccer Game Analysis Automatic soccer game analysis aimed at reconstructing detailed aspects of a game with two major intentions, to recognize the players or detect the events . On the player recognition front, jersey number recognition [ 3 , 1 ] and game state reconstruction [ 39 ] have achieved significant progress. However, they usually rely on high quality, single full-view camera footage and individual player tracking [ 4 ] for identification. Event classification methods [ 48 , 40 ] employ complex pipelines with decomposed subtasks, for example, shot boundary detection [ 34 ] and play-break segmentation [ 42 ] . Soccer commentary generation represents a higher-level abstraction of event classification to further describe the scene. SoccerNet-Caption [ 25 ] firstly introduced anonymized soccer commentary, which has become the standard task format [ 37 , 20 ] . MatchVision [ 36 ] proposed a unified framework with a pretrained spatiotemporal visual encoder. TimeSoccer [ 47 ] enables game-level commentary generation with global context modeling. Despite significant progress in commentary generation, there remain limited efforts in aligning player with event. Goal [ 31 ] generates commentary with knowledge triples. However, it suffers from entity mismatching and knowledge deficiency In this paper, we perform player alignment from anonymous commentary via a two-stage knowledge-enhanced visual reasoning framework. Our method achieves high entity alignment accuracy and effective incorporation of professional domain knowledge. II-B Reasoning with Video-LMMs Video large multi-modal models (Video-LMMs) [ 19 , 45 ] have shown remarkable performance in zero-shot video understanding, video grounding, optical character recognition, and event captioning with timestamps. These capabilities make them inherently suited for addressing various subtasks in soccer video analysis. Despite their potential, Video-LMMs still face challenges in reasoning within multi-modal contexts [ 17 , 16 ] , particularly when it comes to integrating knowledge from specialized domains like soccer. Related cross-modal alignment work, such as HarmoniVox [ 52 ] , studies audiovisual harmony, but does not address soccer-specific temporal reasoning or entity-aligned commentary generation. The domain adaptation of VLM [ 13 ]