BoxComm: Benchmarking Category-Aware Commentary Generation and Narration Rhythm in Boxing DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; XX XX–XX, 2026; XX, XX ISBN: 978-1-4503-XXXX-X/2026/06 CCS: Computing methodologies Computer vision CCS: Information systems Multimedia streaming Kaiwen Wang, Kaili Zheng, Rongrong Deng, Yiming Shi, Chenyi Guo, Ji Wu Abstract . Recent multimodal large language models (MLLMs) have shown strong capabilities in general video understanding, driving growing interest in automatic sports commentary generation. However, existing benchmarks for this task focus exclusively on team sports such as soccer and basketball, leaving combat sports entirely unexplored. Notably, combat sports present distinct challenges: critical actions unfold within milliseconds with visually subtle yet semantically decisive differences, and professional commentary contains a substantially higher proportion of tactical analysis compared to team sports. In this paper, we present BoxComm , a large-scale dataset comprising 445 World Boxing Championship match videos with over 52K commentary sentences from professional broadcasts. We propose a structured commentary taxonomy that categorizes each sentence into play-by-play, tactical, or contextual, providing the first category-level annotation for sports commentary benchmarks. Building on this taxonomy, we introduce two novel and complementary evaluations tailored to sports commentary generation: (1) category-conditioned generation, which evaluates whether models can produce accurate commentary of a specified type given video context; and (2) commentary rhythm assessment, which measures whether freely generated commentary exhibits appropriate temporal pacing and type distribution over continuous video segments, capturing a dimension of commentary competence that prior benchmarks have not addressed. Experiments on multiple state-of-the-art MLLMs reveal that current models struggle on both evaluations. We further propose EIC-Gen, an improved baseline incorporating detected punch events to supply structured action cues, yielding consistent gains and highlighting the importance of perceiving fleeting and subtle events for combat sports commentary. Our dataset and code are publicly available at https://gouba2333.github.io/BoxComm/. Figure 1. Overview of general commentary taxonomy, motivation and our BoxComm commentary benchmark. 1. Introduction Sports commentary generation is a challenging multimodal problem that lies at the intersection of video understanding, domain-specific reasoning, and language generation. A competent commentary system must not only recognize what is happening in a match, but also understand why it matters and decide when to verbalize it. Such systems have broad practical value, including improving accessibility for visually impaired audiences, scaling content production for broadcasting platforms, and enabling interactive fan experiences ( 34 ; 2 ) . Recent advances in multimodal large language models (MLLMs) ( 13 ; 30 ; 18 ; 1 ) have demonstrated strong capabilities in general video understanding, motivating research into their application for sports commentary generation. To support this, several datasets and benchmarks have been proposed for sports-related tasks. For instance, SoccerNet-Caption ( 21 ) and MatchTime ( 25 ) provide large-scale soccer commentary data, while BH-Commentary ( 39 ) focuses on basketball. These datasets enable the evaluation of models on commentary generation and facilitate the development of methods that incorporate multimodal reasoning. Subsequent studies ( 16 ; 38 ; 24 ) have applied MLLMs to these benchmarks, showing that fine-tuning on domain-specific data can improve performance. Despite these efforts, existing benchmarks focus exclusively on team sports, implicitly assuming that the challenges of commentary generation are largely shared across sporting disciplines. However, combat sports (such as boxing) present distinct challenges in two key respects. First, the actions of interest occur at an extremely fast pace, often within hundreds of milliseconds, and the visual differences between action types are subtle (e.g., a jab versus a hook differs primarily in arm trajectory) yet carry decisive semantic implications for the narrative. This stands in contrast to team sports, where key events such as goals, passes, and fouls are typically more visually salient and temporally spaced. Second, the commentary structure of combat sports differs markedly from that of team sports. Following the commentary typology discussed in ( 7 ) , we identify three types of professional sports commentary: play-by-play (real-time description of ongoing action), tactical (analysis of strategy, technique, and matchup dynamics), and contextual (background information such as fighter records, historical significance, and crowd reactions). As illustrated in Figure 1 , our analysis reveals that the commentary of combat sports exhibits a substantially higher proportion of tactical content (45.6% vs 21.7%) compared to team sports, where play-by-play dominates. This shift reflects that in combat sports, the strategic dimension including reading the opponent, controlling distance, choosing when to engage, is central to both the competition and its narration. To fill this gap, we introduce BoxComm, a large-scale dataset for boxing commentary generation. BoxComm comprises 445 World Boxing Championship match videos with over 52K sentence-level commentary annotations drawn from professional broadcasts. Each commentary sentence is temporally aligned with the video and annotated with its commentary type (play-by-play, tactical, or contextual), making BoxComm the first sports commentary benchmark to provide category-level annotations. We design two complementary evaluations to assess different aspects of commentary generation ability. The first, category-conditioned commentary generation, provides the model with a video segment, preceding commentary context, and a target commentary type, and evaluates the accuracy of the generated sentence. This evaluation measures whether the model can produce commentary that is grounded in the video content and consistent with the specified commentary category. The second, commentary rhythm assessment, evaluates generation in a free-form, streaming setting. The model generates commentary continuously over an entire match without category constraints. The resulting sentences are classified into the three commentary types, and their temporal distribution is compared with the ground-truth distribution. This evaluation assesses whether the model can maintain appropriate commentary rhythm and type distribution over time, a key aspect of professional sports narration that has not been addressed in prior benchmarks. We evaluate a range of state-of-the-art MLLMs on both evaluations and find that they exhibit significant limitations in boxing commentary generation. Even after task-specific fine-tuning, the models struggle to accurately describe fast-paced boxing actions and maintain appropriate commentary rhythm. These results indicate that fine-grained perception and domain-specific reasoning for combat sports remain challenging for current MLLMs. To address this, we propose Event-Informed Commentary Generation (EIC-Gen), which augments the MLLM input with structured action cues. Specifically, we employ a dedicated extraction pipeline to capture fine-grained punch events and convert them into concise natural language templates. During the category-conditioned generation task, these explicit text-based action cues are concatenated with the recent commentary history and supplied to the MLLM. This approach yields consistent improvements in sentence-level generation, highlighting the importance of perceiving rapid and subtle actions for accurate combat sports commentary. Our contributions are summarized as follows: • We introduce BoxComm, a large-scale dataset for combat sports commentary generation, containing 445 World Boxing Championship match videos and more than 52,000 sentence-level commentary annotations. Each sentence is labeled as play-by-play, tactical, or contextual, making BoxComm the first sports commentary benchmark with category-level labels. • We develop two evaluation protocols, category-conditioned commentary generation and commentary rhythm assessment, to evaluate models’ ability to generate commentary of the specified type and maintain appropriate type distribution over time. • We benchmark multiple state-of-the-art MLLMs and find that they struggle to generate accurate and temporally appropriate commentary. We also present EIC-Gen, an improved baseline that incorporates punch event cues, demonstrating the importance of perceiving rapid and subtle actions for combat sports commentary. 2. Related Work 2.1. MLLMs for Video Understanding The extension of large language models to multimodal inputs has driven rapid progress in video understanding. Early video MLLMs such as Video-LLaMA ( 40 ) and Video-ChatGPT ( 20 ) established the paradigm of encoding video frames into visual tokens for LLM-based reasoning. More recent models, including GPT-4o ( 13 ) , Gemini ( 27 ) , InternVL3.5 ( 30 ) , LLaVA-Video ( 41 ) , Qwen3-VL ( 3 ) , and Video-LLaVA ( 18 ) , have substantially advanced performance through improved visual encoding, dynamic resolution handling, and large-scale video instruction tuning. Comprehensive benchmarks such as Video-MME ( 8 ) have been developed to evaluate these models across diverse tasks including temporal grounding, video question answering, and dense captioning. The strong performance of MLLMs on general video understanding has naturally motivated their application to specialized domains. Sports, which demands fine-grained perception, real-time temporal reasoning, and domain-specific expertise, has emerged as a compelling testbed for probing the limits of current MLLMs. 2.2. Sports Video Understanding Early research on sports video analysis centered on event detection and action recognition, with datasets such as SoccerNet ( 10 ; 5 ) , NSVA ( 31 ) and TenniSet ( 6 ) providing large-scale annotations for action spotting and player identification across multiple sports. More recently, benchmarks including SPORTU ( 35 ) and SportR ( 33 ) have been proposed to evaluate MLLM capabilities on sports-specific reasoning tasks, revealing a persistent gap between model and expert performance. At the single-sport level, FineBadminton ( 12 ) provides hierarchical annotations from actions to tactics for badminton. A growing line of work targets sports commentary generation specifically. In soccer, several datasets and methods have been proposed ( 21 ; 23 ; 25 ; 38 ; 24 ; 16 ) , advancing from dense video captioning to end-to-end MLLM-based commentary with temporal alignment. For basketball, BH-Commentary ( 39 ) introduced the first dedicated highlight commentary dataset with an end-to-end generation framework. In tennis, work ranges from early rule-based approaches ( 37 ) to recent MLLM-based systems with large-scale multimodal datasets ( 19 ) . SCBench ( 9 ) provides a cross-sport evaluation spanning six team and individual sports. Despite this progress, all existing commentary benchmarks are confined to team sports and racket sports. Combat sports remain entirely unexplored for commentary generation; the only related efforts address low-level action recognition ( 14 ; 26 ; 28 ) rather than narration. Our work fills this gap with the first large-scale boxing commentary benchmark. 3. BoxComm Dataset 3.1. Dataset Collection The BoxComm dataset is built upon 445 live broadcast matches from the 2025 World Boxing Championships. Sourced from official public broadcasting channels, these videos are utilized strictly under the fair use doctrine for non-commercial academic research. These matches represent top-level competitive boxing and cover a wide range of weight classes, fighter stances including orthodox and southpaw, and diverse tactical styles. To accurately capture the rapid and subtle actions characteristic of boxing, such as jabs that may be as brief as 200 milliseconds, all videos were carefully curated to ensure a minimum frame rate of 25 frames per second and a spatial resolution of at least 720p. This high temporal and spatial fidelity is critical for enabling precise annotation and subsequent evaluation of both content accuracy and temporal dynamics in commentary generation. Based on the video audio, the commentary component of BoxComm is extracted to preserve the temporal structure of professional narration. As illustrated in Figure 2 , we first employ WhisperX to generate fine-grained, word-level timestamps through Automatic Speech Recognition (ASR). Raw ASR outputs often contain domain-specific transcription errors and irregular sentence boundaries. To address these issues, we use GPT-5.2 to correct boxing-specific terminology and carefully re-segment transcripts according to semantic completeness and natural speech pauses. Sentences are kept self-contained, without arbitrarily merging short utterances, which preserves action-level play-by-play bursts that often consist of only one or two words, such as “Solid jab!” These micro-level commentary units are essential for evaluating high-dynamic temporal alignment in subsequent commentary generation tasks. Figure 2. The pipeline for commentary extraction, semantic re-segmentation, and category annotation. 3.2. Category Annotation To provide a structured framework for commentary analysis, each segmented sentence in BoxComm is assigned to one of three categories: Play-by-Play, Tactical, or Contextual. We use GPT-5.2 with explicit class definitions and local neighboring context to disambiguate borderline cases The final BoxComm dataset contains 52K commentary sentences that are both temporally aligned with the video and categorized, forming a detailed resource for studying the content and dynamics of professional boxing commentary. 3.3. Dataset Statistics Figure 3. BoxComm Dataset statistics. (a) Commentary category proportions. (b) Temporal category distribution across the match. (c) Fine-grained punch event attributes. We present the detailed statistics of the BoxComm dataset in Figure 3 . The dataset contains a total of 52K commentary sentences. Tactical commentary is the most frequent, comprising 22.3K sentences. The temporal distribution of these categories varies across the match timeline. Tactical commentary maintains a consistently high proportion from the later part of the first round onwards. In contrast, contextual commentary is high at the start of the match and during the breaks between rounds. Furthermore, BoxComm includes a total of 260K fine-grained punch events (The punch event extraction process is described in the Methods section). These events are categorized by punch type and effectiveness. The duration of these punches is short, with the vast majority of actions lasting only between 0.3 and 0.5 seconds. To establish a standard benchmark, we partition the 445 matches into training and evaluation sets based on their chronological order. The first 405 matches constitute the training set (BoxComm-Train), while the final 40 matches are reserved for the evaluation set (BoxComm-Eval). The BoxComm-Eval set contains a total of 5.6K commentary sentences and 23.4K fine-grained punch events. 4. Benchmark Protocols Figure 4. Evaluation protocols: category-conditioned commentary generation and streaming commentary rhythm assessment. To evaluate models on boxing commentary generation, we design two complementary evaluation protocols, Category-Conditioned Generation and Commentary Rhythm Assessment, as illustrated in Figure 4 . We describe these two protocols in detail below. 4.1. Category-Conditioned Generation Category-Conditioned Generation evaluates whether a model can produce commentary that matches a specified commentary type while remaining consistent with the visual content. For each timestamp t in the dataset, the model is given a short video window V_{[t-k,t]} preceding the target moment, where k_{v}=4 seconds, together with the recent commentary history H_{