CSPO: Alleviating Reward Ambiguity for Structured Table-to-LaTeX Generation
Yunfan Yang, Cuiling Lan, Jitao Sang, Yan Lu · 2026
Read CSPO: Alleviating Reward Ambiguity for Structured Table-to-LaTeX Generation by Yunfan Yang, Cuiling Lan, Jitao Sang, Yan Lu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-10918.txt.
Introduction
Yang and three colleagues propose CSPO, Component-Specific Policy Optimization, a reinforcement learning frame for table-to-LaTeX work that untangles tuning across parts: structure, style, content. Standard post-training uses one lumped reward, blurring behaviors and blocking effective tuning. CSPO gives each part its own reward and sends each signal only through tokens tied to that part, easing ambiguity and enabling targeted part-wise tuning. With layered metrics, broad tests show CSPO works well, underscoring part-wise tuning for reliable structured generation. Code is available on GitHub.
Catalog record
| CREATOR | Yunfan Yang, Cuiling Lan, Jitao Sang, Yan Lu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53713 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.10918
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
CSPO: Alleviating Reward Ambiguity for Structured Table-to-LaTeX Generation Yunfan Yang, Cuiling Lan, Jitao Sang, Yan Lu Abstract Tables contain rich structured information, yet when stored as images their contents remain “locked” within pixels. Converting table images into LaTeX code enables faithful digitization and reuse, but current multimodal large language models (MLLMs) often fail to preserve structural, style, or content fidelity. Conventional post-training with reinforcement learning (RL) typically relies on a single aggregated reward, leading to reward ambiguity that conflates multiple behavioral aspects and hinders effective optimization. We propose C omponent- S pecific P olicy O ptimization (CSPO), an RL framework that disentangles optimization across LaTeX tables components—structure, style, and content. In particular, CSPO assigns component-specific rewards and backpropagates each signal only through the tokens relevant to its component, alleviating reward ambiguity and enabling targeted component-wise optimization. To comprehensively assess performance, we introduce a set of hierarchical evaluation metrics. Extensive experiments demonstrate the effectiveness of CSPO, underscoring the importance of component-specific optimization for reliable structured generation. 1 Introduction Figure 1: Motivating example of reward ambiguity in table image-to-LaTeX generation. denotes a positive reward, while denotes a penalty. (a) Given a table image, multiple LaTeX sequences are generated with varied errors in structure, content, and style (errors marked with ✗ ). Using a single global reward leads to reward ambiguity , where (a1) an incorrect structure is erroneously reinforced, (a2) correct content is unfairly penalized, and (a2) and (a3) receive identical rewards despite differing component fidelity. (b) Our method mitigates this by decomposing the LaTeX code into functional components and assigning component-specific rewards for targeted optimization, thereby alleviating reward ambiguity . Figure 2: Overview of Component-Specific Policy Optimization (CSPO) . Particularly, CSPO decomposes each generated code sequence into functional components ( e . g ., structure, cell appearance, caption, package inclusion, alignment, and line style) using a LaTeX parser. It conducts component-specific rewarding by assessing each component’s fidelity (with a strong LLM as the judge), performs component-specific credit assignment and optimization , effectively mitigating reward ambiguity in table image-to-LaTeX generation. Scientific documents often contain complex tables that encapsulate critical data and insights ( Gemelli et al. 2024 ; Xia et al. 2024 ; Jiang et al. 2025 ) . However, when these tables are embedded in images—such as screenshots or PDFs—their structured information becomes locked within pixels, hindering data extraction, analysis, and reuse. Accurately converting table images into structured LaTeX code is therefore a crucial step toward reliable digitalization and seamless editing. Recent studies have explored the vision-to-code problem through specialized systems tailored for table understanding. LATTE ( Jiang et al. 2025 ) introduces iterative error localization and correction, while DocGenome ( Xia et al. 2024 ) fine-tunes Pix2Struct ( Lee et al. 2023 ) for table parsing. Beyond these task-specific efforts, multimodal large language models (MLLMs) such as GPT-4o and Qwen2.5-VL have demonstrated strong visual-to-text generalization, enabling zero-shot LaTeX generation. Nevertheless, both specialized systems and general MLLMs often introduce structural inconsistencies ( e . g ., incorrect cell merges), lose fine-grained formatting details ( e . g ., mismatched lines), content mistakes, or generate non-compilable code. This motivates the question: how can we effectively align MLLMs for this highly structured table generation? Reinforcement learning (RL) has become a dominant paradigm for post-training alignment, yielding substantial improvements in reasoning, programming, and mathematical problem solving ( Ouyang et al. 2022 ; Qu et al. 2025 ; Yu et al. 2025b ; Guo et al. 2025 ) . However, its use in table image-to-LaTeX generation remains largely underexplored. Unlike free-form language tasks ( e . g ., visual question answering), table-to-LaTeX generation presents distinct challenges, including multi-faceted fidelity (covering structure, content, style), hierarchical syntax ( e . g ., properly nested tabular and multicolumn structures), and execution sensitivity (the code part in Figure 2 shows an example of LaTeX code sequence). Existing RL approaches typically compute a single aggregated reward across the entire output sequence ( Shao et al. 2024 ; Yu et al. 2025a ; Ling et al. 2025 ) . For such a highly structured sequence generation task, this global aggregation is problematic, as it introduces reward ambiguity—where fundamentally heterogeneous aspects such as structure, content, and style are collapsed into a single undifferentiated signal/reward. Consequently, the model struggles to assign credit accurately, leading to unreliable gradients and limited fidelity improvements . Figure 1 illustrates this issue: in (a1), an incorrect structure component receives a positive global reward, mistakenly reinforcing the error; in (a2), the correct content component is wrongly penalized; (a2) and (a3) receive identical aggregated rewards despite differing in component fidelity, failing to distinguish good from bad performance in individual components. To address this challenge, we propose Component-Specific Policy Optimization (CSPO), an RL framework specifically designed to mitigate reward ambiguity by assigning dedicated rewards to distinct functional components of LaTeX tables. As illustrated in Figure 2 , CSPO combines a global reward that captures overall output quality with component-specific rewards that disentangle structure, content, and style. During RL training, CSPO employs a LaTeX parser to decompose each generated code sequence into fine-grained functional components, including package imports, structure, cell appearance, captions, alignment, and line style. It then performs component-specific rewarding by evaluating each component’s fidelity, conducts component-specific credit assignment and optimization , leading to more reliable component-level policy optimization. This approach ensures that improvements in one component are not overshadowed by errors in others, thereby enhancing the model’s ability to generate faithful LaTeX code. Furthermore, to enable more diagnostic and interpretable evaluation, we introduce a suite of hierarchical metrics . Beyond global similarity and compilation checks, these metrics separately measure structural correctness ( e . g ., row/column spans), content fidelity ( e . g ., cell values), and stylistic consistency ( e . g ., line style, font styles), providing granular feedback on model performance. Our contributions are summarized as follows: • We identify reward ambiguity as a fundamental challenge in RL-based structured sequence generation for table-to-LaTeX conversion. • We propose CSPO, an effective RL framework that addresses reward ambiguity through component-specific rewarding and explicit credit assignment , enabling reliable and controllable component-specific optimization . • We introduce hierarchical evaluation metrics for comprehensive and diagnostic assessment, providing useful signals for guiding future table-to-LaTeX model design and optimization. Experimental results demonstrate the effectiveness of our CSPO, highlighting the importance of addressing reward ambiguity and offering a general blueprint for structured sequence generation tasks. 2 Related Work Table Image to Structured Markup. Research on image-based table recognition has evolved from early detection and structure-parsing pipelines to end-to-end systems that directly map images to structured markup. Benchmarks such as PubTabNet ( Zhong et al. 2020 ) were pivotal in this transition, introducing large-scale supervision for image-to-HTML conversion. Encoder–decoder architectures ( e . g ., EDD Zhong et al. 2020 ) focused on HTML/XML outputs, motivating subsequent methods in image-to-structure generation. Some recent studies adopted LaTeX as the target format for its precise layout control, publication-quality rendering, and seamless integration with scientific workflows—capabilities that HTML lacks. DocGenome ( Xia et al. 2024 ) fine-tuned Pix2Struct ( Lee et al. 2023 ) for table image-to-LaTeX conversion. LATTE ( Jiang et al. 2025 ) introduced iterative refinement with delta-view correction to improve renderable LaTeX extraction from