Breaking the SFT Plateau: Multimodal Structured Reinforcement Learning for Chart-to-Code Generation

Lei Chen, Xuanle Zhao, Zhixiong Zeng, Jing Huang, Liming Zheng, Yufeng Zhong & Lin Ma MeituanBeijing, China [email protected], [email protected] Project: https://github.com/DocTron-hub/MSRL · 2025

Read Breaking the SFT Plateau: Multimodal Structured Reinforcement Learning for Chart-to-Code Generation by Lei Chen, Xuanle Zhao, Zhixiong Zeng, Jing Huang, Liming Zheng, Yufeng Zhong & Lin Ma MeituanBeijing, China [email protected], [email protected] Project: https://github.com/DocTron-hub/MSRL (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-13587.txt.

Introduction

Lei Chen and colleagues at Meituan studied chart-to-code generation in 2025, the task of turning a chart image into working code that redraws it. The job demands close reading of a dense image and exact structured output, and ordinary supervised training had stopped improving. The team first built a very large training set from real tables in research papers.

Even with millions of chart and code pairs and the best reported scores, simply adding more supervised data gave smaller and smaller gains. To break that ceiling they propose a reinforcement learning method with rewards at several levels of detail, rewarding both the overall structure and finer matches. The work shows where plain supervised training runs out and how targeted rewards can push a vision and language model further.

Catalog record

CREATORLei Chen, Xuanle Zhao, Zhixiong Zeng, Jing Huang, Liming Zheng, Yufeng Zhong & Lin Ma MeituanBeijing, China [email protected], [email protected] Project: https://github.com/DocTron-hub/MSRL
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-13587

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Breaking the SFT Plateau: Multimodal Structured Reinforcement Learning for Chart-to-Code Generation Lei Chen, Xuanle Zhao, Zhixiong Zeng, Jing Huang, Liming Zheng, Yufeng Zhong & Lin Ma MeituanBeijing, China [email protected], [email protected] Project: https://github.com/DocTron-hub/MSRL Abstract While reinforcement learning (RL) has proven highly effective for general reasoning in vision-language models, its application to tasks requiring in-depth understanding of information-rich images and the generation of structured outputs remains underexplored. Chart-to-code generation is a prime example of this challenge, as it demands complex reasoning over visual charts to generate structured code. Therefore, supervised fine-tuning (SFT) alone is often insufficient, highlighting a critical need for effective RL strategies that appropriately reward the structured outputs. In this paper, we systematically investigate the performance plateau in SFT through large-scale experiments and propose Multimodal Structured Reinforcement Learning (MSRL) for chart-to-code generation, which substantially breaks through this plateau. To this end, we first construct the largest training corpus to date, containing 3 million chart-code pairs. To mitigate the simplistic patterns of prior synthetic data, we curate real-world tables from arXiv papers as the data source. Despite reaching a state-of-the-art performance, our experiments demonstrate that simply scaling the SFT data eventually hits a performance plateau where further increases yield negligible improvements. To break this plateau, our MSRL method leverages a multi-granularity structured reward system that uses multimodal textual and visual feedback to guide the RL training. At the textual level, rule-based rewards validate the fidelity of fine-grained code details. At the visual level, a model-based reward is derived by first rendering the generated code into an image and then employing an evaluator model to assess its structural similarity to the ground-truth chart. To ensure training stability, we implement this multimodal reward within a two-stage curriculum, where the model initially trains on textual rewards alone before progressing to the combined textual and visual signals. Experimental results demonstrate that our MSRL framework significantly breaks the SFT plateau, improving the high-level metrics by 6.2% and 9.9% on ChartMimic and ReachQA benchmarks respectively. Notably, our method significantly outperforms all methods in the chart domain and achieves competitive performance with advanced closed-source models for the first time. † † footnotetext: † Project leader. ∗ Corresponding author. 1 Introduction Large language models (LLMs) have demonstrated impressive reasoning capabilities on complex textual problems, including code generation and mathematical problem-solving OpenAI 2025b ; Guo et al. 2025 . Pivotal to this success is the implementation of reinforcement learning (RL) to optimize the capacities of LLM to generate step-by-step reasoning Wei et al. 2022 and verify final answers Shao et al. 2024 . While multimodal large language models (MLLMs) have achieved impressive performance on established visual reasoning tasks like visual question answering Zhang et al. 2024a ; Huang et al. 2025 , their core reasoning capabilities remain limited, especially when tasks requiring processing information-dense images like charts Xu et al. 2024 ; Chen et al. 2025a , and generating structured outputs like code Xia et al. 2024 ; Zhao et al. 2025 . Chart-to-code generation, a complex task characterized by dense visual inputs and structured code outputs, has recently garnered significant attention Shi et al. 2024 ; Zhang et al. 2024c . Although previous works have attempted to solve this task by using methods like Supervised Fine-Tuning (SFT) Zhang et al. 2024b ; Zhao et al. 2025 or Direct Preference Optimization (DPO) Rafailov et al. 2023 ; Zhang et al. 2025 on synthetic data, their performance remains limited due to the simplistic patterns of the corpus and ineffective generalization strategies. Therefore, a promising direction is to leverage more realistic data and robust strategies to enhance model performance. Figure 1: The SFT plateau and RL performance gain from our experiments. The left figure illustrates that scaling SFT data from 200k to 2.8M leads to a performance plateau after exceeding 2M data points. The right figure denotes the performance gain from our proposed MSRL training strategy. In this work, we first systematically investigate the SFT performance plateau through experiments at various data scales. Based on this analysis, we propose a multimodal structured reinforcement learning (MSRL) strategy designed to optimize the model at multiple granularities. Specifically, to investigate the performance plateau of SFT, we first construct the largest training corpus to date, which covers 24 distinct chart types and comprises 3 million chart-code pairs. To ensure the charts reflect realistic distributions and avoid monotonous trends, we design a two-stage generation pipeline that curates high-quality tables from arXiv papers and subsequently leverages these data to generate diverse and complex plot codes utilizing LLMs. Unlike previous works that apply RL directly after SFT Ni et al. 2025 ; Chen et al. 2025b , our approach first investigates the performance plateau of SFT models. Establishing this baseline is crucial for isolating the true effectiveness of the MSRL strategy from gains that merely compensate for an unsaturated SFT model. To this end, we perform SFT on models with various data scales and reveal that simply increasing the SFT data volume leads to a performance plateau, which cannot be overcome by data quantity alone. To break through this plateau, we propose the MSRL training strategy, which incorporates multi-granularity reward functions from both textual and visual aspects to optimise the model. For the textual aspect, to handle format diversity in code generation, we first filter our training corpus to enforce a consistent structure, making it suitable for information extraction. Building on this standardized data, we develop a comprehensive reward function that verifies code correctness from five key perspectives. However, a reward function based solely on text is often insufficient, as it struggles to capture the overall structures of charts. To address this limitation, we introduce a visual reward mechanism. The core of this method is a render-and-compare process that involves rendering the generated code into an image and then employing an MLLM to score the visual similarity between the rendered image and the original input chart. By leveraging its multi-granularity reward function, MSRL optimizes both global context and fine-grained details. We evaluate models on various chart-to-code benchmarks Shi et al. 2024 ; He et al. 2024 . The results demonstrate that it establishes a new state-of-the-art, as it outperforms all previous open-source models and rivals the performance of powerful proprietary MLLMs. In summary, the main contributions of this work are as follows: • We propose an MSRL training strategy that employs a multi-level reward function, which combines a rule-based component for assessing textual code correctness with a model-based evaluator for evaluating the visual fidelity of the rendered chart. Benefits from the MSRL training strategy, our proposed model achieves state-of-the-art (SOTA) performance compared to all other open-source MLLMs. • We construct the first large-scale chart-to-code training corpus, generated using real-world tables as a data source. After applying several filtering methods, we curate a final dataset of 2.8 million samples for SFT and 33 thousand for RL. • Through SFT experiments at various data scales, we identify the performance bottleneck, confirming that merely increasing SFT data quantity is insufficient to improve performance. 2 Related Work 2.1 Chart MLLMs Chart understanding is a crucial research area encompassing both low-level tasks, such as data extraction Liu et al. 2022 , and high-level tasks like question answering Masry et al. 2022 and summarization Kantharaj

Read the complete text