SketchVL: Policy Optimization via Fine-Grained Credit Assignment for Chart Understanding and More
Muye Huang, Lingling Zhang, Yifei Li, Yaqiang Wu, Jun Liu · 2026
Read SketchVL: Policy Optimization via Fine-Grained Credit Assignment for Chart Understanding and More by Muye Huang, Lingling Zhang, Yifei Li, Yaqiang Wu, Jun Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-05688.txt.
Introduction
Muye Huang and 4 co-authors introduce SketchVL, a 2026 multimodal model that improves chart understanding by literally drawing its reasoning onto the image. Charts pack dense data into visual form, and automated reading requires precise multi-step reasoning. Standard reinforcement training scores whole answers, so it cannot tell which reasoning steps were right or wrong.
SketchVL marks intermediate steps directly on the chart and feeds the annotated image back to itself. During training, a fine-grained reward model scores each drawing action, so correct steps earn more credit and wrong steps draw stronger penalties even inside mixed trajectories. Reported gains over the base model averaged several percent across chart, natural image, and mathematics datasets.
Catalog record
| CREATOR | Muye Huang, Lingling Zhang, Yifei Li, Yaqiang Wu, Jun Liu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-05688
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SketchVL: Policy Optimization via Fine-Grained Credit Assignment for Chart Understanding and More Muye Huang, Lingling Zhang, Yifei Li, Yaqiang Wu, Jun Liu Abstract Charts are high-density visual carriers of complex data and medium for information extraction and analysis. Due to the need for precise and complex visual reasoning, automated chart understanding poses a significant challenge to existing Multimodal Large Language Models (MLLMs). Many MLLMs trained with reinforcement learning (RL) face the challenge of credit assignment. Their advantage estimation, typically performed at the trajectory level, cannot distinguish between correct and incorrect reasoning steps within a single generated response. To address this limitation, we introduce SketchVL, a novel MLLM that optimized with FinePO, a new RL algorithm designed for fine-grained credit assignment within each trajectory. SketchVL’s methodology involves drawing its intermediate reasoning steps as markers on the image and feeding the annotated image back to itself, creating a robust, multi-step reasoning process. During training, the FinePO algorithm leverages a Fine-grained Process Reward Model (FinePRM) to score each drawing action within a trajectory, thereby precisely assigning credit for each step. This mechanism allows FinePO to more strongly reward correct tokens when a trajectory is globally successful, and more heavily penalize incorrect tokens when the trajectory is globally suboptimal, thus achieving fine-grained reinforcement signals. Experiments show that SketchVL learns to align its step-level behavior with the FinePRM, achieving an average performance gain of 7.23% over its base model across chart datasets, natural image datasets, and mathematics, providing a promising new direction for training powerful reasoning models. 1 Introduction Charts, as a primary method of data visualization, are capable of presenting data with accuracy and intuition. They are widely found in technical documents, business reports, and scientific papers. The automated understanding of charts is instrumental in advancing the automated analysis of documents, and its significant research value has progressively drawn widespread attention from researchers. Recent advanced Multimodal Large Language Models, including Qwen2.5VL 1 , Gemma3 41 , GPT-5 35 , and Gemini-2.5 4 , have shown numerous improvements in chart understanding. Concurrently, RL in the multimodal domain, demonstrated by approaches like Vision-R1 15 and VLM-R1 38 , has been shown to effectively enhance models’ multimodal understanding capabilities. Figure 1 : Illustration of our SketchVL FinePO process. SketchVL first decomposes a complex query into a trajectory of visual actions, which are scored by a FinePRM. FinePO then achieves credit assignment by redistributing credit based on the FinePRM scores. Here, Delta_{1-3} represent the process rewards for Actions 1-3, calculated from their FinePRM scores, and the preceding sign indicates whether an action is considered relative advantaged or disadvantaged within the trajectory. The detailed calculation is described in Section 3 . Building on these advances, chart understanding itself exposes a distinctive, stepwise structure: models must first localize legends and axes, then read values, align categories, compare trends, and finally synthesize conclusions. Any minor slip, such as an imprecise crop, a misread tick, or a mismatched legend, can derail the entire chain of reasoning. This step dependency makes chart analysis particularly sensitive to how reinforcement signals are assigned. However, most prevailing RL practices for MLLMs still deliver coarse, outcome-only feedback. Methods such as GRPO 6 compute a single advantage from the final result and broadcast it uniformly to all tokens in the trajectory. When applied to charts’ multi-step pipelines, such uniform credit can inadvertently punish otherwise sound intermediate logic in an incorrect answer, or reward flawed steps simply because the final prediction happens to be right—injecting noise and limiting the benefits of RL. These characteristics call for fine-grained credit assignment: assessing and reinforcing each intermediate step along the reasoning chain. By rewarding correct sub-decisions and precisely penalizing missteps, the learning signal becomes sharper and lower-noise, better matching the compositional nature of chart analysis. Motivated by these observations, we introduce SketchVL, a novel multimodal interactive reasoning model. SketchVL operates within the Reasoning on Image (RoI) paradigm, exemplified by works like ChartSketcher 14 and DeepEyes 60 . This paradigm requires the model to externalize its intermediate reasoning steps into a visible trajectory of marking actions on the image. This explicit, step-by-step decomposition of the reasoning process provides the necessary structure for step-level credit assignment. We optimize SketchVL using our FinePO algorithm, as shown in Figure 1 , which is specifically designed to operate on these action trajectories. FinePO first computes a relative advantage score for each complete trajectory by making comparisons within a group, an approach inspired by GRPO. It then introduces its central mechanism of secondary credit redistribution. This process leverages a specially designed Fine-grained Process Reward Model (FinePRM) to evaluate the quality of each individual action within the trajectory. Based on the FinePRM scores, FinePO ultimately redistributes the trajectory’s overall relative advantage among its constituent actions according to their contributions. To implement SketchVL, we followed the common ‘ColdStart-RL’ training paradigm. For the cold start phase, we reused parts of the ChartSketcher data pipeline to construct a dataset of 50K samples. For the subsequent reinforcement learning phase, we collected 9K mixed-domain data points. Concurrently, to support the FinePO algorithm, we built a sophisticated cross-modal distillation pipeline to generate training data for the FinePRM. This pipeline first decomposes dense visual information from charts into structured textual annotations. A LLM is then used to distill these annotations into a large-scale set of ‘intent-action’ multimodal training pairs, complete with both positive and negative samples. In total, we collected 473K data points through this method, specifically for training the FinePRM’s capability to assess process quality. Our main contributions are threefold: • We introduce SketchVL, a powerful MLLM based on the Reasoning on Image paradigm. We also contribute two large-scale datasets: a 473K multimodal dataset for training FinePRM, and a 50K dataset for RoI cold start. • We propose FinePO, a novel reinforcement learning algorithm that addresses the coarse reward problem in conventional RL through fine-grained, step-level credit assignment. By leveraging a dedicated process reward model FinePRM, FinePO provides a more precise and stable learning signal for policy optimization. • We conduct extensive experiments across multiple benchmarks, demonstrating the effectiveness of FinePO. Our ablation studies further underscore the important role of credit assignment in enhancing RL performance. 2 Related Work Chart Understanding. Research in Chart Understanding focuses on interpreting the visual context of charts for tasks such as question answering and summarization. The seminal work of FigureQA 17 presented a pipeline for chart comprehension, addressing binary classification for related questions. Subsequent research advanced these capabilities through the use of multi-component systems 28 ; 20 ; 25 ; 21 ; 44 . DePlot 61 , for example, utilized several components alongside the mathematical prowess of LLMs to improve results on the PlotQA benchmark 33 . The advent of MLLMs has shifted the paradigm, leading to MLLM-centric approaches becoming the mainstream 30 ; 31 ; 2 ; 13 ; 47 ; 46 ; 58 . By strategically constructing training data and fine-tuning LLaVA 23 , ChartLlama 11 produced a capable chart-expert model. Capitalizing on the inherent language capabilities of MLLMs, recent work has adopted multi-task training to enhance chart comprehension. ChartAssistant 32 , for instance, employed unified multi-task training for better overall performance. The Program-of-Thoughts technique has been applied by TinyChart 57 to boost numerical reasoning, while ChartMoE