Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation

Qianhao Yuan, Jie Lou, Xing Yu, Hongyu Lin, Le Sun, Xianpei Han, Yaojie Lu [2pt] · 2026

Read Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation by Qianhao Yuan, Jie Lou, Xing Yu, Hongyu Lin, Le Sun, Xianpei Han, Yaojie Lu [2pt] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-18740.txt.

Introduction

Qianhao Yuan and six co-authors present Vision-OPD, a 2026 self-distillation method for detailed vision. The same model answers questions better from crops than from full images. That gap shows failures come from weak focus, not weak recognition. Vision-OPD builds a crop-conditioned teacher and a full-image student. It minimizes divergence between their next-token distributions on student rollouts. The student thus gains visual zooming with no external teacher or verifiers. On detailed benchmarks, Vision-OPD matches or beats larger open, closed, and agentic models.

Catalog record

CREATORQianhao Yuan, Jie Lou, Xing Yu, Hongyu Lin, Le Sun, Xianpei Han, Yaojie Lu [2pt]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40566
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.18740

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation Qianhao Yuan, Jie Lou, Xing Yu, Hongyu Lin, Le Sun, Xianpei Han, Yaojie Lu [2pt] Abstract Multimodal Large Language Models (MLLMs) still struggle with fine-grained visual understanding, where answers often depend on small but decisive evidence in the full image. We observe a regional-to-global perception gap: the same MLLM answers fine-grained questions more accurately when conditioned on evidence-centered crops than on the corresponding full images, suggesting that many failures stem from difficulty to focus on relevant evidence rather than insufficient local recognition ability. Motivated by this observation, we propose Vision-OPD (Vision On-Policy Distillation), a regional-to-global self-distillation framework that transfers the model’s own privileged regional perception to its full-image policy. Vision-OPD instantiates two conditional policies from the same MLLM: a crop-conditioned teacher and a full-image-conditioned student. The student generates on-policy rollouts, and Vision-OPD minimizes token-level divergence between the teacher and student next-token distributions along these rollouts. This enables the model to internalize the benefit of visual zooming without external teacher models, ground-truth labels, reward verifiers, or inference-time tool use. Experiments on multiple fine-grained visual understanding benchmarks show that Vision-OPD models achieve competitive or superior performance against much larger open-source, closed-source, and “Thinking-with-Images” agentic models. The code is available at https://github.com/VisionOPD/Vision-OPD . 1 1 footnotetext: Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences 2 University of Chinese Academy of Sciences 3 Xiaohongshu Inc. $\dagger$ $\dagger$ footnotetext: [email protected] , [email protected] , {hongyu,sunle,xianpei,luyaojie}@iscas.ac.cn Figure 1: Average scores across fine-grained visual understanding benchmarks, including V* Bench, ZoomBench, HR-Bench 4K, HR-Bench 8K, MME-RealWorld-EN and MME-RealWorld-CN. 1 Introduction Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in general visual understanding and reasoning [ 7 , 40 , 10 , 31 ] . However, they still struggle with fine-grained visual understanding, where the answer often depends on small but decisive details that occupy only a fraction of the image [ 67 , 45 , 23 ] . In full-image inference, these details are often easy to overlook amid many visual tokens. As a result, MLLMs may produce plausible answers based on the global scene while missing the local evidence that is truly needed for the question. Recent “Thinking-with-Images” methods [ 77 , 70 ] address this issue by equipping MLLMs with agentic visual tool use, enabling them to crop, zoom, and inspect the region of interest during inference. By making local evidence more salient, these methods improve fine-grained visual understanding. However, they introduce substantial inference overhead due to repeated image encoding and model calls. A natural question arises: can the benefit of visual zooming be internalized through training, so that the model can use fine-grained evidence from the full image without additional tool use? Our motivation comes from a simple observation: the same MLLM often answers a fine-grained question more accurately when conditioned on the evidence-centered crop than on the corresponding full image. This performance gap reveals a broader bottleneck: MLLMs can often interpret the relevant evidence once it is made salient, but struggle to exploit it when it is embedded in the global visual context. This observation suggests a natural training signal: the model’s own crop-conditioned behavior can serve as privileged supervision for improving its full-image behavior. A straightforward way to exploit such privileged supervision is supervised fine-tuning (SFT) on crop-conditioned responses. However, this trains the model on trajectories generated under privileged crop inputs, leading to distribution mismatch and exposure bias [ 1 ] . Reinforcement learning with verifiable rewards methods, such as GRPO [ 38 ] and DAPO [ 59 ] , can optimize on-policy rollouts, but usually provide only sparse sequence-level feedback, and require ground-truth labels and verifiers. On-Policy Distillation (OPD) [ 1 , 26 ] combines on-policy sampling with dense token-level supervision, but existing OPD methods typically rely on external stronger teachers or ground-truth label [ 75 ] . We propose Vision-OPD, a regional-to-global self-distillation framework for fine-grained visual understanding. It instantiates two policies from the same MLLM with different visual conditions: a crop-conditioned teacher that observes the evidence-centered crop as a privileged input, and a full-image-conditioned student that observes the full image as in standard inference. The student first generates on-policy rollouts from the full image. For each student-generated prefix, Vision-OPD computes the logit distributions of both the crop-conditioned teacher and the full-image-conditioned student, and minimizes their divergence. In this way, the model transfers its own privileged crop-conditioned behavior to its full-image-conditioned policy on the student’s generation trajectory, without external teachers, ground-truth labels, reward verifiers, or inference-time visual tool use. We conduct extensive experiments to validate the effectiveness of Vision-OPD. With only 6.2K synthetic training data, Vision-OPD enables a 9B model to outperform much larger open-source models (e.g., Qwen3.5-397B), closed-source models (e.g., GPT-5.4, Gemini-3.1-Pro, Gemini-3.5-Flash), and agentic “Thinking-with-Images” methods on fine-grained visual understanding tasks that require dense local evidence. On hold-out tasks beyond the training distribution, Vision-OPD maintains general visual understanding and reasoning ability, indicating that the gains do not come at the cost of forgetting. Further analyses verify the necessity of on-policy sampling and dense token-level supervision, and show that Vision-OPD substantially narrows the regional-to-global perception gap in MLLMs. Our contributions are summarized as follows: • We introduce a regional-to-global self-distillation formulation for fine-grained visual understanding, with privileged crop-conditioned behavior as supervision for full-image inference. • We propose Vision-OPD, an on-policy self-distillation framework where a crop-conditioned policy supervises a full-image policy on the student’s rollouts via token-level supervision. • Comprehensive experiments validate the effectiveness of Vision-OPD. We demonstrate that Vision-OPD can significantly narrow the regional-to-global gap, and the on-policy sampling and dense supervision are important to its success. 2 Preliminary: distillation and on-policy distillation (OPD) Knowledge distillation transfers the behavior of a strong teacher into a weaker student by matching the teacher’s distributions. For an input x and output sequence y=(y_{1},\ldots,y_{|y|}) , traditional supervised distillation is off-policy: the student is trained on teacher-induced prefixes y_{<t} and minimizes \mathcal{L}_{\mathrm{Supervised\ Distillation}}(\theta)=\mathbb{E}_{(x,y)\sim\mathcal{S}}\left[\frac{1}{|y|}\sum_{t=1}^{|y|}D\left(p_{T}(\cdot\mid y_{<t},x)\,\|\,p_{S}(\cdot\mid y_{<t},x)\right)\right]. (1) This supervised distillation objective provides dense token-level supervision, but at inference time the model conditions on its own prefixes rather than those observed in \mathcal{S} , which creates a state-distribution mismatch and can cause errors to compound over long horizons [ 37 ] . On-policy distillation (OPD) mitigates this mismatch by sampling y\sim p_{S}(\cdot\mid x) from the current student and querying the teacher on the prefixes generated by the student [ 1 , 26 , 57 ] . The objective is \mathcal{L}_{\mathrm{OPD}}(\theta)=\mathbb{E}_{x\sim\mathcal{S},\,\hat{y}\sim p_{S}(\cdot\mid x)}\left[\frac{1}{|\hat{y}|}\sum_{t=1}^{|\hat{y}|}D\left(p_{T}(\cdot\mid\hat{y}_{<t},x)\,\|\,p_{S}(\cdot\mid\hat{y}_{<t},x)\right)\right]. (2) OPD combines the on-policy relevance of reinforcement learning with the dense token-level guidance of distillation, making it a natural fit for autoregressive generation and reasoning. Figure 2: A case of the regional-to-global gap, based on Qwen3.5-9B. The global image input leads to the wrong answer, while the cropped region input yields the correct answer. Figure 3: The accuracy of regional inputs is consistently higher than the accuracy of global inputs, revealing a persistent regional-to-global gap across multiple MLLMs. 3 Vision-OPD 3.1 Motivation: less is more for fine-grained visual understanding MLLMs struggle with fine-grained visual understanding where small but decisive evidence is overwhelmed by global context. Recent “Thinking-with-Images” methods [ 77 , 70 , 5 ] demonstrate that zooming into the region of interest improves fine-grained understanding, indicating that the

Read the complete text