Med-Scout: Curing MLLMs’ Geometric Blindness in Medical Perception via Geometry-Aware RL Post-Training

Anglin Liu, Ruichao Chen, Yi Lu, Hongxia Xu, Jintai Chen † · 2026

Read Med-Scout: Curing MLLMs’ Geometric Blindness in Medical Perception via Geometry-Aware RL Post-Training by Anglin Liu, Ruichao Chen, Yi Lu, Hongxia Xu, Jintai Chen † (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-23220.txt.

Introduction

Anglin Liu and four co-authors cure a perceptual flaw in medical multimodal models. These models speak fluently about diagnoses, yet the authors find even top systems suffer spatial blindness. They fail to ground answers in objective spatial constraints, producing plausible but wrong claims.

Their Med-Scout post-training uses geometry-aware reinforcement learning to fix the grounding. Training rewarded on spatial fidelity, not just fluent text, should cut visual hallucinations. In medicine, a confident wrong measurement can misdirect real treatment. Geometric fidelity, not linguistic fluency, is the training target, reversing the priorities that produced the blindness.

Catalog record

CREATORAnglin Liu, Ruichao Chen, Yi Lu, Hongxia Xu, Jintai Chen †
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE73 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-23220

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Med-Scout: Curing MLLMs’ Geometric Blindness in Medical Perception via Geometry-Aware RL Post-Training Anglin Liu, Ruichao Chen, Yi Lu, Hongxia Xu, Jintai Chen † Abstract Despite recent Multimodal Large Language Models (MLLMs)’ linguistic prowess in medical diagnosis, we find even state-of-the-art MLLMs suffer from a critical perceptual deficit: geometric blindness . This failure to ground outputs in objective geometric constraints leads to plausible yet factually incorrect hallucinations, rooted in training paradigms that prioritize linguistic fluency over geometric fidelity. This paper introduces Med-Scout, a novel framework that “cures” this blindness via Reinforcement Learning (RL) that leverages the intrinsic geometric logic latent within unlabeled medical images. Instead of relying on costly expert annotations, Med-Scout derives verifiable supervision signals through three strategic proxy tasks: Hierarchical Scale Localization, Topological Jigsaw Reconstruction, and Anomaly Consistency Detection. To rigorously quantify this deficit, we present Med-Scout-Bench, a new benchmark specifically designed to evaluate geometric perception. Extensive evaluations show that Med-Scout significantly mitigates geometric blindness, outperforming leading proprietary and open-source MLLMs by over 40% on our benchmark. Furthermore, this enhanced geometric perception generalizes to broader medical understanding, achieving superior results on radiological and comprehensive medical VQA tasks. The project page is: https://c0216rc.github.io/Med-Scout/ . 1 Introduction The deployment of Multimodal Large Language Models (MLLMs) in medical imaging necessitates a fundamental shift in priorities compared to general-domain vision-language tasks ( 34 ) . While general-purpose models often prioritize open-ended creativity and linguistic fluency, medical visual perception demands an uncompromising adherence to objective geometric reality. A clinical AI system must not only interpret high-level semantic features but also rigorously respect the intrinsic geometric constraints of the image, such as relative anatomical scales, precise topology, and pixel-level structural consistency. However, despite the rapid evolution of both general ( 19 ; 4 ; 3 ) and clinically adapted MLLMs ( 8 ; 26 ; 23 ) , a critical capability gap remains: while these models excel at generating semantically rich descriptions, they often suffer from geometric blindness , failing to ground their outputs in the strict geometric facts. This misalignment stems largely from the limitations of prevailing training paradigms. Current approaches, including Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) ( 20 ) , predominantly optimize for semantic alignment, maximizing the likelihood of generating clinically plausible text conditioned on visual encodings. While effective for linguistic fluency, these objectives lack explicit mechanisms to penalize violations of physical laws. Consequently, models often generate professionally phrased reports that contradict the actual visual evidence, such as hallucinating lesions or misplacing organs. They essentially master complex medical terminology while failing to perceive the geometric reality of the scan accurately. Given the need for such strict geometric verification, RL methods ( 22 ) , specifically Group Relative Policy Optimization (GRPO) ( 24 ) presents an ideal optimization framework. Unlike standard supervision that relies on likelihood maximization, GRPO enables the model to learn directly from objective feedback signals. This mechanism is particularly effective for instilling geometric awareness, as it allows us to define and enforce explicit constraints that the model must satisfy. Building upon this perspective, we introduce Med-Scout , a geometry-aware post-training framework designed to actively cure MLLMs’ geometric blindness in medical perception. Adopting a data-centric approach, we transform unlabeled medical images into objective supervision signals by decomposing medical perception into three intrinsic geometric proxy tasks: (1) Hierarchical Scale Localization , enforcing awareness of anatomical details across varying scales; (2) Topological Jigsaw Reconstruction , demanding the understanding of global anatomical layouts; and (3) Anomaly Consistency Detection , necessitating comparative scrutiny to identify pixel-level structural artifacts. To drive the optimization of these tasks, we present a specialized Dense Geometric Reward (DGR) approach integrated within the GRPO framework, which provides dense, continuous guidance, effectively steering the geometric-semantic alignment to ensure stable and balanced convergence across the constructed task types. To support rigorous training and evaluation, we construct a comprehensive dataset comprising over 100K geometrically perturbed samples and curate a balanced 10% subset to establish Med-Scout-Bench, a novel benchmark that quantifies geometric blindness. We apply Med-Scout to promote representative general-domain and medical-domain MLLMs. Experiments demonstrate that our approach not only significantly reduces geometric blindness on Med-Scout-Bench but also enables strong generalization to standard medical VQA ( 15 ; 18 ; 39 ; 13 ; 41 ; 5 ) and report generation tasks ( 11 ; 14 ) . This confirms that the geometric capabilities acquired through our post-training effectively transfer to broader medical perception. Our main contributions are summarized as follows: • Unveiling Geometric Blindness. We conduct a pilot study and identify a critical gap where MLLMs fail to ground outputs in geometric constraints despite semantic fluency. • Geometry-Aware RL Post-Training. We propose Med-Scout, a novel RL framework that leverages specialized geometric proxy tasks to actively cure geometric blindness in MLLMs using dense geometric rewards. • The Med-Scout-Bench. We release Med-Scout-Bench, a novel benchmark constructed from the intrinsic geometric properties of medical images. It serves as a rigorous standard for quantifying geometric perception, addressing a key evaluation gap. • Substantial Performance Improvements. Med-Scout improves geometric perception by over 40% on our benchmark and achieves state-of-the-art generalization on radiological and comprehensive medical VQA benchmarks. 2 Related Work MLLMs for the Medical Domain. The advancement of medical visual understanding is currently driven by two complementary paradigms. On one front, domain-specific adaptation has achieved remarkable precision through biomedical instruction tuning, as demonstrated by a growing array of specialized models ( 16 ; 37 ; 8 ; 26 ; 21 ) . Simultaneously, general-purpose foundation models ( 19 ; 9 ; 28 ; 3 ) have exhibited surprising zero-shot adaptability to radiological tasks, benefiting from massive-scale pre-training ( 38 ) and robust visual representations. However, despite these distinct strengths, both paradigms share a critical vulnerability: they frequently prioritize linguistic fluency over physical grounding. This results in pervasive “geometric blindness,” where models successfully describe pathologies but fail to adhere to the strict spatial constraints and anatomical layouts inherent in medical images. Enhance MLLM with Proxy Tasks. Recent advancements have integrated verifiable proxy tasks into RL to enhance visual grounding. Jigsaw-R1 ( 30 ) and Visual Jigsaw ( 33 ) established that grid reconstruction significantly improves fine-grained perception, and formalize such objective tasks into RL pipelines. ViCrit ( 29 ) employs executable programs for robust verification. Approaches like Euclid ( 36 ) , GeoPQA ( 7 ) , and GeoGPT4V ( 6 ) attempt to augment MLLMs with geometric priors, but their task formulations remain incompatible with the unique requirements of medical perception. All these methods fail to address comprehensive clinical constraints like multi-scale consistency and anomaly focus. Moreover, their reliance on sparse binary feedback lacks the granularity essential for complex medical reasoning. 3 Pilot Study We conducted three preliminary experiments on Qwen3-VL-8B-Instruct ( 3 ) and Lingshu-7B ( 26 ) to investigate the geometric fidelity of MLLMs, as shown in Figure 1 . Our three findings are summarized as follows: Figure 1 : Pilot Study: (Left) Scale Blindness: The model correctly describes findings in a local crop but fails in the full global view. (Middle) Topology Blindness: When the image is rotated, the model fails to update location descriptions. (Right) Anomaly Blindness: The model completely overlooks obvious artificial modifications. 3.1 Finding I: Inconsistency Between Different Scales We investigated whether

Read the complete text