Regulating Anatomy-Aware Rewards via Trajectory-Integral Feedback for Volumetric Computed Tomography Analysis
Tianwei Lin, Zhongwei Qiu, Jie Cao, Jiang Liu, Wenjie Yan, Bo Zhang, Yu Zhong, Wenqiao Zhang, Yingda Xia, Ling Zhang · 2026
Read Regulating Anatomy-Aware Rewards via Trajectory-Integral Feedback for Volumetric Computed Tomography Analysis by Tianwei Lin, Zhongwei Qiu, Jie Cao, Jiang Liu, Wenjie Yan, Bo Zhang, Yu Zhong, Wenqiao Zhang, Yingda Xia, Ling Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-20277.txt.
Introduction
Tianwei Lin and nine co-workers present the Clinical Abnormality Benchmarking Substrate, a system splitting reports into facts. Reinforcement learning relies on word proxy signals causing false scores, where models chase fluency over facts, causing errors. Using it, they find a split in learning, where surface rewards push gradients past facts. They propose Trajectory-Integral Feedback GRPO, adding control to optimization. Treating review as a time path, it steers anatomy rewards with an integral loop penalizing omissions and suppressing falsehoods. Tests on 3D CT show better anomaly detection and truth.
Catalog record
| CREATOR | Tianwei Lin, Zhongwei Qiu, Jie Cao, Jiang Liu, Wenjie Yan, Bo Zhang, Yu Zhong, Wenqiao Zhang, Yingda Xia, Ling Zhang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 76236 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.20277
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Regulating Anatomy-Aware Rewards via Trajectory-Integral Feedback for Volumetric Computed Tomography Analysis Tianwei Lin, Zhongwei Qiu, Jie Cao, Jiang Liu, Wenjie Yan, Bo Zhang, Yu Zhong, Wenqiao Zhang, Yingda Xia, Ling Zhang Abstract Medical vision-language models (VLMs) have rapidly advanced as general-purpose multimodal assistants, yet their deployment in 3D Computed Tomography (CT) analysis remains constrained by a persistent mismatch between optimization objectives and clinical rigor. Current Reinforcement Learning (RL) paradigms still rely on lexical proxy signals that induce “ Evaluation Hallucinations ”, where models optimize linguistic fluency rather than factual clinical correctness, leading to diagnostically critical errors. To bridge this gap, we introduce the Clinical Abnormality Benchmarking Substrate (CABS) , a structured system that decomposes radiology reports into verifiable clinical semantic units. Using CABS, we identify a “ Mechanistic Divergence ” in standard RL, where surface-similarity rewards drive policy gradients to bypass medical facts. We therefore propose Trajectory-Integral Feedback GRPO (TIF-GRPO) , a novel framework integrating control-theoretic principles into policy optimization. By formulating clinical reasoning as a pseudo-temporal trajectory for anomaly discovery, TIF-GRPO regulates anatomy-aware rewards via an integral feedback loop that penalizes persistent omissions as cumulative state errors and suppresses hallucinations as excessive control effort. Experiments on 3D CT benchmarks demonstrate that our approach significantly enhances abnormality detection and clinical faithfulness, establishing a new paradigm for fine-grained regulation in medical VLMs. Our project is available at GitHub . † † affiliationnotice: Equal contribution † Corresponding authors. Figure 1 : Overview of the “ Evaluation Hallucinations ” and “ Mechanistic Divergence ”. (a) Surface-similarity proxy signals induce evaluation hallucinations, where high-scoring predictions mismatch GT clinical facts. (b) Our CABS framework enables accurate abnormality-level measurement, and TIF-GRPO applies trajectory-integral control based on CABS to suppress hallucinations and align optimization with clinical fidelity. 1 Introduction While the 2D slice-level Computed Tomography (CT) offers a foundational view, 3D volumetric CT analysis provides superior clinical utility by transcending the inherent limitations of a fragmented planar view ( 43 ; 55 ; 25 ; 35 ) . By preserving voxel-level spatial topology, 3D CT analysis enables high-fidelity reconstruction of lesion-tissue associations—a prerequisite for precise surgical and diagnostic interventions. To harness this complex spatial data, recent 3D medical vision-language models (VLMs) have emerged, showing transformative potential as AI radiological imaging expert for 3D CT analysis ( 15 ; 21 ; 3 ; 34 ) . However, existing research largely inherits the general-purpose training paradigm for medical domain adaptation. This direct transfer often overlooks a critical clinical boundary: unlike general domains, Factuality and Faithfulness are the non-negotiable baselines of medical analysis. In clinical reporting, diagnostically decisive findings are often sparsely embedded within otherwise routine and semantically homogeneous descriptions; as a result, objectives driven primarily by distributional fitting may overemphasize frequent stylistic regularities while underweighting the small set of clinically critical signals, so that even subtle hallucinations can introduce disproportionate diagnostic risk and undermine diagnostic integrity. This oversight leads to a dual crisis in the current medical VLM paradigm: (i) evaluation hallucinations , where metrics are misaligned with clinical capability, and (ii) the misleading nature of RL rewards built upon these hallucinatory proxy signals. Existing evaluations rely heavily on lexical overlap (e.g., BLEU ( 41 ) , ROUGE ( 33 ) , METEOR ( 4 ) ) or simplistic scoring by coarse-grained medical semantic similarity (e.g., RadGraph ( 19 ) , RaTEScore ( 68 ) , BioBert ( 26 ) ). These proxy signals establish an evaluation essentially detached from high-stakes medical diagnosis, manifesting as a systematic divergence between evaluation scores and clinical competence. As shown in Fig 1 (a), samples scoring high on these metrics often commit fatal clinical errors, such as misinterpreting pathological attributes or mismatching anatomical laterality. More critically, as shown in Fig 1 (b), when RL algorithms ( 40 ; 24 ; 11 ; 10 ) utilize rewards based on superficial accuracy, models fall into Reward Hacking, mimicking the style of ground-truth answers at the expense of true image understanding. This indicates that without a clinically rigorous reward, policy optimization may actively drive models away from medical facts. To correct this goal misalignment, we advocate a paradigm shift in evaluation: from treating medical VLM outputs as unstructured text to interpreting them as compositions of discrete, verifiable clinical facts. Consequently, we propose the Clinical Abnormality Benchmarking Substrate (CABS) , a structured representation framework that decomposes radiology reports into atomic abnormality units grounded in clinical ontology. CABS explicitly encodes each abnormality as a tuple of organ, pathological entity, anatomical location, attributes, diagnostic certainty, and supporting textual evidence, thereby establishing a precise and auditable foundation for clinical factuality. This decomposition enables comprehensive evaluation of a model’s ability to detect and describe abnormalities through organ- and finding-level precision, recall, and F1 scores, providing a faithful measure of true clinical competency. While prior efforts such as RadGraph ( 19 ) and RaTEScore ( 68 ) also structure free-text reports, they ultimately rely on semantic similarity between extracted fragments, which remains a proxy disconnected from ground-truth pathology. Multiple radiologists validated CABS with 98.6% approval across key dimensions, confirming its reliability as a unified substrate for both evaluation and training. Utilizing CABS, we systematically deconstruct the alignment behaviors induced by heterogeneous reward signals in existing medical RL methods. We identify a phenomenon of Mechanistic Divergence : surface-similarity rewards often fail to reliably discriminate between clinically accurate and inaccurate outputs within a response group, leading optimization dynamics to diverge from true clinical fidelity. Furthermore, when driven by sparse clinical feedback, standard RL algorithms face severe instability, often collapsing into safe modes that ignore long-tail abnormalities to maximize average lexical scores. To address these challenges, we propose Trajectory-Integral Feedback GRPO (TIF-GRPO) , a novel RL framework that integrates control-theoretic principles into Group Relative Policy Optimization (GRPO) ( 46 ) . Unlike standard RL methods ( 40 ; 10 ) , which react only to instantaneous rewards, TIF-GRPO reformulates the clinical reasoning process as a pseudo-temporal trajectory . It regulates anatomy-aware rewards through an integral feedback loop that adaptively penalizes persistent diagnostic omissions (False Negatives) as cumulative state errors and suppresses speculative hallucinations (False Positives) as excessive control effort. Our approach grounds the model’s reasoning in the rigorous logic of clinical practice, stabilizing policy gradients and prioritizing diagnostic factuality. Our contributions can be summarized as follows: • Metric: We propose CABS , a system that decomposes medical VLM outputs into clinical abnormality units, unifying the evaluation standards for clinical detection and description. • Analysis: We provide a systematic analysis of existing medical similarity-based RL methods and characterize the Mechanistic Divergence phenomenon, demonstrating how proxy rewards drive policies away from clinical fact correctness. • Methodology: We introduce TIF-GRPO , a novel framework that leverages trajectory-integral control to stabilize policy gradients and provide fine-grained regulation of clinical anatomy-aware reward signals. • Results: Our TIF-GRPO achieves state-of-the-art (SOTA) performance across multiple 3D CT benchmarks, significantly improving clinical factuality and faithfulness. 2 Related Work Recent medical vision-language models (VLMs) have evolved from cross-modal alignment systems to increasingly unified and reasoning-capable architectures. Early efforts such as BioMedGPT ( 62 ) , LLaVA-Med ( 27 ) , and Med-Flamingo ( 38 ) established multimodal adaptation pipelines for medical understanding, while more recent models such as HealthGPT ( 35 ) , MedGemma