Transition-Aware best-of- N N sampling for Longitudinal Chest X-ray Reports

H. Ibrahim Gulluk ,, Max Van Puyvelde, Wim Van Criekinge, Olivier Gevaert · 2026

Read Transition-Aware best-of- N N sampling for Longitudinal Chest X-ray Reports by H. Ibrahim Gulluk ,, Max Van Puyvelde, Wim Van Criekinge, Olivier Gevaert (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-28393.txt.

Introduction

H. Ibrahim Gulluk, Max Van Puyvelde, Wim Van Criekinge, and Olivier Gevaert improve how artificial intelligence picks the best chest X-ray report among several candidates. Radiologists read each new scan against the patient's prior exam, and much of a report describes change. Existing report generators, however, score candidates without any sense of that before-and-after transition.

Gulluk and colleagues embed each report's sentences as a set, compress the prior-to-current shift into one directional vector, and score candidates by closeness to real transition vectors banked from training data. They try four set-distance designs, from mean shift to optimal transport. Across three prompts and three vision-language generators, transition-aware selection beat random choice everywhere, with the largest relative gains in the Impression section, where change matters most.

Catalog record

CREATORH. Ibrahim Gulluk ,, Max Van Puyvelde, Wim Van Criekinge, Olivier Gevaert
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.28393

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Transition-Aware best-of- N N sampling for Longitudinal Chest X-ray Reports H. Ibrahim Gulluk ,, Max Van Puyvelde, Wim Van Criekinge, Olivier Gevaert Abstract In longitudinal clinical practice, every chest X-ray is read in the context of the patient’s prior exam, and much of what the radiologist communicates is the change from one visit to the next. To the best of our knowledge, we present the first training-free best-of- N N sampling scheme for pre-trained chest X-ray report generators that is explicitly aware of this longitudinal prior → \rightarrow current transition. We call it transition-aware best-of- N N sampling : each report is split into sentences and embedded into an unordered set in ℝ d \mathbb{R}^{d} ; each (prior, current) pair is reduced to a fixed-dim directional vector via a set-to-set distance designed to encode the change between the two sets; and candidates are scored by cosine distance from their candidate transition vector to a cached bank of ground-truth training transition vectors, aggregated as min or kNN . We instantiate the framework with four directional set distances (mean-shift, novelty residual, directed-Hausdorff anchor, and cost-weighted optimal transport) and evaluate on a multi-visit AP/PA cohort, running inference under three prompts on three vision–language generators. Transition-aware best-of- N N outperforms random selection across the board, with the largest relative gains on the Impression section. 1 Introduction Chest radiography is the most frequently performed imaging modality in medicine, and automating chest X-ray report generation can substantially ease radiologist workload while improving reporting consistency. Modern vision–language models (VLMs) can already draft credible reports [ 13 , 10 , 7 , 2 , 4 ] , and a simple training-free way to extract further quality from such generators is best-of- N sampling : drawing N candidate reports per image and selecting the one that is the best in terms if the preferences or the performance. The vast majority of recent chest X-ray report generators – and the corresponding best-of- N pipelines built on top of them – treat each test image in isolation, scoring candidates against a distribution of single-image reference reports [ 8 , 2 , 4 ] . In real clinical practice, however, chest X-rays are rarely read in isolation: most exams are acquired as follow-ups, and the radiologist’s report is largely organised around how the patient has changed since the prior study (“stable interval”, “new opacity”, “improved aeration”). Modelling this longitudinal context has been shown to improve generation quality when fed to the decoder [ 3 , 14 ] , but to our knowledge the same idea has not yet been exploited at the selection stage of a best-of- N pipeline. We argue that for a longitudinal image the appropriate notion of best response should be selected based on the transitions from the prior visit to the current visit. As some transitions might not be clinically valid although those can be generated by a language model. We turn this intuition into a concrete pipeline. Each report is split into sentences and embedded into an unordered set in \mathbb{R}^{d} by a frozen sentence transformer (Sec. 3.1 , Fig. 1 ). A directional set-to-set distance then reduces each (prior, current) pair to a single fixed-dim vector (Sec. 3.2 , Fig. 2 (a)). A bank of such ground-truth training transition vectors is cached once. At test time, each candidate report becomes its own transition vector and is cosine-scored against the bank (Fig. 2 (b)). Crucially, an individually plausible-looking candidate report can nonetheless describe a clinically implausible change from the patient’s prior study – e.g. silently dropping a chronic finding or hallucinating an acute worsening that the image does not support. Scoring in transition space penalises exactly these candidates, because their displacement from the prior departs from the displacements observed in training; report-space scoring, by contrast, only checks that the candidate looks like some real report. The transition bank therefore provides a prior on how reports change between visits , not just on what individual reports look like. We instantiate this idea with a small family of directional set distances, and benchmark it on a multi-visit cohort drawn from a publicly available chest X-ray dataset across several vision–language generators and prompting regimes; transition-aware best-of- N outperforms random selection on a wide range of text-overlap and clinical-content metrics. 2 Related work Early systems combined a CNN encoder with an LSTM/Transformer decoder [ 13 , 10 ] or retrieved canned sentences from a memory of training reports [ 7 ] . More recent work pairs domain-specific visual encoders with general-purpose LLMs: R2GenGPT [ 21 ] aligns a Swin-Transformer visual feature stream to a frozen LLaMA-2 via a learned projection; CheXagent [ 4 ] instruction-tunes a clinical LLM jointly with a CXR vision encoder on a curated mixture of 28 CXR datasets; MAIRA-2 [ 2 ] couples the RAD-DINO image encoder with Vicuna-7B and a per-finding grounding head to produce spatially grounded reports. Several recent works exploit the patient’s prior study at generation time. BioViL-T [ 3 ] self-supervises a CNN+ViT encoder that explicitly fuses current and prior CXR pairs and is used for temporal classification, phrase grounding and longitudinal report generation. The authors [ 14 ] condition the report decoder on the prior study (handling its absence with a learned placeholder) and add a CXR-BERT semantic-similarity reward during fine-tuning. Both target the decoder; in contrast we operate downstream at the best-of- N selection stage and require no changes to the generator. Representing a report as an unordered set of sentence embeddings is a natural fit for chest X-ray reading: standard set-to-set distances – Chamfer, Hausdorff, optimal transport [ 20 ] , Hungarian matching [ 9 ] – yield continuous, permutation-invariant scores between two reports. The closest existing recipe to ours is SDR [ 8 ] , which uses such sentence-set distances both as a GRPO reward and as the scorer for single-image best-of- N selection; it remains image-independent and does not encode change between visits. Best-of- N sampling with a learned verifier is a standard test-time scaling tool in the LLM literature [ 5 , 11 ] : N candidates are drawn from a stochastic generator and the highest-scoring one is kept. Our pipeline has the same shape but the scorer operates in transition space rather than at the candidate level. 3 Method 3.1 Sentence-set report representation Findings Stable position of right IJ tunneled hemodialysis catheter. Catheter tip projects over the distal SVC. Cardiac and mediastinal contours are within normal limits. The lungs are clear. No acute osseous abnormality. Impression Right IJ tunneled hemodialysis catheter with the tip overlying the distal SVC. No evidence of acute cardiopulmonary process. Chest X-ray Sentence Transformer FROZEN all-mpnet-base-v2 \mathbf{e}^{F}_{1} \mathbf{e}^{F}_{2} \mathbf{e}^{F}_{3} \mathbf{e}^{F}_{4} \mathbf{e}^{F}_{5} \mathbf{e}^{I}_{1} \mathbf{e}^{I}_{2} \mathcal{E}(r) \subset\mathbb{R}^{d} Figure 1 : Sentence-level encoding of a chest X-ray report. Each visit pairs a radiograph with a free-text report composed of a Findings and an Impression section. We split both sections into individual sentences and embed each sentence independently with the frozen pre-trained all-mpnet-base-v2 sentence transformer, producing one d -dimensional vector per sentence. The resulting unordered collection of sentence embeddings \mathcal{E}(r)=\{\mathbf{e}^{F}_{1},\dots,\mathbf{e}^{F}_{5},\mathbf{e}^{I}_{1},\mathbf{e}^{I}_{2}\}\subset\mathbb{R}^{d} serves as the report representation throughout Sec. 3 . We adopt the notation of the SDR formulation [ 8 ] : a chest X-ray report r

Read the complete text