X-WIN: Building Chest Radiograph World Model via Predictive Sensing
Zefan Yang Ge Wang James Hendler Mannudeep K. Kalra Pingkun Yan · 2025
Read X-WIN: Building Chest Radiograph World Model via Predictive Sensing by Zefan Yang Ge Wang James Hendler Mannudeep K. Kalra Pingkun Yan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-14918.txt.
Introduction
Zefan Yang, Ge Wang, and three co-authors build X-WIN, a chest X-ray model that learns three-dimensional anatomy from a flat image. Chest radiographs are cheap and common, but a two-dimensional projection stacks structures on top of each other, which hides depth cues. The team teaches the model using chest CT scans, which do carry full volume information.
The model learns to predict X-ray views of the same chest from new angles, inside a learned latent space. A guided contrastive loss ties together views that belong to the same patient, so shared anatomy is captured. A model that has internalized body structure this way should read ordinary X-rays more reliably for diagnosis.
Catalog record
| CREATOR | Zefan Yang Ge Wang James Hendler Mannudeep K. Kalra Pingkun Yan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-14918
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
X-WIN: Building Chest Radiograph World Model via Predictive Sensing Zefan Yang Ge Wang James Hendler Mannudeep K. Kalra Pingkun Yan Abstract Chest X-ray radiography (CXR) is an essential medical imaging technique for disease diagnosis. However, as 2D projectional images, CXRs are limited by structural superposition and hence fail to capture 3D anatomies. This limitation makes representation learning and disease diagnosis challenging. To address this challenge, we propose a novel CXR world model named X-WIN, which distills volumetric knowledge from chest computed tomography (CT) by learning to predict its 2D projections in latent space. The core idea is that a world model with internalized knowledge of 3D anatomical structure can predict CXRs under various transformations in 3D space. During projection prediction, we introduce an affinity-guided contrastive alignment loss that leverages mutual similarities to capture rich, correlated information across projections from the same volume. To improve model adaptability, we incorporate real CXRs into training through masked image modeling and employ a domain classifier to encourage statistically similar representations for real and simulated CXRs. Comprehensive experiments show that X-WIN outperforms existing foundation models on diverse downstream tasks using linear probing and few-shot fine-tuning. X-WIN also demonstrates the ability to render 2D projections for reconstructing a 3D CT volume. 1 Introduction Chest X-ray radiography (CXR) is the most widely-used imaging technique for thoracic disease detection 39 . However, as 2D projectional images, CXRs are limited by structural superposition, and hence fail to capture 3D anatomies of various organs. In comparison, chest computed tomography (CT) is a medical imaging technique which similarly measures the X-ray attenuation by different tissues, but can obtain detailed 3D internal structure via tomographic reconstruction 41 . However, CT scanning is nearly five times the cost of CXR 14 . CT also delivers notably more radiation to patients compared to CXR, which can potentially cause adverse health effects. Moreover, CT accessibility is considerably more restricted compared to CXR, especially in underdeveloped regions 4 , due to high installation and maintenance costs. The tradeoffs between rich spatial information on CT and safety, cost-effectiveness, and accessibility of CXR motivate us to devise a deep learning method that can learn spatial knowledge from CT to attain enhanced disease diagnosis on CXRs. Figure 1 : Illustration of our motivation. (a) Given routine frontal and lateral CXRs, radiologists can reconstruct a 3D model of the chest volume based on their anatomical knowledge. This capability helps them make diagnostic decisions despite the occlusion of structures. (b) Inspired by our observation in (a), we propose a novel chest X-ray world intelligence network (X-WIN) that builds a world model by predicting new projection views under various transformations of an X-ray source. Recently, world modeling 19 ; 7 ; 3 ; 6 ; 10 has emerged as a technique to embed an internal understanding of the world into an agent model, which have learned meaningful visual representations capturing spatial dynamics of the environment. Such representations can improve scene understanding, navigation, and action planning. In the field of medical imaging, Yang et al. recently introduced CheXWorld 52 , a world model for CXR interpretation, which devises action-conditioned self-supervised objectives to learn visual representations of radiographs. However, CheXWorld remains limited to 2D and does not embed any 3D knowledge into the world model. Such an absence of 3D information leads to a substantial capability gap between the current CXR world model and radiologists. Based on our observation illustrated in Fig. 1 , radiologists can cognitively reconstruct an internal 3D model of the chest region when perceiving frontal and lateral CXRs. To bridge this capability gap, we believe that the CXR world model should capture 3D spatial information and propose a novel method to distill volumetric knowledge from CT. Specifically, we introduce an innovative X-ray World Intelligence Network (X-WIN), which encodes spatial information from CT by learning to predict its projections in latent space. These projections originate from the same CT volume and characterize 3D structure from different views. More specifically, our proposed framework consists of two synergistic networks. The first network contains an encoder and a light-weight action-conditioned predictor to generate latent projection prediction. It receives the routine frontal or lateral X-ray projection as input. The second network is an exponential moving average encoder that receives multiple projections of a target CT volume to supervise latent projection prediction. The generation of new projections is controlled by an action to move the X-ray source. This framework matches our goal of distilling volumetric knowledge into a CXR model, as the exponential moving average encoder encodes knowledge from a CT volume and guides the learning of the encoder that processes routine CXRs. We deploy X-WIN on downstream CXR interpretation tasks when the training finishes. To train the proposed model, we introduce three major losses, including an affinity-guided contrastive alignment loss, a masked image modeling loss (MIM) 2 , and a structure-preserving domain adaptation loss. The affinity-guided contrastive alignment loss ensures that the proposed model encodes sufficient nuanced information during projection prediction. It is a softened contrastive alignment loss 33 based on mutual similarities to capture rich correlated information across multiple projections. The MIM loss is incorporated to enable the model to encode local and contextual features and enhance its adaptability to downstream CXR tasks. The MIM loss is applied to both real and simulated CXRs. Such a loss enforces the model to reconstruct local patches on CXRs, enabling it to learn fine-grained features of anatomical structures and disease abnormalities. The structure-preserving domain adaptation loss is incorporated to learn a cohesive representation space between the real and simulated domains. In this loss, simulated-domain representations are encouraged to resemble the real-domain representations under the guidance of a domain classifier, while preserving structural information via patch-level supervision. In summary, our proposed method, X-WIN, effectively distills knowledge from 3D CT via world modeling for an enhanced application on CXRs. Extensive experiments on cutting-edge CXR models and standard benchmarks demonstrate that the learned representations achieve substantial improvements over existing methods with comparable model capacities. Additionally, X-WIN exhibits an effective capability to render 2D projections for tomographic reconstruction, demonstrating a robust understanding of 3D structures. Our contributions are as follows: • We propose the X-WIN framework for radiograph world modeling, making the first approach to integrate 3D spatial knowledge into a CXR world model. • We introduce an affinity-guided contrastive alignment objective, which enhances the encoding of discriminative features and exploits rich correspondences in different projections. • The learned representations of our model have been applied on standard CXR interpretation tasks, achieving state-of-the-art performance with linear probing. Our method exhibits effective 3D reconstruction capability. Figure 2 : Overview of the X-WIN framework. Top: Our proposed approach distills spatial knowledge from CT by learning to predict its projections in latent space. We generate new projections by rotating a X-ray source controlled by actions. Bottom: Masked image modeling and a domain classifier are incorporated to learn a cohesive representation space between real and simulated domains. EMA denotes exponential moving average. 2 Related Work 2.1 World Models World models are conceptualized as models that possess implicit spatial cognition of the 3D world 29 . In the AI field, the recent Cambrian-S 49 constructs implicit 3D spatial cognition through predictive sensing , where models learn