Schema-Adaptive Tabular Representation Learning with LLMs for Generalizable Multimodal Clinical Reasoning
Hongxi Mao, Wei Zhou, Mengting Jia, Tao Fang, Huan Gao, Bin Zhang, Shangyang Li · 2026
Read Schema-Adaptive Tabular Representation Learning with LLMs for Generalizable Multimodal Clinical Reasoning by Hongxi Mao, Wei Zhou, Mengting Jia, Tao Fang, Huan Gao, Bin Zhang, Shangyang Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-11835.txt.
Introduction
Mao and six colleagues tackle tables of data whose layouts differ across sources, an acute problem in clinical medicine. They turn table columns into plain-language statements, then encode them with a pretrained language model. The resulting codes line up across unseen layouts without retraining or manual feature work. Combined with MRI data for dementia diagnosis, the method tops baselines on NACC and ADNI data and beats board-certified neurologists in look-back tests.
Catalog record
| CREATOR | Hongxi Mao, Wei Zhou, Mengting Jia, Tao Fang, Huan Gao, Bin Zhang, Shangyang Li |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30450 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.11835
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Schema-Adaptive Tabular Representation Learning with LLMs for Generalizable Multimodal Clinical Reasoning Hongxi Mao, Wei Zhou, Mengting Jia, Tao Fang, Huan Gao, Bin Zhang, Shangyang Li Abstract Machine learning for tabular data remains constrained by poor schema generalization, a challenge rooted in the lack of semantic understanding of structured variables. This challenge is particularly acute in domains like clinical medicine, where electronic health record (EHR) schemas vary significantly. To solve this problem, we propose Schema-Adaptive Tabular Representation Learning, a novel method that leverages large language models (LLMs) to create transferable tabular embeddings. By transforming structured variables into semantic natural language statements and encoding them with a pretrained LLM, our approach enables zero-shot alignment across unseen schemas without manual feature engineering or retraining. We integrate our encoder into a multimodal framework for dementia diagnosis, combining tabular and MRI data. Experiments on NACC and ADNI datasets demonstrate state-of-the-art performance and successful zero-shot transfer to unseen schemas, significantly outperforming clinical baselines, including board-certified neurologists, in retrospective diagnostic tasks. These results validate our LLM-driven approach as a scalable, robust solution for heterogeneous real-world data, offering a pathway to extend LLM-based reasoning to structured domains. † † † Equal contribution. † † ∗ Corresponding author. 1 Introduction Machine learning excels in structured data modeling but struggles with cross-domain generalization, particularly in tabular domains where varying schemas cause models to fail across datasets 35 ; 30 ; 27 ; 25 ; 47 . This limits AI scalability and reliability in fields like healthcare with diverse electronic health records. At the root of this generalization crisis lies schema heterogeneity . Real-world data rarely share consistent column names, coding systems, or data formats. For instance, a key biomarker may appear as a continuous value in one database but as a categorical code in another 14 . Conventional machine learning models are trained on fixed, syntactic representations and lack the ability to reconcile these schema variations, resulting in brittle, non-transferable embeddings 18 ; 41 ; 6 ; 37 ; 33 ; 21 . Manual feature harmonization offers a partial remedy but is inherently non-scalable, error-prone, and unsustainable in complex real-world pipelines. To overcome these limitations, we advocate a new approach from schema-dependent learning to semantic schema understanding . This work introduces Schema-Adaptive Tabular Representation Learning , a framework that leverages the powerful semantic reasoning of Large Language Models (LLMs) to align heterogeneous structured data through natural language. Instead of treating column-value pairs as numeric tokens, our approach converts them into semantic text, capturing variable metadata and context. Encoded by a pretrained LLM, these texts produce schema-agnostic embeddings, enabling zero-shot transfer across new datasets without manual alignment or retraining 24 ; 10 ; 17 ; 19 ; 46 . We situate this methodological contribution within a challenging, multimodal setting: differential dementia diagnosis. This task serves as a rigorous stress test for our framework, requiring the integration of semantically encoded tabular features with neuroimaging data and the prediction of co-occurring etiologies (e.g., Alzheimer’s and vascular pathologies). By embedding our schema-adaptive encoder into a cross-modal transformer with a multi-objective contrastive learning scheme, we test its robustness under real-world heterogeneity, label imbalance, and limited data availability 15 ; 36 ; 42 . Importantly, the medical domain here is not our end goal but a high-stakes validation ground for schema-level generalization—a core capability essential to scalable, language-driven machine learning. Our contributions are summarized as follows: 1. We propose a schema-adaptive representation framework that reformulates tabular data as semantically compositional text, allowing pretrained LLMs to achieve zero-shot schema alignment without explicit feature harmonization or fine-tuning. 2. We integrate this encoder into a multimodal architecture to evaluate its robustness under extreme heterogeneity and limited supervision, achieving state-of-the-art performance and outperforming human experts in retrospective diagnostic settings. 3. Through comprehensive analysis, we demonstrate how LLM-based semantic encoding enhances multimodal learning, improves sample efficiency, and yields interpretable decision patterns grounded in domain knowledge. 2 Related Work 2.1 Tabular Representation Learning Classical tabular models like Gradient Boosted Decision Trees are purely syntactic and fail to generalize across schemas 3 ; 28 . Recent deep learning approaches, including TabPFN 12 ,TIP 5 and TransTab 39 , have begun to incorporate semantic information but remain largely schema-dependent, limiting their robustness under significant schema shifts. The advent of LLMs has catalyzed a new direction. Seminal works in NLP like TaPas 11 and other table pretraining methods 38 have focused on deep, intra-table reasoning for tasks like question answering. Models such as TableGPT 31 ; 34 and TableDreamer 45 extend this to generative tasks, primarily on clean, single-source benchmarks. However, the critical challenge of robust generalization across noisy, heterogeneous real-world schemas remains largely unaddressed. Our work is positioned to fill this specific gap. We focus on cross-schema generalization, proposing a simple yet robust method that leverages a pretrained LLM as a semantic encoder. By transforming column-value pairs into descriptive statements, our framework achieves semantic alignment across diverse, unseen table structures, a key capability underexplored by prior models focused on single-schema understanding. Figure 1: Overview of our proposed architecture. Patient records comprising tabular and imaging data are processed via modality-specific encoders. The LLM-based table encoder handles schema heterogeneity, while a frozen Swin UNETR encodes images. These representations are fused in a gated transformer with label-specific [CLS] tokens. A composite loss, optimized via MGDA, enables robust multi-label learning under class imbalance. 2.2 Multimodal Learning with Structured Inputs Multimodal learning seeks to unify representations across diverse data types, such as images, text, and structured records, to enable holistic reasoning. Early approaches relied on late-fusion strategies, combining predictions from modality-specific encoders 40 , but these often failed to capture cross-modal dependencies during joint representation learning. Recent methods, inspired by CLIP 29 , use contrastive objectives for modality alignment 8 ; 13 ; 5 , but image-tabular and text-tabular integration lags due to structured data’s lack of semantic continuity. Our framework reinterprets tabular data as linguistic tokens, enabling integration into language-aligned multimodal architectures via cross-modal attention, enhancing performance through language-driven schema representation while ensuring comparability with prior work. 2.3 Multi-Label Learning and Optimization under Imbalance Multi-label classification, prevalent in clinical and biomedical tasks, faces challenges from class imbalance and correlated outputs. Standard supervised contrastive learning (SCL) 16 performs well for single-label tasks but struggles with co-occurring or hierarchical labels. Extensions such as MulSupCon 44 , lREG 1 , and JSCL 22 incorporate label-aware reweighting to address these issues, yet their performance falters on highly imbalanced datasets. Prototype-based methods like C-GMVAE 2 and Proto 7 model label distributions explicitly but are often confounded by high inter-label correlations. Our framework uses multi-label classification to evaluate schema generalization, employing a composite objective with focal loss, contrastive regularization, and multi-objective optimization, ensuring performance gains via an LLM-driven, schema-adaptive encoder for robust structured-data learning. 3 Methodology We present a generalizable framework for learning schema-invariant representations from heterogeneous structured data by leveraging the semantic priors of large language models (LLMs). The framework rests on three key components: (1) a schema-adaptive tabular encoder that performs language-based semantic tokenization of structured inputs; (2) a modality-specific encoder for auxiliary data (e.g., neuroimaging) to assess cross-modal robustness; and (3) a unified transformer-based fusion backbone with a multi-objective optimization scheme for complex multi-label prediction. Figure 1 illustrates the overall architecture.