Retrieval-aligned Tabular Foundation Models Enable Robust Clinical Risk Prediction in Electronic Health Records Under Real-world Constraints

Minh-Khoi Pham, Thang-Long Nguyen Ho, Thao Thi Phuong Dao, Tai Tan Mai, Minh-Triet Tran, Marie E. Ward, Una Geary, Rob Brennan, Nick McDonald, Martin Crane, Marija Bezbradica · 2026

Read Retrieval-aligned Tabular Foundation Models Enable Robust Clinical Risk Prediction in Electronic Health Records Under Real-world Constraints by Minh-Khoi Pham, Thang-Long Nguyen Ho, Thao Thi Phuong Dao, Tai Tan Mai, Minh-Triet Tran, Marie E. Ward, Una Geary, Rob Brennan, Nick McDonald, Martin Crane, Marija Bezbradica (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-01841.txt.

Introduction

Pham and colleagues study clinical forecasts from digital health records, hit by dimensionality, variety and skew. Tabular in-context learning and search methods do well on benchmarks, but clinic results remain unclear. The authors benchmark older, deep, and in-context learning models across scale, rarity, and transfer. PFN-based in-context models prove strong with little data but degrade under distance-based search as variety grows. They propose AWARE, using guided vectors and tools, lifting AUPRC by up to 12.2 percent in extreme imbalance.

Catalog record

CREATORMinh-Khoi Pham, Thang-Long Nguyen Ho, Thao Thi Phuong Dao, Tai Tan Mai, Minh-Triet Tran, Marie E. Ward, Una Geary, Rob Brennan, Nick McDonald, Martin Crane, Marija Bezbradica
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE125694
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.01841

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Retrieval-aligned Tabular Foundation Models Enable Robust Clinical Risk Prediction in Electronic Health Records Under Real-world Constraints Minh-Khoi Pham, Thang-Long Nguyen Ho, Thao Thi Phuong Dao, Tai Tan Mai, Minh-Triet Tran, Marie E. Ward, Una Geary, Rob Brennan, Nick McDonald, Martin Crane, Marija Bezbradica Abstract Accurate clinical outcome prediction from structured electronic health records (EHRs) remains challenging due to high-dimensional heterogeneous features, severe outcome imbalance, and distribution shift—constraints that are far more pronounced than in general tabular benchmarks. Although recently developed tabular foundation models, particularly in-context and retrieval-augmented approaches, achieve strong results on generic datasets, their behavior under realistic clinical constraints remains unclear. In this work, we present the first systematic multi-cohort EHR benchmark comparing classical machine learning, deep tabular models, and tabular in-context learning (TICL) methods, including prior-fitted network (PFN)–based approaches and their retrieval-augmented variants (RA-TICL), across small clinical datasets and large real-world ICU cohorts. Through controlled stress tests varying data scale, feature dimensionality, outcome rarity, and cross-cohort generalization, we identify regime-dependent performance patterns. PFN-based TICL models are notably sample-efficient without retraining in low-data regimes. Meanwhile classic distance-based retrieval, which has been used to adapt PFN-based models to much larger datasets, leads to degradation as feature heterogeneity and class imbalance intensify in EHR domain. To address this limitation, we propose Attention Weighting for Aligned Retrieval Embeddings (AWARE), a task-aligned retrieval framework that improves neighborhood quality through supervised embedding retrieval learning and lightweight adapter fine-tuning. Across stress tests, AWARE yields relative AUPRC improvements that increase with data complexity, from approximately 0.3–4.0% as training size scales, to 7.0% under high feature dimensionality, and up to 12.2% under extreme outcome imbalance, demonstrating that retrieval alignment becomes increasingly critical in complex EHR settings. Collectively, our findings establish retrieval quality and retrieval–inference coupling as central bottlenecks for deploying tabular in-context learning in clinical prediction. 1 Introduction Figure 1 : Conceptual comparison of major learning paradigms for EHR foundation models, illustrating differences in representation, adaptation strategy, and computational trade-offs across native tabular models, sequential event-based models, language-based abstractions, and tabular in-context learning. ✓ highlights advantages while ✗ highlights disavantages. Accurate prediction from structured electronic health records (EHRs) is central to modern clinical decision support, enabling risk stratification, resource allocation, and timely intervention. However, EHR-based prediction remains methodologically challenging due to high-dimensional heterogeneous features, severe outcome imbalance, irregular temporal structure, and cross-institutional variation ( 77 ) . While recent years have seen rapid advances in foundation models for computer vision and natural language processing, tabular learning, and EHR modeling in particular, has also attracted growing interest, albeit at a more gradual pace and with comparatively fewer large-scale, systematic evaluations ( 86 ) . Most EHR foundation models have primarily relied on sequential or language-based representations that emphasize longitudinal structure and large-scale pretraining ( 91 ) . Although effective in selected settings, these paradigms often require extensive pretraining on real-world clinical data, operate over long token sequences, and impose complicated standardization of heterogeneous medical events into universal formats. Such design choices introduce (1) substantial training computational and architectural complexity, and (2) practical and governance-related constraints associated with real sensitive patient data, motivating interest in alternative paradigms that are more efficient. This motivates us to adopt a newly alternative approach that has been introduced recently in tabular deep learning, that can address these issues of EHRs. We highlight the main differences between these EHR-based learning paradigms in Figure 1 . Recent advances in tabular in-context learning (TICL), such as TabPFN ( 35 ) and its retrieval-augmented variants (RA-TICL), namely LocalPFN, TabDPT ( 83 ; 89 ) , offer a foundation-style paradigm for inference-time adaptation, relying only on synthetic data, and no task-specific retraining. For EHR-based clinical prediction, such approaches are particularly attractive because they can substantially reduce the need for repeated model development and retraining when new prediction tasks arise across cohorts or institutions ( 53 ) . This enables rapid adaptation to shifting patient populations, changing outcome definitions, or newly available features, while lowering training and operational costs associated with conventional supervised learning pipelines. However, strong performance on generic tabular benchmarks might not guarantee reliable results in clinical outcome prediction from structured EHR data ( 6 ) . In particular, it remains unclear how reliably TICL and retrieval-augmented TICL behave in high-dimensional, sparse, and imbalanced EHR representations where retrieval-based context construction maybe noisy or unstable ( 56 ) . In this work, we introduce an adaptive retrieval framework that learns task-aligned embeddings to improve neighborhood quality for RA-TICL. We evaluate this approach alongside classical machine learning baselines, deep tabular models, and prior-fitted network (PFN)–based TICL methods across diverse small-scale benchmarks and large real-world EHR cohorts. Our experiments isolate the effects of feature heterogeneity, class imbalance, dataset scale, and cross-cohort generalization, enabling a fine-grained analysis of when inference-time adaptation succeeds and fails, and how retrieval quality mediates this behavior. Figure 2 illustrates the diversity of the benchmark suite in terms of scale, dimensionality, rarity, and clinical domain. a b c d Figure 2 : Landscape of EHR benchmarks evaluated in this study. The datasets span wide variation in cohort size, feature dimensionality, outcome prevalence, and clinical domain, illustrating key challenges of (a) heterogeneity, (b) rarity, (c) cross-task, cross-institutional generalizability and (d) distribution shift faced by tabular learning methods in real-world EHR settings. See detailed descriptions of reported datasets in Table A2 . This work makes the following contributions to the literature : • A first systematic, multi-cohort benchmark of tabular foundation models for EHR-based clinical prediction . We evaluate classical machine learning baselines, deep tabular architectures, and TICL/RA-TICL methods across multiple real-world EHR cohorts and EHR-based supervised learning tasks , enabling controlled comparisons under shared preprocessing and evaluation protocols. • An adaptive retrieval framework (AWARE) for RA-TICL. We introduce a task-aligned embedding-based retrieval strategy and a parameter-efficient adapter finetuning strategy, that both improves neighborhood quality and alignment under high-dimensional, imbalanced EHR conditions, enabling more stable retrieval-augmented in-context learning for clinical tasks. • A stress test of inference-time adaptation under clinical constraints. We conduct a rigorous evaluation of these tabular foundation models under different clinical data constraints. We identify settings where TICL is most effective (e.g., low-data and moderately imbalanced regimes) and where performance degrades as feature heterogeneity, cohort scale, and outcome rarity increase. Taken together, our findings provide a clinically grounded evaluation of TICL and RA-TICL for EHR-based clinical prediction, discussing their strengths and limitations under realistic clinical data constraints. Based on these insights, we offer guidance for the reliable deployment of outcome prediction systems and outline directions for future method development. 2 Preliminaries and Problem Setup Our goal in this paper is to evaluate in-context and retrieval-based tabular learning under non-sequential EHR pipelines, which we call snapshot-based 1 1 1 or also called static EHR EHR prediction tasks. This differs from prior work ( 22 ; 58 ) focused on trajectory-dependent outcomes, such as length of stay, readmission, and long-horizon mortality, that require explicit sequence modeling. We focus on these tasks that operate on fixed-length, temporally-aggregated tabular representations. This setting reflects common deployment pipelines and isolates structural challenges such as feature heterogeneity, class imbalance, and context construction without introducing explicit sequence modeling. We formulate these tasks as supervised learning problems over tabular representations derived from structured EHR data.

Read the complete text