FeatEHR-LLM: Leveraging Large Language Models for Feature Engineering in Electronic Health Records

Hojjat Karami, David Atienza, Jean-Philippe Thiran, Anisoara Ionescu · 2026

Read FeatEHR-LLM: Leveraging Large Language Models for Feature Engineering in Electronic Health Records by Hojjat Karami, David Atienza, Jean-Philippe Thiran, Anisoara Ionescu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-22534.txt.

Introduction

Karami and three colleagues present FeatEHR-LLM, turning messy hospital records into tabular features with a large language model. Clinical time series are irregular and gappy, while older tools miss clinical meaning or assume tidy inputs. To protect privacy, the model sees only schemas and task descriptions, never raw records, while tool routines let it query irregular data and emit feature code for uneven timing and gaps. Across eight tasks on four intensive-care datasets, it tops the mean ranking on seven, gaining up to six percentage points over baselines.

Catalog record

CREATORHojjat Karami, David Atienza, Jean-Philippe Thiran, Anisoara Ionescu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34211
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.22534

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

FeatEHR-LLM: Leveraging Large Language Models for Feature Engineering in Electronic Health Records Hojjat Karami, David Atienza, Jean-Philippe Thiran, Anisoara Ionescu Abstract Feature engineering for Electronic Health Records (EHR) is complicated by irregular observation intervals, variable measurement frequencies, and structural sparsity inherent to clinical time series. Existing automated methods either lack clinical domain awareness or assume clean, regularly sampled inputs, limiting their applicability to real-world EHR data. We present FeatEHR-LLM , a framework that leverages Large Language Models (LLMs) to generate clinically meaningful tabular features from irregularly sampled EHR time series. To limit patient privacy exposure, the LLM operates exclusively on dataset schemas and task descriptions rather than raw patient records. A tool-augmented generation mechanism equips the LLM with specialized routines for querying irregular temporal data, enabling it to produce executable feature-extraction code that explicitly handles uneven observation patterns and informative sparsity. FeatEHR-LLM supports both univariate and multivariate feature generation through an iterative, validation-in-the-loop pipeline. Evaluated on eight clinical prediction tasks across four ICU datasets, our framework achieves the highest mean AUROC on 7 out of 8 tasks, with improvements of up to 6 percentage points over strong baselines. Code is available at github.com/hojjatkarami/FeatEHR-LLM . Keywords: Feature engineering; Electronic health records; Irregular time series; Large language models; Clinical prediction 1 Introduction Electronic Health Records (EHRs) contain rich longitudinal data that capture patient health trajectories. Within these records, time series measurements, such as vital signs and laboratory results, form the backbone of clinical information. However, modeling this data is challenging because clinical time series are inherently irregularly sampled, manifesting as variable observation frequencies and event-driven measurements. The resulting gaps represent structural sparsity, with measurements recorded only when clinically indicated, rather than statistical missingness [ 14 , 45 , 13 ] . Despite these complexities, machine learning (ML) is increasingly leveraged to analyze EHRs for critical clinical decision support [ 6 , 41 ] . To effectively utilize this complex temporal data, feature engineering remains fundamental. While modern deep learning (DL) models can process longitudinal sequences end-to-end, their widespread clinical deployment is often hindered by opacity, high data requirements, and heavy computational footprints [ 8 ] . Consequently, explicit feature engineering remains critical. It is strictly necessary for interpretable, efficient tree-based models favored in clinical settings [ 3 , 43 ] . Furthermore, explicitly engineered features can be injected into advanced DL architectures as static covariates, complementing learned representations to improve overall predictive performance and transparency [ 24 , 28 ] . Historically, explicit feature engineering relied on costly manual collaboration with healthcare professionals, or transformation-based methods [ 42 , 17 , 43 ] that automate generation but lack clinical context and fundamentally struggle with irregular observation intervals. Recently, Large Language Models (LLMs) have emerged to bridge this gap. By leveraging vast domain knowledge, LLMs inject zero-shot clinical context, improving sample efficiency and generating transparent, rule-based features [ 16 , 31 , 1 ] . However, existing LLM-based frameworks focus primarily on static tabular data and assume clean, structured inputs. Applying them to longitudinal EHRs, with their complex temporal dependencies, context-exceeding sequence lengths, and structural sparsity, remains a significant, unaddressed challenge. To address these limitations, we introduce FeatEHR-LLM, a novel framework leveraging the clinical knowledge of LLMs to generate clinically motivated tabular features from irregularly sampled EHR time series. Crucially, FeatEHR-LLM addresses structural sparsity and variable frequencies by equipping the LLM with specialized programmatic tools to query and aggregate irregular temporal data. To reduce privacy exposure, the framework operates at the metadata level, exposing only the dataset schema and clinical task description to the LLM rather than raw patient-level records. The LLM generates executable feature-extraction code without direct access to sensitive patient records. During execution, FeatEHR-LLM transforms longitudinal measurements into engineered features that seamlessly integrate into standard ML pipelines. As illustrated in Figure 1 , our framework executes both univariate and multivariate engineering to capture complex physiological interactions across multiple variables. The major contributions of this work are as follows: • We propose FeatEHR-LLM, an LLM-empowered feature generation framework designed for irregularly sampled EHR time series. By exposing only dataset schemas and clinical task descriptions rather than raw records, our approach reduces privacy exposure to the LLM. • We directly address irregular sampling via a tool-augmented generation mechanism. By equipping the LLM with specialized temporal data-reading tools, it writes feature-extraction code that explicitly handles structural sparsity and variable measurement frequencies. • We develop a multi-source feature generation mechanism integrating LLM knowledge, task context, and inter-feature interactions through univariate and multivariate engineering. This is paired with an iterative strategy and in-loop validation, allowing the LLM to adaptively refine the generated features. • We implement a robust feature-to-code translation pipeline with syntax and runtime verification. We evaluate FeatEHR-LLM on eight clinical tasks across four datasets, achieving the highest mean AUROC on 7 out of 8 tasks (improving up to 6% over baselines). 2 Related Works In this section, we review feature engineering methods for tabular and time series data, with emphasis on approaches that generate explicit, inspectable features (including LLM-assisted pipelines). We intentionally exclude end-to-end learning-based representation methods (e.g., deep sequence encoders) because they learn latent embeddings rather than explicit engineered features, and therefore belong to a different methodological paradigm and evaluation setting than the one targeted in this work. Transformation-Based Feature Engineering Transformation-based methods automate feature creation through predefined mathematical operations. For tabular data, frameworks like OpenFE [ 42 ] , AutoFeat [ 17 ] , and FETCH [ 23 ] successfully generate massive candidate spaces, relying on selection algorithms to filter for predictive utility. For time series, libraries extract temporal descriptors across statistical, spectral, and fractal domains (e.g., tsfresh [ 7 ] , TSFEL [ 9 ] , Kats [ 18 ] , and Catch22 [ 29 ] ). While effective at bulk generation, these methods suffer from two major limitations. First, they lack semantic awareness; they blindly apply mathematical transformations without understanding the domain context of the variables, often leading to a “curse of dimensionality” filled with redundant or clinically meaningless features. Second, they are overwhelmingly designed for regularly sampled signals. They fundamentally struggle with the highly irregular sampling rates, variable sequence lengths, and informative missingness that characterize clinical EHR time series, requiring ad-hoc imputation steps that can destroy underlying temporal patterns. LLM-Based Feature Engineering Recent work has leveraged Large Language Models (LLMs) to inject semantic domain knowledge into feature engineering. Current approaches generally formulate this as a code-generation or rule-synthesis task. Methods like CAAFE [ 16 ] and FeatLLM [ 15 ] generate human-readable transformations from dataset descriptions or few-shot examples, while others like LLM-FE [ 1 ] , FeRG-LLM [ 22 ] , OCTree [ 31 ] , MedFeat [ 43 ] , and FAMOSE [ 5 ] integrate iterative feedback mechanisms such as validation-driven refinement, evolutionary search, memory-guided selection, or ReAct-style reasoning-execution cycles to improve generated features [ 40 ] . Despite these advancements, synthesizing the current literature reveals that LLM-based feature engineering remains constrained to static, tabular data. Adapting these pipelines to time series, particularly EHR data, presents unresolved challenges. First, existing methods operate on row-wise schema descriptions and lack the temporal reasoning required to generate complex longitudinal aggregations (e.g., tracking a patient’s rolling health deterioration). Second, context

Read the complete text