HypEHR: Hyperbolic Modeling of Electronic Health Records for Efficient Question Answering

Yuyu Liu, Sarang Rajendra Patil, Mengjia Xu, Tengfei Ma · 2026

Read HypEHR: Hyperbolic Modeling of Electronic Health Records for Efficient Question Answering by Yuyu Liu, Sarang Rajendra Patil, Mengjia Xu, Tengfei Ma (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-21027.txt.

Introduction

Liu and co-authors propose HypEHR, a compact Lorentzian model for EHR questions that avoids costly pipelines. Medical code systems and patient paths show hyperbolic shape, so HypEHR embeds codes, visits, and questions in that space. It answers through matching cross-attention with typed pointer heads. Training on next-visit diagnosis guesses plus rank-aware controls aligns its vectors with the ICD. On two MIMIC-IV-based benchmarks, HypEHR nears language-model methods while using far fewer weights, and its code is public.

Catalog record

CREATORYuyu Liu, Sarang Rajendra Patil, Mengjia Xu, Tengfei Ma
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44455
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.21027

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

HypEHR: Hyperbolic Modeling of Electronic Health Records for Efficient Question Answering Thanks: Accepted at Findings of ACL 2026 Yuyu Liu, Sarang Rajendra Patil, Mengjia Xu, Tengfei Ma Abstract Electronic health record (EHR) question answering is often handled by LLM-based pipelines that are costly to deploy and do not explicitly leverage the hierarchical structure of clinical data. Motivated by evidence that medical ontologies and patient trajectories exhibit hyperbolic geometry, we propose HypEHR, a compact Lorentzian model that embeds codes, visits, and questions in hyperbolic space and answers queries via geometry-consistent cross-attention with type-specific pointer heads. HypEHR is pretrained with next-visit diagnosis prediction and hierarchy-aware regularization to align representations with the ICD ontology. On two MIMIC-IV-based EHR-QA benchmarks, HypEHR approaches LLM-based methods while using far fewer parameters. Our code is publicly available at https://github.com/yuyuliu11037/HypEHR . 1 Introduction Electronic health record (EHR) question answering (EHR-QA) aims to answer natural-language clinical questions over a patient’s longitudinal record 5 . For example, “has patient been admitted to the emergency room on the first hospital visit” or “is there any microbiological test result on the current hospital visit for patient’s blood culture?” 3 . Recent datasets over MIMIC-III/IV have driven progress, but most methods sit at three extremes: (i) EHR representation learning methods, including sequential and graph-based models that encode temporal and heterogeneous clinical structures 27 ; 22 ; 20 ; 9 , (ii) text-to-SQL or graph semantic parsers 39 ; 21 ; 35 ; 4 , and (iii) retrieval-augmented pipelines built on large language models (LLMs) such as GPT-3.5/4 19 ; 11 ; 40 . These approaches can be accurate, yet they are computationally heavy, hard to deploy under strict privacy constraints, and largely ignore the strong structural priors present in EHR data. Prior research in EHR representation learning indicates that medical codes and longitudinal patient trajectories are intrinsically hierarchical, exhibiting properties that align closely with hyperbolic geometry 26 ; 6 . While Euclidean embeddings distort tree-like structures, hyperbolic spaces can embed hierarchies with arbitrarily low distortion. Building on this insight, 25 demonstrate that hyperbolic embeddings of the medical code hierarchy can improve temporal health event prediction, but their resulting patient representations are ultimately modeled in a Euclidean space. This raises a central research question: can a compact model, explicitly aligned with the intrinsic geometry of EHRs at the patient level, compete with billion-parameter LLMs in complex question answering? Figure 1: Comparison of workflows between text-to-SQL ( top ) and our method ( bottom ). The text-to-SQL-based methods typically rely on large-scale pretrained large language models to generate accurate SQL queries, whereas our method is specifically adapted to medical data, enabling comparable performance with a significantly smaller number of parameters. We address this question with HypEHR ( Hyp erbolic modeling of E lectronic H ealth R ecords), a novel compact EHR-QA framework based on hyperbolic clinical sequence modeling. The comparison of pipelines between HypEHR and text-to-SQL is shown in Figure 1 . The resulting model achieves performance comparable to large language models, while being orders of magnitude smaller (22M) than typical LLM-based pipelines (trillions of parameters) and thus more suitable for on-premise, privacy-conscious clinical settings. 2 Methodology Figure 2: The overall framework of our proposed HypEHR. 2.1 Problem Definition and Model Overview Given a question q and the visit history of patient \mathcal{H}_{p}=\{v_{1},\dots,v_{T}\} , where each v_{t} is associated with a set of medical concepts (including diagnosis codes, procedure codes, drug codes, admission time and other laboratory records), the model must look up the right visit, find the correct department, and return a clear answer. From the perspective of answer types, the common QA pairs can be categorized into four classes: boolean, concept, numerical, and integer count. We summarize the main modules of the HypEHR framework in Figure 2 to provide an overview and the notations used throughout this paper in Table 3 . HypEHR consists of two stages. The first stage, patient encoder pretraining , learns a hyperbolic patient encoder by joint targets of next-visit diagnosis prediction and hierarchy-aware regularization. The next stage, question-answer training , trains answer type-specified heads using embeddings from the frozen patient encoder. 2.2 Hyperbolic Clinical Sequence Encoder We now present the first stage of our model — patient encoder pretraining . Each medical concept c\in v_{t} is embedded into a Lorentzian hyperbolic manifold \mathbb{H}^{d}_{L} via a learnable embedding function e_{c}\in\mathbb{H}^{d}_{L} . Within a visit, we aggregate the code embeddings using a hyperbolic attention mechanism to obtain a visit representation h_{t}\in\mathbb{H}^{d}_{L} . The sequence \{h_{t}\}_{t=1}^{T} is then processed by a multi-layer Lorentz Transformer encoder, which adapts self-attention, residual connections, and normalization to the Lorentz manifold, yielding contextualized visit states \{z_{t}\}_{t=1}^{T} and a global summary representation z_{\text{[CLS]}}\in\mathbb{H}^{d}_{L} . This global summary is then mapped to all diagnosis codes, producing next-visit diagnosis prediction loss \mathcal{L}_{\text{diag}} . To encode hierarchical relationships in diagnosis code embeddings, we train patient encoder in a multi-task fashion with a hierarchy-aware regularizer \mathcal{L}_{\text{hier}} using ICD code trie built by chapters \to blocks \to categories \to subcategories, encouraging embeddings of codes that share ancestors in the ontology to be closer in hyperbolic distance than unrelated codes. We further decompose \mathcal{L}_{\text{hier}} into a radial hierarchy term and a relative hierarchy term to jointly enforce depth ordering and local separation. Overall, the encoder parameters are optimized to minimize a joint objective \mathcal{L}=\mathcal{L}_{\text{diag}}+\lambda\mathcal{L}_{\text{hier}} (1) where \mathcal{L}_{\text{diag}} is the binary cross-entropy loss and \displaystyle\mathcal{L}_{\text{hier}} \displaystyle=\mathcal{L}_{\text{rad}}+\mu\,\mathcal{L}_{\text{rel}}, (2) \displaystyle\mathcal{L}_{\text{rad}} \displaystyle=\sum_{(p,c)\in\mathcal{P}}\max\!\Big(0,\;\|e_{p}\|_{\mathbb{H}}-\|e_{c}\|_{\mathbb{H}}+\beta\Big), (3) \displaystyle\mathcal{L}_{\text{rel}} \displaystyle=\sum_{(a,a^{+},a^{-})\in\mathcal{T}}\max\!\Big(0,\;d_{\mathbb{H}}(e_{a},e_{a^{+}}) \displaystyle\hskip 32.00002pt-d_{\mathbb{H}}(e_{a},e_{a^{-}})+\alpha\Big) (4) with \|e\|_{\mathbb{H}}:=d_{\mathbb{H}}(e,\mathbf{o}) , and \mathbf{o} denotes the origin of the Lorentz hyperboloid. Here \mathcal{P} denotes parent–child pairs extracted from the ICD trie, and \mathcal{T} denotes triplets where a^{+} is an ancestor-related (e.g., parent/same-branch) code of a and a^{-} is a non-ancestor code; \alpha,\beta>0 are margin hyperparameters, and \lambda,\mu>0 balance the relative importance of each loss term. This yields a geometry-aware representation of patient trajectories that is later reused for downstream question answering. 2.3 Hyperbolic EHR-QA Model Given a natural-language question q about a patient p , our model combines a natural language encoder with the Lorentzian patient encoder described above. The question q is first encoded by a biomedical pre-trained language encoder into a Euclidean vector u_{q}\in\mathbb{R}^{d_{e}} , which is then projected into the hyperbolic manifold via an affine map followed by an exponential map at the origin, yielding a question representation z_{q}\in\mathbb{H}^{d}_{L} . We then perform hyperbolic cross-attention from z_{q} over the sequence of visit states \{z_{t}\}_{t=1}^{T} : attention scores are defined as negative scaled hyperbolic distances s_{t}=-\gamma d_{\mathbb{H}}(z_{q},z_{t}) and normalized via softmax to obtain weights \alpha_{t} . A hyperbolic weighted Fréchet mean of visit states, z^{\mathrm{visit}}_{p|q}=\mathrm{HypAgg}(\{\alpha_{t},z_{t}\}_{t=1}^{T}) , serves as a question-conditioned patient summary. We implement Hyperbolic Aggregation (HypAgg) as the exponential map at the origin of the weighted average of log-mapped points, which approximates the Riemannian barycenter on the Lorentz manifold. For fine-grained reasoning, we additionally apply a second-stage hyperbolic attention over code embeddings within the top- k attended visits to construct a code-level rationale vector z^{\mathrm{code}}_{p|q}\in\mathbb{H}^{d}_{L} . Depending on the QA pair type, specialized answer heads consume (z_{q},z^{\mathrm{visit}}_{p|q},z^{\mathrm{code}}_{p|q}) to produce logits over the corresponding classes. The QA-specific components on top of the frozen language and patient encoders are trained using standard cross-entropy losses. Details of QA heads can be found in Appendix D . 3 Experiments

Read the complete text