WISTERIA: Learning Clinical Representations from Noisy Supervision via Multi-View Consistency in Electronic Health Records
Ruan Dong, Yuanyun Zhang, Shi Li · 2026
Read WISTERIA: Learning Clinical Representations from Noisy Supervision via Multi-View Consistency in Electronic Health Records by Ruan Dong, Yuanyun Zhang, Shi Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-09765.txt.
Introduction
Dong and two colleagues rethought how medical AI should learn from digital health records, where labels are messy by nature. Billing codes, rough phenotypes, and patchy notes make guidance weak, noisy, and tied to each hospital. Standard record models still treat those labels as ground truth. WISTERIA instead treats each label as a noisy glimpse of a hidden medical state.
The framework builds several weak label sources and forces agreement across the label patterns they imply, a quiet denoising step that recovers real medical structure from disagreeing labelers. Ontology-aware shaping adds meaning-based structure to the label space. WISTERIA improved prediction scores across standard record tests, resisted label noise, and transferred better across hospitals than sequence-based pretraining.
Catalog record
| CREATOR | Ruan Dong, Yuanyun Zhang, Shi Li |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.09765
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
WISTERIA: Learning Clinical Representations from Noisy Supervision via Multi-View Consistency in Electronic Health Records Ruan Dong, Yuanyun Zhang, Shi Li Abstract Representation learning in electronic health records (EHR) has largely followed paradigms inherited from natural language processing, relying on sequence modeling and reconstruction-based objectives that treat clinical labels as ground truth. However, real-world clinical supervision is inherently weak, arising from heterogeneous, noisy, and institution-specific labeling processes such as billing codes, heuristic phenotypes, and incomplete annotations. In this work, we propose WISTERIA, a weakly-supervised representation learning framework that models labels as stochastic observations of an underlying latent clinical state. Instead of optimizing against a single supervision signal, WISTERIA constructs multiple weak supervision operators and learns representations by enforcing consistency across their induced label distributions. This multi-view formulation induces an implicit denoising mechanism, allowing the model to recover clinically meaningful structure by reconciling disagreement between noisy labelers. We further incorporate ontology-aware regularization in the label space to impose semantic structure over supervision signals. Empirically, WISTERIA improves predictive performance across standard EHR benchmarks, demonstrates strong robustness to label noise, and exhibits superior cross-institutional generalization compared to sequence-based pretraining objectives. These results suggest that explicitly modeling the supervision process—rather than treating labels as fixed targets—provides a more appropriate inductive bias for learning robust and clinically meaningful representations from EHR data. 1 Introduction Recent progress in healthcare foundation models has largely adhered to a now-familiar paradigm: increasing model scale, pretraining on large heterogeneous corpora, and transferring learned representations to downstream clinical tasks Vaid et al. 2023 ; Burkhart et al. 2025 ; Thieme et al. 2023 ; He et al. 2024 . This paradigm has been successfully instantiated across modalities, including structured electronic health records (EHR), imaging, and physiological signals, with a growing emphasis on scale, modality diversity, and pretraining objectives Burger et al. 2025 ; Guo et al. 2025 ; Liang et al. 2024 ; Awais et al. 2025 ; Thakur 2024 . In EHR modeling, patient histories are typically represented as sequences of discrete tokens—diagnoses, procedures, medications, and lab events—processed by architectures adapted from language modeling He et al. 2022 ; Devlin et al. 2019 . These models inherit architectural principles from deep learning in vision and NLP, including residual connections, hierarchical feature extraction, and attention-based sequence modeling He et al. 2017 ; He et al. 2015 ; He et al. 2019 . While effective, this formulation implicitly assumes that clinical information is fully captured by the order and identity of observed events. This assumption becomes particularly limiting under weak supervision, which is the dominant regime in real-world clinical data. Clinical labels are rarely direct measurements of physiological state; instead, they are noisy proxies derived from billing codes, heuristic phenotypes, or institution-specific annotation pipelines. As a result, supervision is sparse, biased, and heterogeneous, reflecting the process of documentation rather than the underlying clinical reality. Sequence-based pretraining objectives absorb these artifacts, often entangling clinically meaningful structure with noise induced by coding practices. In this work, we introduce WISTERIA ( W eakly-supervised I nference of S tructured la T ent E mbeddings for cl R inical represent A tions), a new paradigm for EHR representation learning that explicitly models supervision as a stochastic, multi-view process. Rather than treating labels as fixed targets, WISTERIA treats them as noisy observations generated from an unobserved latent clinical state. Formally, let z\in\mathcal{Z} denote a latent representation of patient physiology, and let x\in\mathcal{X} denote the observed EHR. We posit that clinical labels y\in\mathcal{Y} are generated through a noisy channel: y\sim p(y\mid z;\psi), where \psi parameterizes institution-specific labeling functions. Under this view, observed labels are not ground truth, but stochastic realizations of a broader supervision process. To operationalize this idea, we construct a set of weak supervision operators \{\mathcal{W}_{k}\}_{k=1}^{K} , each mapping a patient record x to a pseudo-label distribution \tilde{y}_{k}=\mathcal{W}_{k}(x) . These operators capture diverse sources of supervision, including heuristic phenotyping rules, ontology-based label propagation, co-occurrence statistics, and cross-modal agreement signals. Each operator provides a noisy and partial view of the latent state z , and no single operator is assumed to be correct. We learn representations by enforcing consistency across these supervision views. Let h_{\theta}(x) denote the learned embedding, and let g_{\theta,k} be a prediction head associated with operator \mathcal{W}_{k} . The training objective is: \mathcal{L}_{\mathrm{WISTERIA}}=\sum_{k=1}^{K}\mathbb{E}_{x}\left[\ell\big(g_{\theta,k}(h_{\theta}(x)),\,\tilde{y}_{k}\big)\right]+\lambda\sum_{k\neq k^{\prime}}\mathbb{E}_{x}\left[D\big(g_{\theta,k}(h_{\theta}(x)),\,g_{\theta,k^{\prime}}(h_{\theta}(x))\big)\right], where \ell is a divergence (e.g., cross-entropy) and D enforces agreement between supervision views. This objective encourages the representation to capture structure that is stable across heterogeneous, noisy supervision signals. This formulation induces an implicit denoising mechanism. Each operator \mathcal{W}_{k} can be interpreted as a noisy estimator of p(y\mid z) ; enforcing agreement across operators recovers a shared latent signal under a multi-view consistency principle. Unlike conventional supervised learning, which treats labels as fixed, WISTERIA models the generative process of supervision itself, allowing the representation to disentangle signal from annotation noise. To incorporate clinical structure, we introduce a label-space geometry regularizer . Let \mathcal{Y} be endowed with a graph G=(V,E) derived from clinical ontologies. Predictions are regularized to be smooth with respect to the graph Laplacian L : \mathcal{L}_{\mathrm{graph}}=\sum_{k=1}^{K}\mathbb{E}_{x}\left[g_{\theta,k}(h_{\theta}(x))^{\top}L\,g_{\theta,k}(h_{\theta}(x))\right]. This encourages semantically related clinical concepts to have consistent representations, enabling generalization across coding systems and improving robustness in long-tail settings. A key conceptual shift in WISTERIA is that invariance is enforced in the supervision space rather than the input space. Instead of relying on data augmentations over EHR sequences, we construct multiple noisy supervisory signals and require the representation to reconcile them. This is particularly well-suited to clinical data, where input perturbations are difficult to define, but supervision signals are abundant and diverse. Moreover, WISTERIA introduces a new scaling axis: the number and diversity of weak supervision operators. Increasing K provides additional constraints on the latent representation, improving identifiability and robustness without requiring additional labeled data. This complements traditional scaling strategies based on dataset size and model capacity He et al. 2015 ; He et al. 2019 ; He et al. 2017 , but operates by scaling supervision rather than data. In summary, WISTERIA makes the following contributions. It reframes EHR representation learning as inference over latent clinical states under weak, noisy supervision. It introduces a multi-view consistency objective that aggregates heterogeneous supervisory signals. It incorporates ontology-aware regularization directly in the label space, enabling structured reasoning over clinical concepts. Finally, it demonstrates that modeling supervision as a stochastic process yields representations that are more robust to noise, bias, and sparsity, suggesting a new direction for healthcare foundation models beyond sequence-centric pretraining. 2 Related Works Self-supervised learning (SSL) has become the dominant paradigm for representation learning in healthcare, enabling the emergence of foundation models across structured EHRs, clinical text, imaging, and physiological signals Lowe et al. ; Zhang and Li 2025a ; Lin et al. April 2025 ; Huiliang et al. 2025 ; Ran et al. 2025 ; Zhang and Li 2025b ; Lee and Lindsey 2024 ; Chou et al. 2025 ; Zhou et al. 2026 ; Zhang et al. 2026 ; Zhang and Li 2026 . These approaches follow a now-standard recipe: large-scale pretraining on unlabeled data produces representations that transfer effectively to downstream clinical tasks. This trajectory mirrors developments in natural language processing and computer vision, where masked modeling and generative objectives Devlin et al. 2019 ; He et al.