OC-Distill: Ontology-aware Contrastive Learning with Cross-Modal Distillation for ICU Risk Prediction
Zhongyuan Liang, Junhyung Jo, Hyang-Jung Lee, Sang Kyu Kim, Irene Y. Chen · 2026
Read OC-Distill: Ontology-aware Contrastive Learning with Cross-Modal Distillation for ICU Risk Prediction by Zhongyuan Liang, Junhyung Jo, Hyang-Jung Lee, Sang Kyu Kim, Irene Y. Chen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-16878.txt.
Introduction
Liang and colleagues propose OC-Distill, a two-stage framework for ICU risk prediction using only vital signs at inference. Early forecast of decline and left length of stay enables timely help and better supply sharing. Contrastive pretrain treats all patients as equally strong foes, missing similarity between related cases. An ontology-aware contrast goal uses the ICD tree to score patient match and learn grounded maps. Cross-modal distillation then moves note data into the encoder, hitting state-of-the-art results on MIMIC with vital signs alone.
Catalog record
| CREATOR | Zhongyuan Liang, Junhyung Jo, Hyang-Jung Lee, Sang Kyu Kim, Irene Y. Chen |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49862 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.16878
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
OC-Distill: Ontology-aware Contrastive Learning with Cross-Modal Distillation for ICU Risk Prediction Zhongyuan Liang, Junhyung Jo, Hyang-Jung Lee, Sang Kyu Kim, Irene Y. Chen Abstract Early prediction of severe clinical deterioration and remaining length of stay can enable timely intervention and better resource allocation in high-acuity settings such as the ICU. This has driven the development of machine learning models that leverage continuous streams of vital signs and other physiological signals for real-time risk prediction. Despite their promise, existing methods have important limitations. Contrastive pretraining treats all patients as equally strong negatives, failing to capture clinically meaningful similarity between patients with related diagnoses. Meanwhile, downstream fine-tuning typically ignores complementary modalities such as clinical notes, which provide rich contextual information unavailable in physiological signals alone. To address these challenges, we propose OC-Distill, a two-stage framework that leverages multimodal supervision during training while requiring only vital signs at inference. In the first stage, we introduce an ontology-aware contrastive objective that exploits the ICD hierarchy to quantify patient similarity and learn clinically grounded representations. In the second stage, we fine-tune the pretrained encoder via cross-modal knowledge distillation, transferring complementary information from clinical notes into the model. Across multiple ICU prediction tasks on MIMIC, OC-Distill demonstrates improved label efficiency and achieves state-of-the-art performance among methods that use only vital signs at inference. Code: https://github.com/the-chen-lab/OC-Distill 1 Introduction Figure 1 : Overview of our two-stage framework. (a) We compute patient similarity from the ICD diagnosis hierarchy, providing more informative scores than flat diagnosis matching. (b) Ontology-aware contrastive pretraining down-weights clinically similar negative pairs while strongly repelling dissimilar pairs. (c) Knowledge distillation finetuning transfers representations from a multimodal teacher to a vitals-only student, enabling inference without clinical notes at deployment. Early prediction of clinical deterioration in the intensive care unit (ICU) is critical for timely intervention and effective resource allocation. The availability of continuous physiological monitoring has enabled the development of machine learning (ML) models that can identify patients at elevated risk of adverse outcomes by processing streams of vital signs and other bedside time-series data ( 36 ; 30 ; 12 ; 39 ) . Despite substantial progress, current approaches face two key limitations that constrain their effectiveness in real-world ICU settings. First, contrastive learning (CL) has shown promise for pretraining physiological signals, yet existing methods treat all patients as equally strong negatives during training ( 6 ; 31 ; 21 ) , ignoring that patients often share related disease profiles. Recent work has attempted to address this by computing the percentage of exact ICD code matches between patients as a proxy for clinical similarity ( 9 ) , but this flat matching approach overlooks the hierarchical structure of diagnosis taxonomies. Because the ICD taxonomy contains thousands of specific codes, exact matches are sparse even among clinically related patients. For example, a patient with ureter stones and a patient with kidney stones share substantial pathophysiological commonalities encoded in the ICD hierarchy, yet flat matching assigns them zero similarity. Consequently, pretraining strategies that ignore these hierarchical relationships may learn representations that fail to reflect clinical similarity. Second, training on physiological time series alone ignores other rich modalities in the electronic health record (EHR). In particular, clinical notes contain clinical reasoning and contextual information, such as patient history and behavioral observations, that are not captured in physiological signals alone ( 32 ; 24 ) . While recent work explores multimodal fusion approaches ( 33 ; 40 ; 26 ; 2 ) , these methods require all modalities as inputs at inference time—an assumption that often fails in practice. For example, clinical notes are time-consuming to generate, whereas ICU workflows demand rapid, continuous monitoring. Furthermore, different modalities are often stored in disparate systems, making real-world deployment of such approaches impractical. To address these limitations, we introduce OC-Distill ( O ntology-aware C ontrastive learning with cross-modal Distill ation), a two-stage framework for ICU risk prediction that leverages multimodal supervision during training while requiring only vital signs at inference (Fig 1 ). In the first stage, we develop an ontology-aware CL approach that incorporates patient similarity from the ICD diagnosis hierarchy. Rather than applying a uniform contrastive penalty to all negative pairs, we down-weight clinically similar patients during pretraining, allowing the model to learn representations that preserve the structure of disease relationships. In the second stage, we finetune the pretrained encoder via cross-modal knowledge distillation, where a teacher model trained on both vital signs and clinical notes transfers its representations to a student model initialized from the pretrained encoder that operates exclusively on vitals. This enables the student to benefit from clinical context during training without requiring multimodal inputs at deployment. We evaluate our approach on MIMIC ( 12 ; 15 ) across multiple ICU prediction tasks, label fractions, and observation horizons. Across these settings, incorporating similarity from the ICD ontology into contrastive pretraining yields more clinically informed representations, demonstrating strong label efficiency in both linear evaluation and end-to-end fine-tuning. Furthermore, finetuning with knowledge distillation from clinical notes provides additional gains, and the full OC-Distill pipeline achieves state-of-the-art performance among methods that operate on vital signs alone at inference time. In summary, we make the following contributions: 1. An ontology-aware CL method that integrates patient similarity derived from the ICD ontology, yielding more clinically grounded and label-efficient representations. 2. A multimodal distillation approach that distills knowledge from clinical notes into a vitals model, improving downstream performance without requiring multimodal inputs at inference. 3. A comprehensive empirical evaluation on the MIMIC benchmarks, demonstrating state-of-the-art performance across tasks, label fractions, and observation horizons. 2 Related Work 2.1 Contrastive Pretraining for Clinical Time Series CL is widely used for pretraining representations from unlabeled data due to its simple objective and strong empirical performance ( 14 ; 22 ; 11 ) . A prominent example is SimCLR, which learns embeddings that pull together augmented views of the same instance while uniformly pushing apart other instances in the batch using the NT-Xent loss ( 6 ) . Although originally developed for vision, CL has since been broadly adopted for representation learning on clinical time series ( 41 ; 23 ; 43 ; 31 ; 27 ; 25 ; 21 ) . However, existing methods treat all patients as equally strong negatives, ignoring that patients in clinical settings often share related diagnoses and disease profiles. While recent work has explored flat ICD code overlap as a similarity signal ( 9 ) , this ignores the hierarchical structure of diagnosis taxonomies. We therefore leverage the full ICD ontology to derive fine-grained patient similarity and down-weight clinically similar negatives during contrastive pretraining, grounding the objective in clinically meaningful disease relationships. 2.2 Multimodal Learning for Clinical Time Series Recent work shows that multimodal models can outperform unimodal baselines by leveraging complementary clinical signals ( 19 ; 17 ) . Accordingly, many approaches incorporate fusion mechanisms and report improved performance when combining physiologic time series with additional modalities, such as clinical notes ( 40 ; 26 ; 2 ; 17 ; 13 ) . While effective, fusion-based approaches require all modalities to be available as inputs at deployment, an assumption that often fails in practice. We instead study training-time multimodal supervision, leveraging auxiliary modality (e.g., clinical notes) during training while using only