EchoingECG : An Electrocardiogram Cross-Modal Model for Echocardiogram Tasks

Yuan Gao, Sangwook Kim, Chris McIntosh · 2025

Read EchoingECG : An Electrocardiogram Cross-Modal Model for Echocardiogram Tasks by Yuan Gao, Sangwook Kim, Chris McIntosh (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-25791.txt.

Introduction

Yuan Gao, Sangwook Kim, and Chris McIntosh teach a cheap heart test to predict results of a costlier one in a paper from 2025. An electrocardiogram, or ECG, records the heart's electrical signals and is widely available. An echocardiogram uses ultrasound and gives richer detail but needs hospital resources. Their model, EchoingECG, learns from paired examples of both tests.

A teacher model trained on ultrasound and text guides a student that sees only ECG data. The student also reports its own uncertainty for each reading. Tests showed better predictions than leading ECG foundation models across several training sizes. Outside validation supported the results. The uncertainty scores flagged readings the model found hard to judge.

Catalog record

CREATORYuan Gao, Sangwook Kim, Chris McIntosh
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-25791

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

EchoingECG : An Electrocardiogram Cross-Modal Model for Echocardiogram Tasks Yuan Gao, Sangwook Kim, Chris McIntosh Abstract Electrocardiogram (ECG) is a widely used tool for assessing cardiac function due to its low cost and accessibility. Emergent research shows that ECGs can help make predictions on key outcomes traditionally derived from more complex modalities such as echocardiograms (ECHO), enabling the use of ECGs as a more accessible method to predict broader measurements of cardiac function. ECHO, in particular, are of great importance because they require considerable hospital resources while playing a key role in clinical cardiac assessment. To aid this use case, we introduce EchoingECG , a probabilistic student-teacher model that leverages uncertainty-aware ECG embeddings and ECHO supervision to improve ECG-based cardiac function prediction. Our approach integrates Probabilistic Cross-Modal Embeddings (PCME++), a probabilistic contrastive framework, with ECHO-CLIP, a vision-language pre-trained model trained on ECHO-text pairs, to distill ECHO knowledge into ECG representations. Through experiments and external validation, we showed that EchoingECG outperforms state-of-the-art foundation ECG models in zero-shot, few-shot, and fine-tune settings for ECHO predictions based on ECG. We also highlighted that variance estimation (enabled through our method) enhanced our understanding of model performance by identifying underlying regions of uncertainty within ECGs. The code is available: https://github.com/mcintoshML/EchoingECG . 1 Introduction Electrocardiograms (ECGs) are widely used for cardiac assessment due to their accessibility and cost-effectiveness [ 3 ] . A growing area of research leverages the use of ECGs to estimate echocardiogram (ECHO) variables to evaluate heart function [ 31 , 35 ] , which could enhance early disease detection and expand healthcare accessibility, particularly in resource-limited settings. However, replicating ECHO-derived insights from ECGs remains challenging due to (1) complexity of ECG signals and (2) scarcity of ECG data labeled with ECHO outcomes. Figure 1: The probabilistic approach employs PCME++ to train the VLPM within a probabilistic space. ECHO Teacher (top) : ECHO-CLIP extracts the distribution’s mean \mu and variance \log(\sigma^{2}) by utilizing multiple frames from ECHO videos. Student Model (bottom): Using PCME++, each encoder generates an l_{2} -normalized vector \mu and a vector \log(\sigma^{2}) to represent a normally distributed random variable. EchoingECG is trained to bind both ECG-Text L_{ecg-text} and ECG-ECHO L_{ecg-echo} . For the first challenge , cross-modality contrastive learning offers a promising approach to learning semantically rich embeddings that capture clinically relevant signal patterns [ 23 , 26 , 10 ] . Emerging models such as ECG-CLIP, MEDBind [ 10 ] , and others adapt this paradigm, using text descriptions (ECG impressions) for training. These ECG models demonstrate broad applicability in various downstream tasks. However, traditional cross-modal contrastive learning to ECG-ECHO and/or ECG-text integration introduces additional complexities. ECG signals exhibit subtle variations, with distinct patterns recurring at non-uniform time intervals and manifesting differently across recordings [ 4 , 11 ] . For example, a 120-second ECG is subdivided into many 10-second windows; some windows capture irregular beats, while others do not, yet all windows correspond to the paired report or ECHO. Consequently, injective embeddings in traditional contrastive learning may be limiting as a deterministic embedding space, which means there exists only one-to-one mappings. To address this, we propose contrastive cross-modal learning using probabilistic vision-language models (VLPMs), introducing more flexible embeddings. Unlike traditional contrastive learning, probabilistic VLPMs enable many-to-many mappings, better capturing the inherent uncertainty in cross-modality binding. Although we are subject to natural physiological and measurement variations in our framework, different ECG recordings can converge on the same underlying cardiac condition observed in the text or ECHO. This many-to-many mapping reflects the reality that various representations (ECG signals) may all indicate the same cardiac state, thereby enhancing the model’s robustness. We employ Probabilistic Cross-Modal Embeddings (PCME++) [ 8 , 7 ] , which explicitly model uncertainty by learning both the mean and variance of latent representations. By applying the variational information bottleneck principle, PCME++ enables the network to capture uncertainty inherent in paired modalities. This probabilistic structure allows all ECG windows to map to all ECHO frames and/or possible texts, with lower certainty assigned to mappings between less relevant segments. To address the second challenge –the need for large-scale annotated ECG-ECHO datasets–recent research has explored using foundation models as teachers to enhance learning in domains with limited labeled data. The Unified Representation of Language, Images, and Point Clouds (ULIP) [ 34 ] exemplifies this, where it leveraged large-scale VLPMs (2D model) to assist training 3D point cloud understanding. ULIP demonstrated that models using a foundation teacher on paired unlabeled data could effectively transfer knowledge to different modalities. We extended this idea by leveraging ECHO-CLIP [ 6 ] , a VLPM trained on ECHO-text pairs, as a teacher model to guide ECG representation learning, through training on ECHO-ECG pairs . Our approach distills echo-based cardiac information into ECG representations and, through this alignment, enables ECHO-based training while mitigating large-scale annotated datasets. Contributions: We introduce EchoingECG , a probabilistic student-teacher framework that integrates PCME++ and ECHO-CLIP for enhanced ECG modeling. Our contributions are: (1) the development of a probabilistic multimodal ECG model that captures structural and functional insights from ECHO, (2) a knowledge distillation approach using a student-teacher method that reduces the dependency on large-scale annotated datasets, and (3) experimental evidence showing that probabilistic modeling in EchoingECG enables an uncertainty-aware evaluation of downstream tasks. 2 Methods and Materials 2.1 Probabilistic Contrastive Learning Temporal variability is critical in ECGs, as disease manifestation can occur over time. Traditional deterministic contrastive learning methods, InfoNCE [ 28 ] , enforce one-to-one mappings, which often struggle to capture the inherent many-to-many relationships in ECGs–where distinct signal patterns can encode similar clinical insights simultaneously. Using a probabilistic embedding space: our framework explicitly models this variance, allowing ECG representations to account for signal dynamics, noise, and artifacts. To achieve this, we utilized the PCME++ framework [ 7 ] , described below. Traditional VLPM binding of ECG embedding x_{ecg} and text embedding x_{text} involves deterministic contrastive learning [ 29 ] . Instead, following the probabilistic approach in [ 7 ] , we defined a modality embedding x_{m} as a random variable with a multivariate normal distribution: x_{m}\sim\mathcal{N}\left(\mu_{m},\sigma_{m}^{2}\right) , where \mu_{m}\in\mathbb{R}^{D} is the mean vector and \sigma_{m}^{2}\in\mathbb{R}^{D} denotes the diagonal covariance , as proposed in [ 7 ] . In practice, we optimized \log(\sigma^{2}) , to reduce numerical instability. PCME++, Figure 1 , optimizes embeddings to maximize the mutual likelihood of pairs (e.g., ECG signal, Z_{ecg} , and its corresponding text, Z_{text} ), while separating unmatched pairs. This is achieved through PCME++’s Closed-Form Sampled Distance (CSD) metric (Eq. 1 ), which measures the probabilistic distance between embeddings by combining their \mu and \sigma^{2} . d\left(Z_{m1},\ Z_{m2}\right)\ =\ ||\ \mu_{m1}\ -\ \mu_{m2}\ ||^{2}_{2}\ +\ ||\ \sigma_{m1}^{2}\ +\ \sigma_{m2}^{2}||_{1}\ (1) The properties in Eq. 1 are described in detail in [ 7 ] . In short, it involves the first term encouraging distributions between modality pairs to have a proper distance between pairs and the second term encouraging embeddings to reflect certainty through minimal \sigma^{2} for aligned pairs and larger \sigma^{2} for ambiguity. 2.2 Echo Teacher for ECG-to-ECHO Binding While ECG-text binding using CSD is applied directly from [ 7 ] , EchoingECG used a ECHO Teacher model that extends learning to strengthen semantic representation through cross-modality binding (ECG-ECHO). We focused on enriching

Read the complete text