Enhancing Contrastive Learning-based Electrocardiogram Pretrained Model with Patient Memory Queue

Xiaoyu Sun, Yang Yang, Xunde Dong · 2025

Read Enhancing Contrastive Learning-based Electrocardiogram Pretrained Model with Patient Memory Queue by Xiaoyu Sun, Yang Yang, Xunde Dong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-06310.txt.

Introduction

Xiaoyu Sun, Yang Yang, and Xunde Dong work on heart signal analysis in a 2025 paper about electrocardiogram, or ECG, models. Labeled ECG data are scarce, so models often learn first from unlabeled traces. One strong idea treats recordings from the same patient as a matched set, since they share traits, but a small training batch holds too few such pairs.

Their answer is a patient memory queue, a large store that keeps past patient traces available during training. They also add two new ways to vary the data, giving the model more views of matched and unmatched pairs. Tests on three public data sets, at three data levels, beat earlier methods and held up better when labeled data were scarce.

Catalog record

CREATORXiaoyu Sun, Yang Yang, Xunde Dong
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-06310

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Enhancing Contrastive Learning-based Electrocardiogram Pretrained Model with Patient Memory Queue Xiaoyu Sun, Yang Yang, Xunde Dong Abstract In the field of automatic Electrocardiogram (ECG) diagnosis, due to the relatively limited amount of labeled data, how to build a robust ECG pretrained model based on unlabeled data is a key area of focus for researchers. Recent advancements in contrastive learning-based ECG pretrained models highlight the potential of exploiting the additional patient-level self-supervisory signals inherent in ECG. They are referred to as patient contrastive learning. Its rationale is that multiple physical recordings from the same patient may share commonalities, termed patient consistency, so redefining positive and negative pairs in contrastive learning as intra-patient and inter-patient samples provides more shared context to learn an effective representation. However, these methods still fail to efficiently exploit patient consistency due to the insufficient amount of intra-inter patient samples existing in a batch. Hence, we propose a contrastive learning-based ECG pretrained model enhanced by the P atient M emory Q ueue (PMQ), which incorporates a large patient memory queue to mitigate model degeneration that can arise from insufficient intra-inter patient samples. In order to further enhance the performance of the pretrained model, we introduce two extra data augmentation methods to provide more perspectives of positive and negative pairs for pretraining. Extensive experiments were conducted on three public datasets with three different data ratios. The experimental results show that the comprehensive performance of our method outperforms previous contrastive learning methods and exhibits greater robustness in scenarios with limited labeled data. The code is available at https://github.com/3hiuwoo/PMQ. † † paperid: 0527 1 Introduction The electrocardiogram (ECG) is a non-invasive method for measuring the heart’s electrical activity and has gained increasing importance for detecting and diagnosing cardiac diseases. Numerous deep learning methods have been introduced to learn the intricate patterns inherent in the complex periodic rhythms of ECG [ Liu et al.(2021)Liu, Wang, Li, and Qin ] . However, due to the challenge of obtaining high-quality manual labels of ECG, which is labor-intensive for physicians, these models are hindered by data scarcity. Self-supervised learning (SSL) [ Devlin et al.(2019)Devlin, Chang, Lee, and Toutanova , He et al.(2020)He, Fan, Wu, Xie, and Girshick , He et al.(2021)He, Chen, Xie, Li, Dollár, and Girshick , Gidaris et al.(2018)Gidaris, Singh, and Komodakis , Zhang et al.(2016)Zhang, Isola, and Efros , Dong et al.(2024)Dong, Wu, Wang, Qiu, Zhang, Wang, and Long ] offers a way to address the problem by taking advantage of the extensive unlabeled data available on the Internet. As a way of SSL, contrastive learning [ Jaiswal et al.(2020)Jaiswal, Babu, Zadeh, Banerjee, and Makedon ] has demonstrated significant advantages in computer vision and has attracted widespread attention across various domains. The core idea of contrastive learning is the instance discrimination pretext task [ Wu et al.(2018)Wu, Xiong, Yu, and Lin ] that compels the model to learn similar representations for positive samples augmented from the same data instance and dissimilar representations for negative samples from different data instances. Motivated by its success, the research community has adopted and further developed contrastive learning for ECG analysis. Among them, a line of previous work leverages the additional data level of the ECG series: patient level [ Kiyasseh et al.(2021)Kiyasseh, Zhu, and Clifton , Diamant et al.(2022)Diamant, Reinertsen, Song, Aguirre, Stultz, and Batra , Wang et al.(2023)Wang, Han, Wang, and Zhang ] . These methods extend contrastive learning by leveraging considerably more positive samples under intra-patient contexts to learn a more generalizable representation for ECG data. We refer to them as patient contrastive learning, as they utilize the fact that multiple physical recording instances can share meaningful context such as periodic cardiac patterns in ECG if derived from the same patient [ Kiyasseh et al.(2021)Kiyasseh, Zhu, and Clifton ] . Benefiting from the additional data level, these methods have demonstrated several advantages over contrastive learning methods designed for instance level, such as less dependency on augmentation design [ Diamant et al.(2022)Diamant, Reinertsen, Song, Aguirre, Stultz, and Batra ] and patient-specific representation [ Kiyasseh et al.(2021)Kiyasseh, Zhu, and Clifton ] . Existing methods, however, are limited in their ability to fully capture patient-level shared context. Specifically, during training on large pretraining datasets, only a mini-batch of data is sampled at each iteration. As a result, data from the same patient are unlikely to appear within the same batch, leading to a scarcity of intra-patient positive pairs. This limitation reduces the approach to standard instance-level contrastive learning [ Chen et al.(2020)Chen, Kornblith, Norouzi, and Hinton ] . To alleviate this, methods like COMET [ Wang et al.(2023)Wang, Han, Wang, and Zhang ] have elaborated on warping the random batch sampler to ensure the quantity of positive samples. Rather than thoroughly shuffling the data, it first groups samples from the same trial into sets, and then shuffles the order of samples within each set. In the end, all patient sets are sorted while preserving the internal order of samples. As a result, all samples from the same trial will be included in the same batch. Because samples from the same trial are naturally from the same patient, thus the number of positive samples is increased. However, due to the incomplete shuffling and constrained batch size, each batch still contains too few patients to provide diverse inter- and intra-patient samples. Figure 1: Difference between ours method (PMQ) and previous methods. Different similarity matrices used for computing contrastive loss between two augmented views of a mini-batch of size B with corresponding patient IDs \{\,p_{1},\,\ldots,\,p_{B}\,\} . Each element at row i and column j represents the cosine similarity between the first augmented view of the i -th sample and the second augmented view of the j -th sample. (a) Early patient contrastive learning methods, such as CLOCS [ Kiyasseh et al.(2021)Kiyasseh, Zhu, and Clifton ] , randomly shuffle the dataset and sample a mini-batch irrespective of patient identity. In addition to diagonal elements, off-diagonal entries (\,i,\,j\,) are also treated as positives if p_{i}=p_{j} . (b) Approaches like COMET [ Wang et al.(2023)Wang, Han, Wang, and Zhang ] apply hierarchical shuffling to retain trial-level grouping, sampling mini-batches sequentially. This increases the likelihood of neighboring samples sharing the same patient ID within a batch. (c) In contrast to prior methods, we incorporate an external patient memory queue containing M representations and their associated patient IDs. At each iteration, an additional similarity matrix is computed between the mini-batch and the patient memory queue. Motivated by this gap, we introduce a patient memory queue as an auxiliary module to the end-to-end paradigm. An shown in Figure 1 , by storing a substantial number of prior representations from different patients, the queue ensures an adequate quantity of intra-inter patient samples [ Robinson et al.(2020)Robinson, Chuang, Sra, and Jegelka , Awasthi et al.(2022)Awasthi, Dikkala, and Kamath ] , enhancing the patient contrastive learning model by effectively exploiting patient consistency. Our contributions are summarized as the following: 1. We proposed a patient contrastive learning method that incorporates a dynamic P atient M emory Q ueue, entitiled PMQ, which stores a great number of positive and negative samples under patient context. This enables the pretraining process to maximize the exploration of context information in ECG series. 2. We

Read the complete text