OpenECG: Benchmarking ECG Foundation Models with Public 1.2 Million Records
Zhijiang Wan, Qianhao Yu, Jia Mao, Wenfeng Duan, Cheng Ding · 2025
Read OpenECG: Benchmarking ECG Foundation Models with Public 1.2 Million Records by Zhijiang Wan, Qianhao Yu, Jia Mao, Wenfeng Duan, Cheng Ding (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-00711.txt.
Introduction
Wan, Yu, Mao, Duan, and Ding introduce OpenECG, a benchmark for heart signal foundation models, in a 2025 paper. The set holds more than a million twelve-lead ECG records from nine centers, all public. The goal is fair testing of models trained on open data.
Three self-supervised methods were trained on two network designs. Tests left one data set out at a time to check generalization. Training on diverse data helped a lot. Two methods beat the contrastive one, and their gains leveled off well before all data were used. The authors say public data can match private sets for building strong ECG models.
Catalog record
| CREATOR | Zhijiang Wan, Qianhao Yu, Jia Mao, Wenfeng Duan, Cheng Ding |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-00711
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
OpenECG: Benchmarking ECG Foundation Models with Public 1.2 Million Records Zhijiang Wan, Qianhao Yu, Jia Mao, Wenfeng Duan, Cheng Ding Abstract This study introduces OpenECG, a large-scale benchmark of 1.2 million 12-lead ECG recordings from nine centers, to evaluate ECG foundation models (ECG-FMs) trained on public datasets. We investigate three self-supervised learning methods (SimCLR, BYOL, MAE) with ResNet-50 and Vision Transformer architectures, assessing model generalization through leave-one-dataset-out experiments and data scaling analysis. Results show that pre-training on diverse datasets significantly improves generalization, with BYOL and MAE outperforming SimCLR, highlighting the efficacy of feature-consistency and generative learning over contrastive approaches. Data scaling experiments reveal that performance saturates at 60-70% of total data for BYOL and MAE, while SimCLR requires more data. These findings demonstrate that publicly available ECG data can match or surpass proprietary datasets in training robust ECG-FMs, paving the way for scalable, clinically meaningful AI-driven ECG analysis. I Introduction Electrocardiography (ECG) is a fundamental tool for diagnosing cardiovascular diseases (CVDs), which are among the leading causes of mortality worldwide. ECG enables clinicians to detect arrhythmias, myocardial infarction, and other heart conditions ( Moreno-Sánchez et al. 2024 ) . Despite its importance, several challenges hinder the effective utilization of ECG in clinical practice: First, the diagnostic accuracy can differ significantly among cardiologists due to varying levels of training and experience. Second, continuous ECG monitoring generates vast amounts of data, making it difficult for cardiologists to analyze and interpret manually within a reasonable timeframe. Artificial intelligence (AI) offers promising solutions to these challenges. Recent advancements in AI, particularly deep learning, have demonstrated remarkable potential in automating ECG analysis, improving diagnostic accuracy, and reducing the burden on clinicians ( Muzammil 2024 ) . Among these advancements, the concept of foundation models (FMs) ( Kolides 2023 ) has emerged as a transformative approach. Foundation models, which are large-scale pre-trained models, have revolutionized various domains, including natural language processing and computer vision. Examples such as GPT ( Floridi 2020 ) for text and CLIP ( Zhang 2021 ) for vision demonstrate how pre-training on diverse datasets can enable models to generalize across multiple tasks. Inspired by this success, several studies have explored the application of foundation models for ECG analysis. Apple developed the first ECG foundation model using ECG data from 106,643 participants ( Abbaspourazad 2023 ) . However, this model is limited to single-lead I ECG and was tested on the same cohort. Additionally, its primary tasks—predicting age, BMI, and sex—do not include heart rhythm analysis. Other foundation models, such as AnyECG ( Wang 2024 ) and ECG-FM ( McKeen 2024 ) , combined data from multiple sources but failed to evaluate their generalizability effectively. Beyond the conventional self-supervised approach to training ECG foundation models, another strategy involves incorporating corresponding diagnostic reports. Models like ECG-Chat ( Zhao 2024 ) , ECG Semantic Integrator ( Yu 2024 ) , and MERL ( Flet-Berliac 2019 ) leverage ECG-text pairs from MIMIC-IV-ECG ( Gow 2023 ) . However, since these datasets are collected from a single center, their ability to generalize across diverse populations and recording environments is limited. This underscores the need for multi-center, diverse datasets to develop more robust and widely applicable ECG foundation models. To address this limitation, our study systematically curated all publicly available 12-lead ECG datasets, integrating data from multiple institutions worldwide. These datasets encompass 1,233,337 ECG recordings from 483,837 patients across 9 centers, including both annotated clinical diagnoses and unannotated raw signals for self-supervised learning. By aggregating data from diverse sources, we aim to build an ECG foundation model that is not only technically superior but also clinically meaningful—one that can assist physicians across different healthcare settings with reliable and unbiased diagnostic support. A key challenge in developing clinically useful AI models is determining the most effective learning paradigm for ECG analysis. In this study, we evaluate three distinct self-supervised learning approaches to building ECG foundation models: (1) SimCLR ( Chen 2020 ) , which learns representations by distinguishing between augmented versions of the same ECG signal; (2) BYOL (Bootstrap Your Own Latent) ( Grill 2020 ) , which removes negative sample constraints and focuses on feature consistency across transformations; and (3) Masked Autoencoder (MAE) ( He 2022 ) , a generative approach that reconstructs missing ECG segments, simulating real-world noisy or incomplete recordings. By comparing these methods, we aim to identify the most clinically relevant approach for learning robust ECG representations that generalize across institutions and patient populations. Instead of finding a ‘winner model’, this study represents the first comprehensive effort to unify all publicly available 12-lead ECG datasets into a single benchmark for foundation model training. More importantly, it underscores the vast yet underutilized potential of open datasets in AI-driven ECG research. Contrary to the prevailing notion that high-performing AI models require proprietary data, we demonstrate that publicly available data, when curated and leveraged effectively, can yield clinically valuable AI models capable of supporting cardiologists in their daily practice. Fig. 1: The key dataset statistics of previous ECG Foundation Models. II Related work II-A Architecture design of FMs for ECGs Transformer-based and CNN-based backbones are the two primary approaches for constructing pre-trained foundation models (FMs) in ECG signal processing. Our previous work ( Han 2024 ) categorizes the current Transformer-based backbones for ECG-FMs into seven distinct groups: ViT-series (Vision Transformer), Bidirectional Encoder Representations from Transformers (BERT-series), Contrastive Language–Image Pre-training (CLIP-series), Generative Pre-trained Transformer (GPT-series), Large Language Model Meta AI (LLaMA-series), and DALL-E-series. Prior to the widespread adoption of Transformers, various traditional pre-training techniques employed deep neural networks (e.g., CNN, RNN, and MLP) and their variants as the basic architectures for pre-training FMs. Given that ECG signals are time-series data, 1D convolutional layers and pooling operations are commonly employed to capture short-term temporal patterns, such as those related to heart rhythm. Pooling operations, in particular, play a crucial role in reducing the dimensionality of the data while preserving key features, such as the R-peak information. Unlike previous studies that focus on designing advanced and specialized architectures tailored for processing ECGs, our approach diverges by choosing commonly used FM architectures. We aim to investigate the effect of combining various pre-trained models, each built on different architecture types and learning strategies, and applying them across multiple-center ECG datasets. This approach allows us to explore the generalization capability of these FMs, particularly in terms of how well they adapt to diverse datasets and real-world variability. By evaluating the impact of different pre-training strategies on model performance across multiple centers, we seek to shed light on the robustness of these models and their potential for more generalized ECG signal processing tasks. II-B SSL-based pre-training strategies of ECG-FMs Unlike traditional supervised learning, which relies heavily on labeled datasets that can be costly and time-consuming to curate, SSL leverages the inherent structure of the data itself to generate meaningful training signals. This approach reduces the dependence on manual annotations and allows for the utilization of large-scale, real-world datasets that are often unlabeled. In NLP field, SSL techniques such as masked language modeling in BERT or autoregressive learning in GPT have demonstrated remarkable success in capturing semantic and syntactic nuances. In computer vision field, contrastive learning methods like SimCLR ( Chen 2020 ) and MoCo (