EchoJEPA: A Latent Predictive Foundation Model for Echocardiography

Alif Munim, Adibvafa Fallahpour, Teodora Szasz, Ahmadreza Attarpour, River Jiang, Brana Sooriyakanthan, Maala Sooriyakanthan, Heather Whitney, Jeremy Slivnick, Barry Rubin, Wendy Tsang, Bo Wang · 2026

Read EchoJEPA: A Latent Predictive Foundation Model for Echocardiography by Alif Munim, Adibvafa Fallahpour, Teodora Szasz, Ahmadreza Attarpour, River Jiang, Brana Sooriyakanthan, Maala Sooriyakanthan, Heather Whitney, Jeremy Slivnick, Barry Rubin, Wendy Tsang, Bo Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-02603.txt.

Introduction

Alif Munim and 11 co-authors pretrain a foundation model for heart ultrasound on eighteen million echocardiograms, in a 2026 arXiv paper. Ultrasound images are corrupted by speckle noise and acquisition artifacts that obscure anatomy. Models that chase pixels learn the noise along with the signal.

EchoJEPA instead uses a latent predictive objective, learning anatomical representations that ignore speckle. The corpus spans three hundred thousand patients, the largest yet for this modality. A multi-view probing framework with frozen backbones tests what the representations contain. Left ventricular ejection fraction estimates improved by about twenty percent over leading baselines, and right ventricular pressure estimates by about seventeen percent. With one percent of labeled data the model classified views far better than baselines with all of it. It also generalized to children without fine-tuning.

Catalog record

CREATORAlif Munim, Adibvafa Fallahpour, Teodora Szasz, Ahmadreza Attarpour, River Jiang, Brana Sooriyakanthan, Maala Sooriyakanthan, Heather Whitney, Jeremy Slivnick, Barry Rubin, Wendy Tsang, Bo Wang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-02603

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

EchoJEPA: A Latent Predictive Foundation Model for Echocardiography Alif Munim, Adibvafa Fallahpour, Teodora Szasz, Ahmadreza Attarpour, River Jiang, Brana Sooriyakanthan, Maala Sooriyakanthan, Heather Whitney, Jeremy Slivnick, Barry Rubin, Wendy Tsang, Bo Wang Abstract Foundation models for echocardiography often struggle to disentangle anatomical signal from the stochastic speckle and acquisition artifacts inherent to ultrasound. We present EchoJEPA, a foundation model trained on 18 million echocardiograms across 300K patients, representing the largest pretraining corpus for this modality to date. By leveraging a latent predictive objective, EchoJEPA learns robust anatomical representations that ignore speckle noise. We validate this using a novel multi-view probing framework with frozen backbones, where EchoJEPA outperforms leading baselines by approximately 20% in left ventricular ejection fraction (LVEF) estimation and 17% in right ventricular systolic pressure (RVSP) estimation. The model also exhibits remarkable sample efficiency, reaching 79% view classification accuracy with only 1% of labeled data versus 42% for the best baseline trained on 100%. Crucially, EchoJEPA demonstrates superior generalization, degrading by only 2% under physics-informed acoustic perturbations compared to 17% for competitors. Most remarkably, its zero-shot performance on pediatric patients surpasses fully fine-tuned baselines, establishing latent prediction as a superior paradigm for robust, generalizable medical AI. † † affiliationnotice: Equal contribution † Co-senior authors. 1 Introduction Echocardiography is the most widely used cardiac imaging modality, with approximately 30 million studies performed annually in the United States alone ( 37 ) . Its accessibility and lack of ionizing radiation make it the first-line tool for evaluating cardiac structure and function ( 20 ) . Recent efforts have sought to develop foundation models for echocardiography that learn generalizable representations from large unlabeled video archives, promising to reduce annotation burden and improve diagnostic consistency ( 18 ; 38 ; 1 ) . However, these models face challenges arising from the unique signal properties of ultrasound that distinguish it from natural video domains. Ultrasound video is dominated by stochastic speckle patterns, depth-dependent intensity attenuation, and acoustic shadows, artifacts that vary across acquisitions and bear no relationship to cardiac anatomy ( 5 ) . Existing foundation models use supervised multitask learning ( 18 ) , contrastive vision-language alignment ( 38 ) , or masked autoencoding ( 19 ; 16 ; 34 ) , yet none explicitly targets noise-invariant representations: supervised models inherit annotation noise, contrastive models align to report language rather than anatomy, and reconstruction models must faithfully reproduce speckle to minimize their loss. We demonstrate that latent prediction provides a superior objective. Joint-embedding predictive architectures (JEPA) train a predictor to infer embeddings of masked regions from visible context, targeting an exponential moving average teacher rather than raw pixels ( 4 ; 3 ; 7 ) . This downweights unpredictable artifacts like speckle while reinforcing temporally coherent structures like chamber geometry and wall motion. We introduce EchoJEPA , the first foundation-scale application of joint-embedding predictive architectures to echocardiography, pretrained on 18 million videos across 300K patients. EchoJEPA adapts V-JEPA2 with domain-appropriate temporal resolution and augmentation ( 3 ) . To handle variable study composition, we develop a multi-view probing framework with factorized video stream embeddings that integrates information across views without view-specific components, offering a superior alternative to prior multi-view embedding scheme ( 32 ) . Contributions. Our key contributions are as follows: • EchoJEPA. A foundation model using latent prediction pretrained on 18 million videos across 300K patients, the largest echocardiography corpus to date, achieving state-of-the-art performance on left ventricular ejection fraction (LVEF) estimation and right ventricular systolic pressure (RVSP) prediction, demonstrating that latent prediction outperforms pixel reconstruction for ultrasound. • Multi-view probing framework. A method using factorized video stream embeddings and attention masking to integrate information across echocardiographic views without view-specific components. • Unified evaluation protocol. A standardized benchmark with frozen backbones, identical probes, and consistent hyperparameter search across all baseline models, enabling fair comparison of representation quality. • Robustness benchmarks. Physics-informed perturbations using depth attenuation and acoustic shadow ( 30 ) , revealing that EchoJEPA degrades 86% less than the next-best baseline under acoustic perturbations. • Public release. We open-source EchoJEPA-L, a state-of-the-art echocardiography foundation model trained on MIMIC-IV-Echo ( 14 ) , alongside our evaluation framework at https://github.com/bowang-lab/EchoJEPA . 2 Related Work 2.1 Self-Supervised Video Learning Self-supervised video representation learning follows two dominant paradigms ( 15 ) . Reconstruction-based methods learn by imputing masked spatiotemporal content in pixel space. Masked Autoencoders ( 16 ) demonstrated that high masking ratios with lightweight decoders enable scalable pretraining, and VideoMAE ( 34 ) and ST-MAE ( 12 ) extended this to video by masking space-time tubelets. Prediction-based methods instead target learned representations rather than raw pixels. I-JEPA ( 2 ) predicts embeddings of masked image regions from visible context, while V-JEPA ( 4 ) and V-JEPA2 ( 3 ) extend this to video using an EMA target encoder for stable prediction targets. Notably, pixels reward fidelity to surface statistics while embeddings reward capture of semantically stable structure ( 24 ) . We test which paradigm suits domains where pixel-level fidelity is dominated by stochastic nuisance factors, a regime unexplored by prior work on natural video. 2.2 Foundation Models for Echocardiography Building on foundation model success in healthcare, echocardiography has followed suit ( 22 ; 23 ; 11 ) . EchoNet-Dynamic ( 26 ) released 10,030 annotated apical-4-chamber videos and established supervised baselines for ejection fraction estimation, and EchoNet-Pediatric ( 29 ) extended this to pediatric populations. Recent foundation models pursue broader task coverage through self-supervised pretraining. PanEcho ( 18 ) trains a ConvNeXt encoder with temporal transformer on over one million videos, achieving view-agnostic prediction through post-hoc averaging of clip-level outputs. EchoPrime ( 38 ) scales to 12 million video-report pairs with contrastive vision-language learning, using view classification and attention-based pooling for study-level predictions. Similarly, EchoCLIP ( 8 ) aligns visual representations with text reports via contrastive learning on 1 million video-text pairs. EchoFM applies masked autoencoding to 290,000 videos to learn spatiotemporal representations ( 19 ) . Distinct from diagnostic foundation models, EchoWorld ( 39 ) applies world modeling to robotic probe guidance, integrating motion-aware attention to predict visual dynamics conditional on 6-DOF probe pose data. In the low-data regime, Ellis et al. ( 10 ) utilized V-JEPA for segmentation on the small CAMUS dataset ( 21 ) , introducing auxiliary localization tasks to compensate for the lack of inductive bias in Vision Transformers ( 9 ) . Unlike these approaches, we scale latent prediction to 18 million videos to build a general-purpose diagnostic encoder without reliance on hardware pose data or auxiliary losses. 2.3 Robustness Evaluation in Medical Imaging Standard evaluation protocols emphasize i.i.d. test performance, underestimating failure modes in clinical deployment where distribution shift is ubiquitous ( 28 ) . Echocardiography exhibits acquisition variability since image quality depends on patient body habitus, probe positioning, operator expertise, and equipment vendor ( 27 ) . Patients most likely to benefit from automated analysis, such as those with obesity or limited acoustic windows, are those whose images deviate most from training distributions, yet prior robustness evaluations focus on ImageNet-C corruptions ( 17 ) or adversarial perturbations ( 13 ) , neither capturing ultrasound-specific degradation. Figure 1 : EchoJEPA architecture. Echocardiograms from multiple views are partitioned into spatio-temporal tubelets and split into masked and

Read the complete text