SADGE: Structure and Appearance Domain Gap Estimation of Synthetic and Real Data

Patryk Bartkowiak, Bartosz Kottrys, Dominik Michels, Soren Pirk, Wojtek Palubicki · 2026

Read SADGE: Structure and Appearance Domain Gap Estimation of Synthetic and Real Data by Patryk Bartkowiak, Bartosz Kottrys, Dominik Michels, Soren Pirk, Wojtek Palubicki (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-22467.txt.

Introduction

Bartkowiak and colleagues propose SADGE, a score predicting synthetic image performance on real data, without later training. Current scores mostly measure meaning match between real and synthetic images. Shape match is considered less often. Neither looks nor shape alone predicts later performance; their interplay dictates synthetic data utility. SADGE is tested across five benchmark families and 15 variants, covering 79,000 image pairs. It achieves the strongest later association under linear and rank criteria, at Pearson r 0.88 and Spearman rho 0.77. Fusing DINOv3 looks with MASt3R shape wins best.

Catalog record

CREATORPatryk Bartkowiak, Bartosz Kottrys, Dominik Michels, Soren Pirk, Wojtek Palubicki
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE58523
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.22467

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SADGE: Structure and Appearance Domain Gap Estimation of Synthetic and Real Data Patryk Bartkowiak, Bartosz Kottrys, Dominik Michels, Soren Pirk, Wojtek Palubicki Abstract We propose SADGE, a quantitative similarity metric that predicts the performance of synthetic image datasets for common computer vision tasks without downstream model training. Estimating whether a synthetic dataset will lead to a model that performs well on real-world data remains a bottleneck in model development. Existing evaluation metrics (e.g., PSNR, FID, CLIP) primarily measure semantic alignment between real and synthetic images (Appearance Similarity Score). Less commonly, structural similarity between images is considered to assess the domain gap (Geometric Similarity Score). However, to the best of our knowledge there exists no studies that evaluate which similarity metric is the best downstream predictor for a given synthetic dataset. In this paper, we show over a wide variety of different synthetic datasets and downstream tasks that neither appearance nor geometry alone can reliably predict downstream performance; rather, it is their non-linear interplay that dictates synthetic data utility. Specifically, we measure how commonly used Appearance and Geometric Similarity metrics (e.g., CLIP, PSNR, LPIPS, SSIM) computed between synthetic and real images correlate with downstream performance in object detection, semantic segmentation, and pose estimation. Across five public synthetic-to-real benchmark families and 15 dataset-level variants (79k image pairs), SADGE achieves the strongest association with downstream transfer performance under both linear and rank-based criteria, reaching Pearson r = 0.879 r=0.879 and Spearman ρ = 0.768 \rho=0.768 ( n = 15 n=15 , approximate p = 8.3 × 10 − 4 p=8.3\times 10^{-4} ). We compute for each combination of geometry-based methods (SSIM, SuperPoints, MASt3R, LoFTR) and appearance-based approaches (FID, DINOv2, DINOv3, SigLIP2, SAM3, PSNR, CLIP, LPIPS) SADGE scores across all benchmark families. The best configuration is obtained by fusing DINOv3 appearance similarity with MASt3R geometric consistency through a constrained bilinear interaction, outperforming both the strongest geometry-only baseline (LoFTR, ρ = 0.582 \rho=0.582 ) and the strongest appearance-only baseline (PSNR, ρ = 0.536 \rho=0.536 ). Figure 1: SADGE predicts the utility of a synthetic image dataset for downstream visual recognition by jointly modeling appearance similarity and geometry consistency between real and synthetic images. For each real image, comparison is performed either using an aligned real–synthetic pair or by retrieving the best synthetic match from a candidate subset. After dataset-level aggregation, the appearance and geometry scores are fused with a constrained bilinear interaction model to produce the final SADGE score. 1 Introduction Synthetic data is widely used to scale vision systems when real annotations are expensive or difficult to obtain, particularly for edge cases 36 ; 25 . Yet a fundamental problem remains unresolved: given a synthetic dataset, can we predict – before downstream training – whether it will improve real-world performance, or instead encode biases that fail to transfer? This is a timely question because evaluating synthetic data is still largely a trial-and-error process. Practitioners typically generate candidate datasets, train models, and measure performance on held-out real data to determine whether a rendering pipeline, domain-randomization strategy, or generative process is effective. In industrial and safety-critical settings, this loop is particularly costly: seemingly minor changes in illumination, background, material appearance, or object placement can substantially affect transfer performance 10 ; 50 ; 51 ; 16 . A reliable pre-training estimate of synthetic-data usefulness would therefore make synthetic pipeline design significantly more principled and efficient. With reliable pre-training metrics, users could compare candidate rendering configurations, domain-randomization schedules, rendering settings, asset libraries, filtering strategies, and graphics-based versus generative synthetic sources, and then only train the most promising dataset candidates. Recently, the need for such an estimate has only grown as synthetic data generation has diversified. Classical graphics-based pipelines render large datasets from simulators and CAD assets 12 ; 26 ; 18 ; 32 , offering explicit control over geometry, camera pose, lighting, materials, and sensor effects. In parallel, image-generative models synthesize training data by sampling from learned image distributions 49 ; 33 , often increasing semantic and stylistic diversity but without explicit guarantees of geometric consistency. Currently, common proxy metrics such as CLIP similarity, FID, LPIPS, PSNR, or DINO embeddings estimate the synthetic-to-real gap 23 ; 46 ; 20 . Specialized, training-free metrics such as CLER 23 have been introduced that employ CLIP-derived, class-centered representations to predict datset relevance limited to simpler classification tasks. In practice, these metrics are often treated with the assumption that appearance alignment is a strong estimator of downstream transfer quality. However, there is still no evidence how strongly such similarity metrics correlate with downstream task performance across a significant amount of synthetic datasets and tasks. Furthermore, it remains unclear which properties of a synthetic dataset are actually predictive of real-world task accuracy across different domains settings. In this work, we argue that synthetic-data usefulness is not captured by appearance alone, but by jointly considering it with image structure properties, i.e. geometry. In fact, to our knowledge, geometry has been entirely neglected for assessing synthetic data fidelity. We therefore introduce SADGE ( Structural and Appearance Domain Gap Estimator ), a zero-shot metric for estimating synthetic-data usefulness without training a downstream task model on the dataset being evaluated. SADGE combines appearance with geometric similarity, aggregates them at the dataset level, and fuses them into a single score designed to track real-world performance. The key idea is that appearance scores measure whether synthetic images lie near the target domain, while geometric scores measure whether they preserve structurally meaningful relationships. We show that these signals are complementary: neither appearance-only nor geometry-only metrics consistently predict transfer performance on their own, but their combination yields a substantially stronger and more stable predictor, significantly outperforming all commonly used metrics. Figure 2: Pearson correlation with downstream performance on all datasets (top-left panel) and leave-one-dataset-out evaluations (remaining panels: excluding ASD/agricultural, DIMO, RarePlanes, TUD-L, and VKITTI2). Bar colors denote metric families: green is SADGE (ours), red is structure-oriented metrics (MASt3R, LoFTR, SuperPoint+LightGlue, and SSIM), and blue is appearance-oriented metrics (LPIPS, PSNR, CLIP, SigLIP, DINOv2/DINOv3, SAM3, and FID). SADGE ranks first in every panel (overall r=0.879 ; leave-one-out r\in[0.637,0.907] ). Excluding DIMO causes the largest drop and shifts the strongest baselines toward appearance metrics, while splits that retain DIMO show more competitive geometry-metric correlations. We evaluate SADGE on benchmarks that isolate common synthetic-data failure modes: illumination shifts in DIMO 7 , weather and appearance variation in Virtual KITTI2 4 , industrial object rendering for 6D pose estimation in TUD-L 14 ; 15 , aerial synthetic-to-real transfer in RarePlanes 37 , and agricultural synthetic-to-real transfer in the Agricultural Synthetic Dataset (ASD) 5 . Across these settings, SADGE achieves a strong Pearson correlation of r=0.879 with downstream task performance such as for pose estimation, semantic segmentation, and object detection, significantly outperforming widely used similarity metrics and providing a more reliable basis for ranking candidate synthetic datasets before expensive training. In summary, our contributions are: (1) we introduce SADGE, a pre-training metric for synthetic-data utility that jointly models appearance and geometry similarity; (2) we propose a unified evaluation protocol for aligned and retrieval-based real–synthetic matching, enabling assessment across paired and unpaired datasets; (3) we conduct a large-scale correlation study across DIMO, VKITTI2, RarePlanes, TUD-L, and ASD, showing that common appearance-only or geometry-only metrics are typically moderate predictors,

Read the complete text