SyncFix: Fixing 3D Reconstructions via Multi-View Synchronization

Deming Li 1 Abhay Yadav 1 Cheng Peng 2, Rama Chellappa 1 Anand Bhattad 1 · 2026

Read SyncFix: Fixing 3D Reconstructions via Multi-View Synchronization by Deming Li 1 Abhay Yadav 1 Cheng Peng 2, Rama Chellappa 1 Anand Bhattad 1 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-11797.txt.

Introduction

Li and four colleagues introduce SyncFix, a method that repairs three-dimensional reconstructions by keeping multiple views consistent while a diffusion model refines them. It treats refinement as joint latent bridge matching, syncing distorted and clean views through the denoising process. Training uses image pairs alone, yet inference handles any number of views, with quality rising as views are added, and it beats current baselines even without clean reference images.

Catalog record

CREATORDeming Li 1 Abhay Yadav 1 Cheng Peng 2, Rama Chellappa 1 Anand Bhattad 1
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43067
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.11797

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SyncFix: Fixing 3D Reconstructions via Multi-View Synchronization Deming Li 1 Abhay Yadav 1 Cheng Peng 2, Rama Chellappa 1 Anand Bhattad 1 Abstract We present SyncFix , a framework that enforces cross-view consistency during the diffusion-based refinement of reconstructed scenes. SyncFix formulates refinement as a joint latent bridge matching problem, synchronizing distorted and clean representations across multiple views to fix the semantic and geometric inconsistencies. This means SyncFix learns a joint conditional over multiple views to enforce consistency throughout the denoising trajectory. Our training is done only on image pairs, but it generalizes naturally to an arbitrary number of views during inference. Moreover, reconstruction quality improves with additional views, with diminishing returns at higher view counts. Qualitative and quantitative results demonstrate that SyncFix consistently generates high-quality reconstructions and surpasses current state-of-the-art baselines, even in the absence of clean reference images. SyncFix achieves even higher fidelity when sparse references are available. View 1 View 2 View 3 View 2 View 3 3DGS Difix3D+ Ours GT Figure 1: Independent diffusion refinement (DifiX3D+) processes each view separately, leading to inconsistent geometry across views (see close-view of table in the bottom panel and highlighted regions above). SyncFix (Ours) instead refines all views jointly, enforcing cross-view agreement during denoising and producing stable 3D structure. 1 Introduction Radiance fields [ 21 ] and 3D Gaussian splatting [ 10 ] under sparse-view or off-trajectory settings often produce scenes plagued by floaters, texture artifacts, and geometric distortions. Recent approaches attempt to mitigate these issues by leveraging the rich priors of 2D generative models [ 35 , 37 ] . Typically, these methods render novel views, refine each image using a diffusion model conditioned on a reference view, and distill the refined outputs back into the 3D representation. However, these refinement methods treat each view as an independent 2D generation task. This ignores important cross-view geometric and semantic constraints and that all views are projections of the same underlying 3D scene. Therefore, independent refinement can produce inconsistent results for ambiguous structures or regions with large artifacts across different viewpoints. When these conflicting signals are distilled back into the 3D model, optimization becomes unstable, leading to appearance drift and degraded geometry. In this paper, we impose multi-view consistency directly during the refinement process rather than correcting it post hoc. To this end, we introduce SyncFix , a latent bridge matching formulation that models refinement as joint conditional generation over multiple views. SyncFix aligns latent representations across views during the denoising process and encourages information exchange via cross-view attention without any explicit camera information. Each view is therefore refined in the context of all others, ensuring consistent semantics and geometry prior to 3D distillation. During training, we train to refine only two views simultaneously, but its permutation-invariant latent interactions using attention naturally extend seamlessly to arbitrary numbers of views at inference. As more views are provided, the cross-view constraints become stricter and the solution space narrows, which leads to systematically better final reconstructions. We also find that SyncFix produces consistent multi-view outputs even when no clean reference frames are available, and it can further benefit from sparse references when they exist. We substantiate these claims using perceptual metrics and through analyses of geometric alignment and semantic correspondence. 2 Related Work Sparse-View 3D Reconstruction. NeRF [ 21 ] and 3DGS [ 10 ] produce high-fidelity geometry from dense captures. With sparse inputs the problem becomes ill-posed: reconstructions develop floaters, fragmented surfaces, and blurred textures. Regularization-based approaches constrain frequency content [ 22 , 40 , 30 ] or inject depth priors [ 5 , 39 ] . Feed-forward Gaussian splatting methods [ 9 , 41 ] leverage learned multi-view encoders [ 32 ] for direct prediction. These methods reduce artifacts but do not eliminate them; the residual degradation motivates generative refinement. Per-View Refinement with 2D Priors. The dominant repair paradigm renders novel views, refines each independently with a 2D diffusion model, and distills the results back into the 3D representation. This follows the Score Distillation Sampling framework [ 31 , 25 ] , originally developed for text-to-3D generation [ 11 , 33 ] and since adapted for reconstruction repair. Nerfbusters [ 35 ] applies diffusion guidance during NeRF optimization. ReconFusion [ 38 ] trains a multi-view conditioned prior for few-view regularization. Instruct-NeRF2NeRF [ 8 ] uses SDEdit [ 20 ] on individual renderings. Difix3D+ [ 37 ] distills a single-step refinement filter and adds limited cross-view conditioning through reference-view cross-attention. FreeFix [ 43 ] manipulates latent trajectories without retraining. These methods ultimately optimize per-view reconstruction objectives, even when limited cross-view signals are introduced. The 2D prior never sees multiple views simultaneously, so it may interpret the same artifact differently from each angle. The 3D distillation stage must reconcile the resulting conflicts. Coupling Views inside the Generator. A natural fix is to condition generation on multiple views at once. Video diffusion models provide implicit coupling through temporal attention: 3DGS-Enhancer [ 16 ] casts view consistency as temporal coherence and refines 3DGS with a confidence-aware loop; ReconX [ 15 ] conditions video diffusion on a global point cloud for sparse-view reconstruction. Multi-view diffusion architectures couple views more directly. MVDream [ 28 ] , SyncDreamer [ 18 ] , CAT3D [ 7 ] , and ViewCrafter [ 42 ] synchronize attention across generated views. 3DEnhancer [ 19 ] applies this idea to 3D enhancement, using multi-view row attention and epipolar aggregation within a latent diffusion model. These methods demonstrate that coupling views improves coherence. However, they remain within the denoising diffusion framework: views interact only through attention over noisy intermediate states, and sampling requires iterative denoising. Flow-Based Multi-View Refinement. Rectified Flow [ 17 ] and Flow Matching [ 14 ] replace iterative denoising with learned ODEs that transport between distributions. Latent Bridge Matching (LBM) [ 4 ] extends this to conditional generation via deterministic bridges between paired latents. SyncLight [ 26 ] adds cross-view attention to LBM for consistent relighting. SyncFix builds on this foundation but targets a different problem: repairing degraded 3D geometry. We learn a joint flow over the product latent space of multiple distorted renderings and their clean counterparts. Where attention-based diffusion models such as 3DGS-Enhancer couple views implicitly through shared noise states, SyncFix couples them with deterministic latent bridge matching, enforcing semantic and geometric agreement before any image is decoded. Thus, SyncFix is a single-pass, intrinsically consistent multi-view refinement without iterative denoising. 3 Method Our goal is to refine 3D reconstructions while preserving multi-view consistency. Existing approaches refine each rendered view independently and rely on the 3D representation to harmonize conflicting corrections. This harmonization is unreliable. For example, one view may introduce high-frequency structures (e.g., a painting on a wall) while another suppresses them, leading to inconsistent geometry and texture when fused back into the 3D representation. We instead model refinement as a joint conditional generation problem over multiple views. Let a distorted 3D reconstruction produce N rendered views X_{D}=\{x_{D}^{(1)},\dots,\allowbreak x_{D}^{(N)}\} , with corresponding clean targets X_{GT}=\{x_{GT}^{(1)},\dots,x_{GT}^{(N)}\} . Marginal methods like Difix3D+ [ 37 ] learn independent mappings P(x_{GT}^{(i)}\mid x_{D}^{(i)}) . SyncFix instead models the joint conditional P(X_{GT}\mid X_{D}) and enforces cross-view consensus during generation. This shift from marginal to joint modeling is central: corrections for each view are conditioned on all

Read the complete text