Spark3R: Asymmetric Token Reduction Makes Fast Feed-Forward 3D Reconstruction

Zecheng Tang, Jiaye Fu, Qiankun Gao, Haijie Li, Yanmin Wu, Jiaqi Zhang, Siwei Ma, Jian Zhang · 2026

Read Spark3R: Asymmetric Token Reduction Makes Fast Feed-Forward 3D Reconstruction by Zecheng Tang, Jiaye Fu, Qiankun Gao, Haijie Li, Yanmin Wu, Jiaqi Zhang, Siwei Ma, Jian Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-06270.txt.

Introduction

Zecheng Tang and seven co-authors speed up feed forward 3D reconstruction from images with asymmetric token reduction. Transformer based models estimate scene geometry and camera poses directly, but video inputs of hundreds or thousands of frames make global attention quadratically expensive. Token merging helps, yet uniform compression ignores that query and key value tokens play different roles in reconstruction.

The authors show query tokens carry view specific geometric requests and resist compression, while key value tokens hold shared scene context and tolerate deep cuts. Spark3R reduces the two kinds separately, merging within query groups and pruning key value tokens, with the rate adjusted per layer. It needs no retraining and plugs into several pretrained models. On thousand frame inputs it reached speedups up to twenty eight times.

Catalog record

CREATORZecheng Tang, Jiaye Fu, Qiankun Gao, Haijie Li, Yanmin Wu, Jiaqi Zhang, Siwei Ma, Jian Zhang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52 KB
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2605.06270

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Spark3R: Asymmetric Token Reduction Makes Fast Feed-Forward 3D Reconstruction Zecheng Tang, Jiaye Fu, Qiankun Gao, Haijie Li, Yanmin Wu, Jiaqi Zhang, Siwei Ma, Jian Zhang Abstract Feed-forward 3D reconstruction models based on Vision Transformers can directly estimate scene geometry and camera poses from a small set of input images, but scaling them to video inputs with hundreds or thousands of frames remains challenging due to the quadratic cost of global attention layers. Recent token-merging methods accelerate these models by compressing the token sequence within the global attention layers, but they apply a uniform reduction to query tokens and key-value tokens, ignoring their functionally distinct roles in 3D reconstruction. In this work, we identify a key property of feed-forward 3D reconstruction models: query tokens encode view-specific geometric requests and are sensitive to compression, while key-value tokens represent shared scene context and tolerate aggressive compression. Guided by this insight, we propose Spark3R, a training-free acceleration framework that decouples the compression of query tokens and key-value tokens by assigning distinct reduction factors, with intra-group token merging applied to query tokens and lightweight token pruning to key-value tokens. Additionally, Spark3R adaptively adjusts the key-value reduction factor across layers, further improving the quality–efficiency trade-off. As a plug-and-play framework requiring no retraining, Spark3R integrates directly into multiple pretrained feed-forward 3D reconstruction models, including VGGT, 𝝅 𝟑 \bm{\pi^{3}} , Depth-Anything-3, and VGGT- 𝛀 \bm{\Omega} , and achieves up to 𝟐𝟖 × \mathbf{28\times} speedup on 1,000-frame inputs while maintaining competitive reconstruction quality. I Introduction Recovering 3D geometry from multi-view images has traditionally relied on per-scene optimization pipelines such as Structure-from-Motion (SfM) [ 23 , 28 ] and Multi-View Stereo (MVS) [ 24 , 41 ] . More recent neural scene representations, including NeRF [ 19 ] and 3D Gaussian Splatting [ 15 ] , retain this per-scene paradigm and still demand substantial computational overhead. Unlike per-scene optimization approaches [ 18 , 12 , 11 ] , feed-forward 3D reconstruction recovers dense scene geometry and camera poses directly from a set of unposed images in a single forward pass, with broad applications in autonomous driving [ 33 ] , robotics [ 40 ] , and multimodal 3D scene understanding [ 37 , 39 ] . State-of-the-art models such as VGGT [ 31 ] , \pi^{3} [ 36 ] , Depth-Anything-3 [ 17 ] , and VGGT- \Omega [ 32 ] achieve this by processing all input frames jointly through global attention layers. While this joint processing yields accurate geometric consistency across an arbitrary number of frames, the global attention layers impose costs quadratic in the frame count, making it prohibitively expensive to scale to video-length inputs of hundreds or thousands of frames. Streaming methods [ 34 , 5 , 42 ] sidestep this cost through sequential processing but forgo joint cross-frame reasoning, often degrading reconstruction quality. Chunk-and-align strategies [ 8 , 9 ] partition the input into fixed-size chunks, confining the model’s own cross-frame reasoning to within each chunk and relying on post-hoc alignment of overlapping frames to recover global consistency. Both lines of work gain efficiency by restricting which frames can interact, rather than accelerating the global attention layers themselves. Rather than restricting which frames can interact, token-merging methods [ 25 , 27 ] offer a more direct strategy: compressing the sequence length within global attention layers so that all frames remain visible to one another at reduced cost. However, these methods apply a single, uniform reduction factor to all token roles, limiting speedups to roughly 10\times at best before reconstruction quality degrades noticeably. We hypothesize that this ceiling stems from ignoring the functionally distinct roles of query tokens and key-value tokens in global attention layers. To test this hypothesis, we separately compress query tokens, key-value tokens, and both jointly at increasing reduction factors (the ratio of the original sequence length to the reduced sequence length) and measure the resulting pose error on VGGT. As shown in Figure 1 , compressing key-value tokens alone keeps the error relatively flat, whereas compressing query tokens alone causes a sharp rise. Joint uniform compression, the strategy adopted by existing token-merging methods, yields the steepest curve of all: it inherits the full sensitivity of query tokens while failing to exploit the high compressibility of the key-value tokens. This pronounced asymmetry reveals substantial untapped headroom that uniform methods leave on the table. We provide complementary analytical explanations for this gap in Sec. IV-A . Fig. 1: Compression sensitivity of different token roles in VGGT. We separately compress query tokens (orange), key-value tokens (blue), and both jointly (red) at increasing reduction factors and report pose error (ATE \downarrow ). Key-value tokens tolerate aggressive compression with negligible quality loss, while query tokens degrade sharply beyond a reduction factor of 12. Joint uniform compression yields the steepest curve. Bottom : point-cloud reconstructions at reduction factor 16. Guided by this insight, we present Spark3R , a training-free, plug-and-play acceleration framework that decouples the compression of query tokens and key-value tokens. Our main contributions are: • We propose asymmetric token reduction for feed-forward 3D reconstruction models, which assigns separate reduction factors and distinct compression operators to query tokens and key-value tokens: intra-group token merging for query tokens and lightweight token pruning for key-value tokens. • We introduce a layer-adaptive key-value reduction schedule that assigns heavier compression to empirically insensitive layers, further improving the quality–efficiency trade-off. • We demonstrate that the resulting framework, Spark3R , integrates directly into VGGT, \pi^{3} , Depth-Anything-3, and VGGT- \Omega without retraining , achieving up to \mathbf{28\times} speedup on 1,000-frame inputs while maintaining competitive reconstruction quality. II Related Work II-A Feed-Forward 3D Reconstruction Feed-forward 3D reconstruction has emerged as a scalable alternative to per-scene optimization methods such as NeRF [ 19 ] and 3D Gaussian Splatting [ 15 ] . Early approaches such as DUSt3R [ 35 ] and MASt3R [ 16 ] predicted pairwise 3D point maps and fused them through post-hoc global alignment. VGGT [ 31 ] introduced an Alternating Attention backbone that interleaves global cross-view self-attention with frame-wise self-attention, enabling joint prediction of depth, poses, and point clouds in a single forward pass. \pi^{3} [ 36 ] proposed a fully permutation-equivariant architecture that eliminates dependence on a fixed reference view, and Depth-Anything-3 [ 17 ] showed that a plain Vision Transformer backbone paired with a unified depth-ray prediction target suffices for state-of-the-art geometry reconstruction. VGGT- \Omega [ 32 ] further scaled this paradigm to billions of parameters and millions of sequences, achieving state-of-the-art results on both static and dynamic benchmarks. To make such scaling tractable, it replaces a fraction of the global attention layers with register attention , in which cross-frame interaction happens only among a small set of per-frame register tokens rather than among all image tokens. Despite these architectural advances, all of the above models still rely on attention layers whose cost grows quadratically with the total token count, creating a fundamental scalability bottleneck as the number of input frames grows. II-B Streaming 3D Reconstruction Streaming methods address the quadratic scaling cost of global attention layers by processing frames incrementally. Recurrent-state approaches such as CUT3R [ 34 ] and TTT3R [ 5 ] maintain a compact state updated via cross-attention with each incoming frame, yielding constant per-step cost but

Read the complete text