Rethinking Dense Optical Flow without Test-Time Scaling
Praroop Chanda, Suryansh Kumar, Visual, Spatial AI Lab, VCCM Section, College of PVFA, Department of ECEN, Department of CSCETexas A&M University, College Station, Texas, USA · 2026
Read Rethinking Dense Optical Flow without Test-Time Scaling by Praroop Chanda, Suryansh Kumar, Visual, Spatial AI Lab, VCCM Section, College of PVFA, Department of ECEN, Department of CSCETexas A&M University, College Station, Texas, USA (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-08000.txt.
Introduction
Praroop Chanda and Suryansh Kumar challenge the assumption that better optical flow needs more computation at test time. Recent leaders estimate the motion between frames with complex architectures and multi-step refinement, paying heavily in inference cost. The authors argue strong priors from foundation models can replace that iterative work.
Their framework estimates dense flow in a single forward pass. It fuses semantic features from a frozen DINO-v2 backbone with geometric cues from a monocular depth foundation model, then solves a global matching problem with no recurrent updates. Without any refinement it posted strong cross-dataset generalization. On the Sintel Final benchmark it clearly beat the leading refinement-based system under comparable training, along with several other established methods in the same setting.
Catalog record
| CREATOR | Praroop Chanda, Suryansh Kumar, Visual, Spatial AI Lab, VCCM Section, College of PVFA, Department of ECEN, Department of CSCETexas A&M University, College Station, Texas, USA |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.08000
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Rethinking Dense Optical Flow without Test-Time Scaling Praroop Chanda, Suryansh Kumar, Visual, Spatial AI Lab, VCCM Section, College of PVFA, Department of ECEN, Department of CSCETexas A&M University, College Station, Texas, USA Abstract Recent progress in dense optical flow has been driven by increasingly complex architectures and multi-step refinement for test-time scaling. While these approaches achieve strong benchmark performance, they also require substantial computation during inference. This raises a fundamental question: Is scaling test-time computation the only way to improve dense optical flow accuracy? We argue that it is not. Instead, powerful visual semantic and geometric priors encoded in modern foundation models can reduce, if not overcome, the need for computationally expensive iterative refinement at test-time. In this paper, we present a framework that estimates dense optical flow in a single forward pass, leveraging pretrained foundation representations, while avoiding iterative refinement and additional inference-time computation, thus offering an alternative to test-time scaling. Our method extracts visual semantic features from a frozen DINO-v2 backbone and combines them with geometric cues from a monocular depth foundation model. We fuse these complementary priors into a unified representation and apply a global matching formulation to estimate dense correspondences without recurrent updates or test-time optimization. Despite avoiding iterative refinement, our approach achieves strong cross-dataset generalization across challenging benchmarks. On Sintel Final, we obtain 2.81 EPE without refinement, significantly improving over state-of-the-art (SOTA) SEA-RAFT under comparable training conditions and outperforming RAFT, GMFlow (without refinement), and recent FlowSeek in the same setting. These results suggest that strong foundation priors can substitute for test-time scaling, offering a computationally efficient alternative to refinement-heavy pipelines. 1 Introduction Dense optical flow is a fundamental problem in computer vision, robotic perception, and machine vision 0 0 footnotetext: *Corresponding Author: Suryansh Kumar . The task seeks to estimate a optical flow vector for every pixel between two consecutive images ( 33 ) . Accurate estimation of optical flow enables a wide range of downstream applications, including video understanding ( 5 ) , saliency detection ( 20 ) , action recognition ( 29 ; 36 ) , dense 3D reconstruction ( 32 ; 16 ; 3 ; 15 ) , shape deformation modeling 17 ; 18 ; 19 ; 14 , and motion modeling ( 7 ) . Over the past decade, deep learning methods have dramatically improved optical flow accuracy. Architectures such as RAFT ( 37 ) and its variants ( 39 ) achieve near-saturating performance on standard benchmarks. However, these improvements come at an increasing computational cost. Modern pipelines rely heavily on large annotated datasets, extensive training schedules, and most critically multi-step test-time refinement, which requires substantial inference-time compute. This trend naturally raises a fundamental question: Is allocating more computation at time-time the only way forward to improve optical flow performance? Recent developments in foundation models for visual representation suggest an alternative viewpoint. Large-scale pretrained models trained once on internet-scale datasets have demonstrated strong transferability across diverse visual tasks, including object detection ( 45 ) , segmentation ( 44 ) , depth estimation ( 42 ; 43 ) , and geometric reasoning ( 42 ; 23 ) . Rather than repeatedly training specialized architectures or increasing inference-time computation, these models show that powerful pretrained representations can generalize across domains and tasks. This observation raises an intriguing possibility for dense optical flow, i.e., instead of relying on increasingly expensive test-time refinement unlike SOTA 39 ; 30 , we can leverage strong visual semantic and geometric priors from foundation models to perform accurate motion estimation directly. Surprisingly, dense optical flow has remained largely disconnected from such paradigm despite the rapid progress of foundation models in other vision domains. Current state-of-the-art approaches ( 39 ; 37 ; 30 ) continue to emphasize architectural innovations, dataset scaling, and iterative update mechanisms, treating optical flow as a task that must be explicitly learned and repeatedly corrected during inference. While these strategies improve benchmark performance, they also reinforce the assumption that accurate optical flow estimation requires task-specific feature encoders and expensive refinement procedures. Nevertheless, as we know, dense optical flow is fundamentally a correspondence problem, where the key challenge is to learn representations that reliably match pixels across frames while respecting scene structure and motion boundaries. Modern vision foundation models already encode many of these properties, including strong semantic discrimination and boundary-aware geometric cues. This observation suggests that dense optical flow may benefit more from reusing powerful pretrained representations than from further increasing architectural complexity or test-time computation. In this work, we revisit the dense optical flow problem from a foundation model perspective. We argue that strong semantic and geometric priors contained in modern pretrained models can significantly reduce the reliance on test-time scaling through iterative refinement. Instead of allocating additional computation during inference, we investigate whether carefully designed representations can enable accurate motion estimation in a single forward pass. Our central hypothesis is that dense optical flow can emerge from sufficiently powerful visual semantic and geometric representations without repeated correction through iterative updates. Concretely, we introduce a dense optical flow framework that leverages two complementary foundation priors. First, we extract semantic visual features from the self-supervised DINO-v2 model ( 27 ) , which provides spatially coherent representations capable of capturing fine-grained visual correspondence. Second, we incorporate geometric structure using a monocular depth foundation model ( 43 ) . Although depth prediction is learned from data rather than derived from explicit geometry, modern depth prediction models produce high-frequency structural cues and sharp boundaries that are particularly informative for correspondence estimation near occlusions and motion discontinuities. By combining these semantic and geometric signals, we construct a unified representation suitable for dense matching. The proposed framework fuses DINO-v2 features and depth foundation representations within a global matching formulation inspired by GMFlow ( 41 ) . Unlike RAFT ( 37 ) , SEA-RAFT ( 39 ) , and very recently proposed FlowSeek ( 30 ) , our model performs optical flow estimation in a single forward pass and does not require iterative refinement to achieve competitive performance. Empirically, our approach demonstrates strong cross-dataset generalization across challenging benchmarks. On the Sintel Final benchmark, we achieve 2.81 EPE without refinement, substantially improving over SEA-RAFT (4.32 EPE) under comparable training conditions and outperforming RAFT, GMFlow (without refinement), and FlowSeek in the same setting. These results suggest that strong foundation priors can partially substitute for test-time scaling in dense optical flow estimation. More broadly, our work highlights the potential of foundation-model-driven inference pipelines to reduce the reliance on refinement-heavy architectures and large task-specific training pipelines. Our contributions are summarized as follows: • We propose a dense optical flow estimation framework that operates without test-time refinement and leverages frozen pretrained foundation models instead of training task-specific encoders. • We show that combining DINO-v2 visual semantic features ( 27 ) with depth foundation representations ( 43 ) provides a powerful prior for dense optical flow correspondence estimation. • Our work demonstrate that strong pretrained representations can reduce reliance on iterative inference procedures. Empirically, our framework achieves competitive and often superior performance compared to refinement-based SOTA approaches such as RAFT ( 37 ) , GMFlow ( 41 ) , SEA-RAFT ( 39 ) , and FlowSeek ( 30 ) , including 2.81 EPE on Sintel Final without refinement.