DA-Flow: Degradation-Aware Optical Flow Estimation with Diffusion Models

Jaewon Min, Jaeeun Lee, Yeji Choi, Paul Hyunbin Cho, Jin Hyeon Kim, Tae-Young Lee, Jongsik Ahn, Hwayeong Lee, Seonghyun Park, Seungryong Kim † · 2026

Read DA-Flow: Degradation-Aware Optical Flow Estimation with Diffusion Models by Jaewon Min, Jaeeun Lee, Yeji Choi, Paul Hyunbin Cho, Jin Hyeon Kim, Tae-Young Lee, Jongsik Ahn, Hwayeong Lee, Seonghyun Park, Seungryong Kim † (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-23499.txt.

Introduction

Min and nine co-authors tackle optical flow in corrupted real-world video. Flow models trained on clean data break down under blur, noise, and compression damage. Their fix borrows features from image restoration diffusion models. Those features sense corruption well but ignore time. The team adds attention across nearby frames, which reveals a knack for matching points. DA-Flow fuses those features with plain convolutional ones in a refining loop. Under heavy damage, it beats current flow methods across several benchmarks by a wide gap.

Catalog record

CREATORJaewon Min, Jaeeun Lee, Yeji Choi, Paul Hyunbin Cho, Jin Hyeon Kim, Tae-Young Lee, Jongsik Ahn, Hwayeong Lee, Seonghyun Park, Seungryong Kim †
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52211
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.23499

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DA-Flow: Degradation-Aware Optical Flow Estimation with Diffusion Models Jaewon Min, Jaeeun Lee, Yeji Choi, Paul Hyunbin Cho, Jin Hyeon Kim, Tae-Young Lee, Jongsik Ahn, Hwayeong Lee, Seonghyun Park, Seungryong Kim † Abstract Optical flow models trained on high-quality data often degrade severely when confronted with real-world corruptions such as blur, noise, and compression artifacts. To overcome this limitation, we formulate Degradation-Aware Optical Flow , a new task targeting accurate dense correspondence estimation from real-world corrupted videos. Our key insight is that the intermediate representations of image restoration diffusion models are inherently corruption-aware but lack temporal awareness. To address this limitation, we lift the model to attend across adjacent frames via full spatio-temporal attention, and empirically demonstrate that the resulting features exhibit zero-shot correspondence capabilities. Based on this finding, we present DA-Flow , a hybrid architecture that fuses these diffusion features with convolutional features within an iterative refinement framework. DA-Flow substantially outperforms existing optical flow methods under severe degradation across multiple benchmarks. † † footnotetext: † : Corresponding author 1 Introduction Optical flow estimation, the task of estimating per-pixel motion fields between consecutive video frames, is a fundamental dense correspondence problem in computer vision. With the advent of deep neural networks [ 31 , 37 , 25 ] , recent methods have achieved remarkable accuracy. Real-world videos are rarely clean; motion blur, sensor noise, compression artifacts, and low resolution frequently co-exist, severely degrading visual quality. Despite the prevalence of such degradations, how optical flow models behave under such degradations remains largely unexplored. Recently, RobustSpring [ 28 ] provided the first comprehensive study on the robustness of dense matching models, benchmarking their generalization from clean synthetic training data to a wide spectrum of real-world degradations. Despite this systematic analysis, a fundamental question remains open: is it truly impossible to accurately estimate optical flow from corrupted inputs? Motivated by this question, we shift the focus from robustness to accuracy by introducing a new task, Degradation-Aware Optical Flow , that directly estimates dense correspondences from severely degraded inputs. This task is fundamentally ill-posed: degradations destroy fine textures and attenuate motion boundaries, leaving insufficient visual evidence for reliable matching. In such regimes, correspondence estimation is not merely a matter of distribution shift but becomes inherently ambiguous. Simply augmenting clean training data with synthetic corruptions does not adequately address this challenge; what is needed are representations that are both rich enough to preserve spatial structure for dense matching and sensitive to degradation patterns to recover information lost during corruption, as illustrated in Fig. 1 . Recent works [ 22 , 41 , 30 , 23 , 12 , 13 , 14 , 32 ] have shown that intermediate features of diffusion models encode rich structural and semantic information, achieving strong performance on correspondence tasks [ 22 , 41 , 30 , 23 ] as well as diverse downstream vision tasks such as depth estimation [ 12 , 13 ] and segmentation [ 14 , 32 ] . These findings suggest that diffusion representations capture geometric and structural cues far beyond what is needed for generation alone. Building on this insight, we observe that diffusion models trained for image restoration [ 17 , 5 , 1 , 8 ] offer an even more suitable foundation. Image restoration is likewise a highly underdetermined inverse problem, and models trained for this task must learn to recover clean structures from degraded inputs. As a result, their intermediate features naturally encode degradation patterns while preserving underlying scene geometry. This motivates our core design choice: leveraging restoration diffusion features to obtain representations that are degradation-aware, structurally rich for dense matching, and equipped with generative priors that can reason beyond corrupted observations. However, these features lack temporal awareness, limiting their effectiveness in producing accurate features for optical flow estimation. Figure 1 : Qualitative comparison of DA-Flow with baselines [ 31 , 37 , 25 ] on Spring benchmark [ 20 ] . Under severe degradations, existing optical flow methods fail to recover reliable correspondences, whereas our DA-Flow accurately estimates the underlying motion. Since our task involves video, a natural consideration is whether video restoration diffusion models [ 39 , 6 , 43 ] , which jointly model degradation and temporal dynamics, could serve as the backbone. However, such models often encode a stack of degraded frames into a temporally compressed latent representation through 3D convolutions or temporal attention. This produces a shared latent tensor where the temporal axis is entangled early in the encoding pipeline. While this design suits perceptual video restoration, where temporal smoothness and global consistency are desirable, it is structurally misaligned with dense correspondence estimation. Optical flow requires comparing spatial features extracted independently from each frame to establish pixel-level correspondences. When degraded frames are jointly encoded into a shared spatio-temporal latent space, their per-frame spatial structure is no longer preserved as separable entities, making the representation ill-suited for explicit pairwise feature matching. To reconcile degradation-aware representation with the structural requirements of dense matching, we take a different approach: instead of adopting a monolithic video diffusion backbone, we start from a pretrained image restoration diffusion model [ 8 ] that preserves full spatial resolution at the frame level. We then lift it to handle multiple frames by injecting cross-frame attention across all layers. This design maintains independent spatial latents for each frame, which is crucial for dense matching, while enabling controlled temporal interaction for motion reasoning. By inheriting strong degradation-aware priors from image restoration pretraining and avoiding temporal latent collapse, our architecture yields representations intrinsically suited for dense correspondence estimation under severe corruption, while remaining substantially more efficient than video diffusion architectures. Building on this representation, we introduce DA-Flow , a D egradation- A ware Optical Flow network built on top of RAFT [ 31 ] . As illustrated in Fig. 2 , DA-Flow combines upsampled diffusion features from the lifted model with conventional CNN-based encoder features into a hybrid representation, enabling the correlation and iterative update stages to benefit from both degradation-aware structural priors and fine-grained spatial detail. Since ground-truth optical flow for real-world degraded videos is unavailable, we train DA-Flow using pseudo ground-truth flow generated by applying a pretrained flow model [ 37 ] to a high-quality video, while feeding the corresponding degraded frames as input. We evaluate on degraded versions of established optical flow benchmarks [ 20 , 3 , 35 ] constructed via realistic degradation pipelines [ 4 , 36 ] , and demonstrate that DA-Flow achieves accurate flow estimation even under severe corruption where existing methods fail. Our main contributions are as follows: • We formulate Degradation-Aware Optical Flow , a new task that estimates accurate dense correspondences from severely corrupted videos. • We lift a pretrained image restoration diffusion model by introducing inter-frame attention and verify that its features encode geometric correspondence even under severe corruption. • We introduce DA-Flow , a degradation-aware optical flow network that substantially outperforms existing methods on degraded inputs. 2 Related Work Optical flow estimation. Optical flow estimation aims to model dense pixel-level motion between consecutive frames and serves as a fundamental component in various video-related tasks, including video generation and scene reconstruction. Modern deep learning approaches have significantly advanced flow estimation, among which RAFT [ 31 ] establishes a

Read the complete text