FractalPINN-Flow: A Fractal-Inspired Network for Unsupervised Optical Flow Estimation with Total Variation Regularization
Sara Behnamian, Rasoul Khaksarinezhad, Andreas Langer · 2025
Read FractalPINN-Flow: A Fractal-Inspired Network for Unsupervised Optical Flow Estimation with Total Variation Regularization by Sara Behnamian, Rasoul Khaksarinezhad, Andreas Langer (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-08670.txt.
Introduction
Estimating motion between video frames, called optical flow, usually needs labeled training data that is expensive to make. Sara Behnamian, Rasoul Khaksarinezhad, and Andreas Langer present FractalPINN-Flow in a 2025 research paper. Their framework learns optical flow without any ground truth, directly from consecutive grayscale frames. Its core is a recursive encoder-decoder network inspired by fractal self-similarity.
Instead of shrinking the image step by step, the network nests encoder-decoder blocks inside each other. Skip connections preserve both fine detail and long-range motion. Training minimizes a classical energy that enforces brightness constancy, plus a total variation term favoring smooth, coherent flow. Experiments on synthetic and benchmark data show accurate, edge-preserving flow fields. The method shines on high-resolution data and wherever annotations are scarce, two common headaches in real video analysis.
Catalog record
| CREATOR | Sara Behnamian, Rasoul Khaksarinezhad, Andreas Langer |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 26 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-08670
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
FractalPINN-Flow: A Fractal-Inspired Network for Unsupervised Optical Flow Estimation with Total Variation Regularization Sara Behnamian, Rasoul Khaksarinezhad, Andreas Langer Abstract We present FractalPINN-Flow, an unsupervised deep learning framework for dense optical flow estimation that learns directly from consecutive grayscale frames without requiring ground truth. The architecture centers on the Fractal Deformation Network (FDN)—a recursive encoder-decoder inspired by fractal geometry and self-similarity. Unlike traditional CNNs with sequential downsampling, FDN uses repeated encoder-decoder nesting with skip connections to capture both fine-grained details and long-range motion patterns. The training objective is based on a classical variational formulation using total variation (TV) regularization. Specifically, we minimize an energy functional that combines L 1 L^{1} and L 2 L^{2} data fidelity terms to enforce brightness constancy, along with a TV term that promotes spatial smoothness and coherent flow fields. Experiments on synthetic and benchmark datasets show that FractalPINN-Flow produces accurate, smooth, and edge-preserving optical flow fields. The model is especially effective for high-resolution data and scenarios with limited annotations. Keywords: Optical Flow, Unsupervised Learning, Neural Networks, Total Variation, Encoder-Decoder, Motion Estimation 1 Introduction Optical flow estimation seeks to recover the apparent motion field between two consecutive grayscale images. Let I_{1},I_{2}:\Omega\to[0,1] denote two discrete images defined on a spatial domain \Omega\subset\mathbb{Z}^{2} , and let w:\Omega\to\mathbb{R}^{2} , w(x)=(u(x),v(x)) , be the displacement field to be estimated. The fundamental assumption underlying most optical flow methods is brightness constancy, which posits that the intensity of each point remains constant as it moves, i.e., I_{1}(x)=I_{2}(x+w(x))\quad\text{for all }x\in\Omega. (1) This results in a nonlinear constraint on w . Since the problem is underdetermined (one equation, two unknowns), additional regularization is required. To simplify the nonlinear data term, many methods approximate the brightness constancy relation ( 1 ) by a linear model: I_{2}(x+w(x))\approx I_{2}(x)+\nabla I_{2}(x)\cdot w(x), where \nabla I_{2} denotes the image gradient. This yields the approximate linear constraint \nabla I_{2}(x)\cdot w(x)+I_{2}(x)-I_{1}(x)=0, as used in the classical Horn–Schunck formulation [ 7 ] . There the flow field is recovered by minimizing \|\nabla I_{2}\cdot w+I_{2}-I_{1}\|_{2}^{2}+\lambda\|\bm{\nabla}w\|_{2}^{2}, where \lambda>0 is a regularization parameter, \|\bm{\nabla}w\|_{2}^{2}=\|\nabla u\|_{2}^{2}+\|\nabla v\|_{2}^{2} encourages global smoothness of the displacement field and \|\cdot\|_{2} denotes the standard Euclidean norm. This quadratic regularization penalizes rapid variations in the flow but tends to oversmooth motion boundaries. A more robust alternative replaces the L^{2} -norm with the L^{1} -norm in both the data fidelity and smoothness terms, yielding a total variation (TV) regularization model, allowing the flow field to exhibit discontinuities. This leads to the energy \|\nabla I_{2}\cdot w+I_{2}-I_{1}\|_{1}+\lambda\|\bm{\nabla}w\|_{1}, where the last term represents the total variation of the flow field w . In practice, this formulation encourages piecewise-smooth flow fields while preserving motion discontinuities [ 22 ] . To efficiently solve TV-regularized optical flow problems, in [ 5 ] a primal-dual finite element method was introduced and combined with an iterative warping algorithm to handle large displacements. Building on this in [ 1 , 8 ] adaptive discretizations schemes were proposed, which can be interpreted as an adaptive multigrid method, further accelerating convergence and sometimes even improving the quality of the estimated flow field. With the advent of deep learning, optical flow estimation has seen dramatic advances. FlowNet [ 4 ] introduced a convolutional neural network (CNN) architecture capable of predicting flow directly from image pairs, opening the door to data-driven solutions, see also [ 18 ] . However, supervised methods require large-scale annotated datasets, which are difficult to obtain for many real-world domains. To address this challenge, unsupervised approaches have gained momentum by optimizing photometric consistency and smoothness priors without ground-truth supervision. These methods can be further classified according to the number of input frames used during training, including multiframe models [ 19 ] , which leverage temporal consistency across sequences, and two-frame models [ 9 , 14 ] , which are based solely on image pairs. Although most of such models incorporate smoothness losses, they typically do not employ TV regularization explicitly. In this paper, we propose FractalPINN-Flow , a novel unsupervised framework that integrates an encoding-decoding strategy with total variation regularization to estimate dense optical flow from grayscale image sequences. Our use of TV minimization in a neural setting is motivated by recent work such as DeepTV [ 12 ] , which demonstrated how classical regularization techniques can be effectively combined with deep architectures to produce sharp, structurally coherent outputs. At the core of our model is a Fractal Deformation Network (FDN) , a recursive encoder-decoder architecture inspired by the self-similar principles of FractalNet [ 13 ] . Unlike conventional hierarchical CNNs, which apply sequential downsampling, the FDN recursively nests encoder-decoder modules at multiple scales, each equipped with skip connections. This design builds a deep multiscale representation that maintains local texture while capturing long-range deformation structures, properties that are especially important in scenes with fine motion patterns or limited training data. The FDN output is passed to a lightweight CNN that predicts the optical flow field. Training is fully unsupervised, uses only two frames and is guided by a composite loss function that consists of a combined L^{1} / L^{2} data term and TV regularization. 2 FractalPINN-Flow: Architecture and Implementation We consider the problem of learning the optical flow from two input images I_{1} , I_{2} by minimizing E_{\mathrm{TV}}(w):=\lambda_{1}\|\nabla I_{2}\cdot w+I_{2}-I_{1}\|_{1}+\lambda_{2}\|\nabla I_{2}\cdot w+I_{2}-I_{1}\|_{2}^{2}+\lambda_{\mathrm{TV}}\|\bm{\nabla}w\|_{1}, (2) where \lambda_{1},\lambda_{2},\lambda_{\mathrm{TV}}\geq 0 , w is represented by a neural network, see Section 2.1 for a detailed description of the network structure, and \|\bm{\nabla}w\|_{1}:=\|\nabla_{x}u\|_{1}+\|\nabla_{y}u\|_{1}+\|\nabla_{x}v\|_{1}+\|\nabla_{y}v\|_{1} (3) is the anisotropic TV. Here, \nabla_{x} and \nabla_{y} denote the finite differences along the horizontal and vertical axes, respectively. This regularization penalizes abrupt discontinuities while preserving meaningful motion boundaries, thereby promoting spatially coherent and piecewise-smooth flow estimates. We note that while the gradients in ( 3 ) could also be implemented in a pointwise manner, as is common in physics-informed neural networks [ 16 ] , we refrain from doing so following the rationale in [ 12 ] . In particular, pointwise evaluation may completely miss jump discontinuities, especially in piecewise constant outputs, by sampling points where the gradient happens to vanish. In contrast, a finite difference discretization reliably captures such variations and provides a more accurate measure of total variation. We note that a combined L^{1} / L^{2} data fidelity term together with total variation regularization, as used in ( 2 ), was first introduced in [ 6 ] and analyzed in [ 11 ] in the context of variational image restoration. We consider this formulation here for optical flow estimation. 2.1 Neural Network We propose a deep learning architecture for dense optical flow estimation that integrates our novel FDN with a CNN-based optical flow prediction head. The method estimates pixel-wise motion between two consecutive frames by leveraging multi-scale fractal features. 2.1.1 Network Architecture High-Level Structure. The architecture of the neural networks we use follows a sequential flow of processing stages, outlined as follows: Input: Two consecutive grayscale frames (I_{1},I_{2}) \downarrow FDN: Encoder-decoder network \downarrow Projection Layer: 32\rightarrow 64 channels \downarrow Optical Flow Network: CNN with 5 layers \downarrow Output: Dense optical flow field (u,v) The FDN encodes multi-scale motion features, which are then mapped to dense flow fields by a convolutional regression head. This modular structure mirrors successful encoder-decoder designs widely used in optical flow [ 20 , 21 ] and image