VibeFlow: Versatile Video Chroma-Lux Editing through Self-Supervised Learning
Yifan Li, Pei Cheng, Bin Fu, Shuai Yang, Jiaying Liu · 2026
Read VibeFlow: Versatile Video Chroma-Lux Editing through Self-Supervised Learning by Yifan Li, Pei Cheng, Bin Fu, Shuai Yang, Jiaying Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-13425.txt.
Introduction
Yifan Li and four colleagues propose VibeFlow, a self-taught tool for editing video colour and light while keeping shape and timing intact. Old tools need costly training on fake paired data, but VibeFlow taps the physical sense inside pre-trained video generation models. A perturbation pipeline that splits structure from colour lets it mix shape from source videos with light and colour cues from reference images. Residual velocity fields and a structural consistency rule then protect shape and timing across relighting, recolouring, and day-night shifts.
Catalog record
| CREATOR | Yifan Li, Pei Cheng, Bin Fu, Shuai Yang, Jiaying Liu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 29054 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.13425
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
VibeFlow: Versatile Video Chroma-Lux Editing through Self-Supervised Learning Yifan Li, Pei Cheng, Bin Fu, Shuai Yang, Jiaying Liu Abstract Video chroma-lux editing, which aims to modify illumination and color while preserving structural and temporal fidelity, remains a significant challenge. Existing methods typically rely on expensive supervised training with synthetic paired data. This paper proposes VibeFlow , a novel self-supervised framework that unleashes the intrinsic physical understanding of pre-trained video generation models. Instead of learning color and light transitions from scratch, we introduce a disentangled data perturbation pipeline that enforces the model to adaptively recombine structure from source videos and color-illumination cues from reference images, enabling robust disentanglement in a self-supervised manner. Furthermore, to rectify discretization errors inherent in flow-based models, we introduce Residual Velocity Fields alongside a Structural Distortion Consistency Regularization, ensuring rigorous structural preservation and temporal coherence. Our framework eliminates the need for costly training resources and generalizes in a zero-shot manner to diverse applications, including video relighting, recoloring, low-light enhancement, day-night translation, and object-specific color editing. Extensive experiments demonstrate that VibeFlow achieves impressive visual quality with significantly reduced computational overhead. Figure 1: Our proposed VibeFlow achieves versatile video color and illumination editing while strictly preserve structural fidelity, enjoying rich applications, including video relighting, recoloring, day-night translation, low-light enhancement, and object-specific color editing. 1 Introduction The rapid evolution of diffusion models has significantly advanced the field of visual content editing. Within this landscape, effective control over illumination and color remains paramount, enjoying a wide range of high-value applications, including cinematic post-production 6 ; 24 , photography retouching 5 , and immersive environment design for embodied AI 28 ; 1 . Distinct from general video stylization 40 which requires proper modifications on texture details and geometry, video chroma-lux editing necessitates strict temporal and structural fidelity while synthesizing photorealistic colors and illumination. Recently, large-scale unified editing models for both images 32 ; 23 and videos 20 ; 7 have emerged, aiming to provide all-in-one editing solutions. While these models offer broad applicability, they struggle to maintain strict structural fidelity during editing 44 ; 18 , leading to inconsistent object identity and texture appearance unintentionally. In contrast, specialized approaches tailored for color/light editing typically rely on task-specific synthetic datasets with supervised training. For example, IC-Light 47 finetunes Stable Diffusion 34 towards image relighting with over 10 million images. CtrlColor 25 and COCO-LC 24 both utilize ControlNet 46 and paired gray/color data for image recoloring. Although effective, these methods predominantly focus on image chroma-lux editing, paying less attention to video applications that present a more critical need. To bridge this gap, existing methods can be categorized into three groups. The first one extends image diffusion models to the video domain by incorporating cross-frame mechanisms for improved coherency, such as temporal attention modules 28 ; 9 , feature inflation 14 , and inter-frame object binding strategy 6 , reducing temporal flickering and fluctuation. However, they still struggle to generate natural and coherent videos as they learn temporal consistency from scratch. The second category represents training-free approaches 22 ; 11 ; 21 , which aim to alleviate the expensive training overhead, however still limited by the lack of video-specific optimization and handcrafted priors, producing unsatisfactory results. The third category involves finetuning video generation models 1 ; 26 ; 45 on specific synthesized data. However, these methods rely heavily on auxiliary rendering data or physical priors (e.g., normal maps, albedo, depth) to supervise the training process, which is computationally expensive and labor-intensive, thereby restricting these methods to specific scenes. In summary, such specialized methods are typically confined to narrow applications and require dedicated training. Building upon these observations, we identify that the fundamental challenge of video chroma-lux editing lies in simultaneously satisfying three critical objectives: temporal coherence, photorealistic color/light manipulation, and strict structural fidelity . Existing approaches struggle to satisfy these demands without incurring prohibitive training/inference costs or relying on scarce paired data. Fortunately, content creation has witnessed a major transformation thanks to the rapid emergence of large-scale video generation models. In contrast, inspired by 39 , our approach stems from a key intuition: since modern video generation models can synthesize realistic and continuous videos with complex lighting and colors, they must inherently possess a profound understanding of these photometric properties. Driven by this insight, we introduce VibeFlow , a novel self-supervised framework with a disentangled data perturbation pipeline, mitigating the need for external physical supervision or paired data . Specifically, we propose an innovative self-supervised proxy task for improved structural fidelity. Based on the video generation backbones that conditioned on a source video and a reference image, we selectively perturb specific components within the conditional inputs and finetune the model to reconstruct the original video, reinforcing the video generation models to learn the implicit disentanglement and adaptive recomposition of structural and color/light components from the conditional source videos and reference images respectively. Without any paired data, our method achieves versatile and precise modulation of photometric attributes, specifically illumination and colors, while rigorously preserving the temporal consistency and fine-grained details of source videos. With the disentangled framework established, an intriguing question naturally arises: If we utilize the first frame of the source video as the reference condition, can the model faithfully reconstruct the input video? While theoretically implying an identity mapping, we empirically observe structural degradation due to the discretization error of flow-based solvers 8 . To mitigate this issue, we propose the Residual Velocity-guided Structural Preservation . We define the Residual Velocity Fields to rectify the sampling trajectory, which is conceptually inspired by 21 ; 10 . Serving as the structural details compensation, the proposed residual velocity is derived from the reconstruction deviation under the original color/light conditions, which is subsequently added to the predicted velocity field of the target conditions, effectively and efficiently rectifying the editing sampling trajectory. Furthermore, to ensure these residual fields encapsulate pure structural information and minimally entangled with the original illumination or colors of source videos, we propose a Structural Distortion Consistency Regularization . By enforcing consistent structural degradation across varying conditions during training, this regularization collectively improves the precise preservation of fine-grained details and motion dynamics. In summary, our framework delivers a straightforward yet powerful approach to flexible video chroma-lux editing, while avoiding the reliance on costly paired data and minimizing training overhead. Remarkably , by fully unleashing the potential capabilities of the video generation backbone, we enable diverse applications with various image editing models 23 ; 17 ; 32 , including video relighting, recoloring, day-night transfer, low-light enhancement, and object-specific color editing. Extensive experiments show that our approach outperforms state-of-the-art methods in both visual quality and temporal consistency. Our contributions can be summarized as follows: • We propose VibeFlow, a novel self-supervised framework for versatile video chroma-lux editing. Without the requirements of paired data or expensive training overhead, our approach achieves high-quality and high-fidelity video color-light editing. • We define Residual Velocity Fields for improved structure preservation and temporal consistency. We first propose a sampling trajectory rectification mechanism, which is further empowered by our proposed structural distortion consistency regularization, collectively ensuring precise preservation of the source video’s fine-grained details and motion dynamics. • Building upon data perturbation pipeline and self-supervised disentangled training, our method demonstrates exceptional zero-shot generalization