VTinker: Guided Flow Upsampling and Texture Mapping for High-Resolution Video Frame Interpolation
Chenyang Wu, Jiayi Fu, Chun-Le Guo, Shuhao Han, Chongyi Li · 2025
Read VTinker: Guided Flow Upsampling and Texture Mapping for High-Resolution Video Frame Interpolation by Chenyang Wu, Jiayi Fu, Chun-Le Guo, Shuhao Han, Chongyi Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-16124.txt.
Introduction
Chenyang Wu, Jiayi Fu, and three co-authors present VTinker, a pipeline for video frame interpolation at high resolution. Interpolation creates intermediate frames between existing ones, and flow-based methods first estimate motion at low resolution, then upscale it. Simple upscaling blurs motion edges, and fine pixel motion is missed, so warping frames pixel by pixel produces ghosting and broken seams in the result.
Guided flow upsampling uses the input frames themselves as guidance to sharpen flow edges after low-resolution estimation. Texture mapping then builds an initial interpolated frame at texture level rather than by naive pixel warping, avoiding ghosting and discontinuities. Together the two components target sharper, more coherent high-resolution interpolated frames.
Catalog record
| CREATOR | Chenyang Wu, Jiayi Fu, Chun-Le Guo, Shuhao Han, Chongyi Li |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-16124
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
VTinker: Guided Flow Upsampling and Texture Mapping for High-Resolution Video Frame Interpolation Chenyang Wu, Jiayi Fu, Chun-Le Guo, Shuhao Han, Chongyi Li Abstract Due to large pixel movement and high computational cost, estimating the motion of high-resolution frames is challenging. Thus, most flow-based Video Frame Interpolation (VFI) methods first predict bidirectional flows at low resolution and then use high-magnification upsampling (e.g., bilinear) to obtain the high-resolution ones. However, this kind of upsampling strategy may cause blur or mosaic at the flows’ edges. Additionally, the motion of fine pixels at high resolution cannot be adequately captured in motion estimation at low resolution, which leads to misalignment in task-oriented flows. With such inaccurate flows, input frames are warped and combined pixel-by-pixel, resulting in ghosting and discontinuities in the interpolated frame. In this study, we propose a novel VFI pipeline, VTinker, which consists of two core components: guided flow upsampling (GFU) and Texture Mapping. After motion estimation at low resolution, GFU introduces input frames as guidance to mitigate detail blurring in bilinear upsampling flows, which makes flows’ edges clearer. Subsequently, to avoid pixel-level ghosting and discontinuities, Texture Mapping generates an initial interpolated frame, referred to as the intermediate proxy. The proxy serves as a cue for selecting clear texture blocks from the input frames, which are then mapped onto the proxy to facilitate producing the final interpolated frame via a reconstruction module. Extensive experiments demonstrate that VTinker achieves state-of-the-art performance in VFI. Code is available at: https://github.com/Wucy0519/VTinker . 1 VCIP, CS, Nankai University 2 NKIARI, Shenzhen Futian {chenyangwu, jiayifu, hansh}@mail.nankai.edu.cn, {guochunle, lichongyi}@nankai.edu.cn Introduction Recent advancements in imaging technologies and devices have made high-resolution video more widely accessible. However, due to hardware limitations and network transmission constraints, high-resolution videos are often at a low frame rate. To improve the frame rate of high-resolution video, video frame interpolation (VFI) that synthesizes intermediate frames between consecutive frames in a video sequence is commonly applied ( Reda et al. 2022 ) . Figure 1: This figure compares VTinker with current flow-based method for high-resolution VFI. After Motion Estimation (M.E.), bidirectional flows are obtained at low resolution. (a) The current method employs bilinear upsampling for the high-resolution flows, followed by a pixel-by-pixel synthesis. (b) VTinker employs the proposed Guided Flow Upsampling (GFU) and generates the final result through texture mapping. Benefiting from the advantage of motion estimation, flow-based VFI methods are becoming mainstream. As shown in Fig. 1 , flow-based high-resolution VFI can be divided into three interrelated stages. (1) Motion Estimation (M.E.) : Given the challenges of larger movements and higher computational demands, motion estimation is performed on the downsampled frames in most high-resolution VFI methods. (2) Upsampling : To apply low-resolution motion estimation to high-resolution frames, flow-based methods commonly use high-magnification upsampling (e.g., bilinear or adaptive-kernel based) on the low-resolution flow. (3) Frame Synthesis : Depending on the upsampled flow, the frame synthesis model predicts an intermediate frame between two consecutive frames. Unfortunately, there are deficiencies in existing flow-based high-resolution VFI pipelines. Specifically, as shown in Fig. 1 (a), in the context of two consecutive high-resolution frames, {I_{0}} & {I_{1}} , the current approach involves frame downsampling and performs flow estimation through motion estimation at a low resolution ( Li et al. 2023 ; Liu et al. 2024 ) . Subsequently, flow-based methods ( Jin et al. 2023a ; Liu et al. 2024 ) upsample the estimated low-resolution flows using algorithms such as Bilinear Upsampling or Adaptive-kernel Based Method that is called adaptive flow upsampling (AFU, proposed in ASFlow ( Luo et al. 2021a ) for flow estimation task and used in SGM ( Liu et al. 2024 ) for VFI). However, in the face of task-oriented flow upsampling during end-to-end training, AFU produces discontinuous boundaries, which also makes it less suitable for flow-based VFI models (such as widespread mosaic-like artifacts in SGM’s results, as shown in Figs. 4 and 5 ). The more commonly used method is bilinear upsampling. However, it generally causes the blurring motion boundaries ( Luo et al. 2021a ) and further leads to blurring or ghosting. Additionally, the robust application for the task-oriented flow, which is estimated at low resolution but applied to high resolution, presents a significant challenge. In the context of motion estimation at low resolution, it is evident that the motion of fine pixels at high resolution is not adequately captured. The downsampling operation causes the model to prioritize overall motion and leads to neglecting finer details. When flow is upsampled, task-oriented flow encounters difficulties in generalizing to high-resolution applications, which results in biased motion alignment. Furthermore, in the frame synthesis stage, the given frames {I_{0}} & {I_{1}} are warped to {I_{t}^{0}} & {I_{t}^{1}} with obtained flows, respectively. They are then merged into an intermediate frame {{\hat{I}}_{t}^{{}^{\prime}}} , which incorporates the predicted mask. Then, a network is used to predict a residual term {Res} and add it to the frame {\hat{I}_{t}^{{}^{\prime}}} to produce the final output {{\hat{I}}_{t}} at the pixel level ( Jin et al. 2023a ; Li et al. 2023 ) . This frame synthesis mechanism, called ‘Mask&Res’ , produces results at the pixel level with dual-source texture from both {I_{0}} and {I_{1}} , as shown in Fig. 1 (a). However, the motion estimation is inaccurate due to the blurred boundary and unalignment of the upsampled flows. This results in the interpolation generally exhibiting ghosting, blurring, and discontinuities, because it is based on a fusion of the two misaligned warped frames. To improve the performance of the high-resolution VFI model and yield clearer results, as shown in Fig. 1 (b), we propose a novel pipeline, VTinker. VTinker consists of two core components: Guided Flow Upsampling (GFU) and Texture Mapping . Inspired by UPFlow ( Luo et al. 2021b ) , which uses high-resolution information as guidance for flow upsampling in flow estimation task, we propose GFU for task-oriented flow refinement. Guided by high-resolution information, the upsampling process using GFU produces sharper flows, aligning more closely with the edges of associated frames. GFU helps to provide better high-resolution motion estimation, which improves the quality of the result and reduces artifacts at the boundaries. In order to reduce ghosting and discontinuities due to the misaligned flows and the ‘Mask&Res’ mechanism, VTinker uses texture mapping to synthesize the interpolated frame. Specifically, VTinker first generates an intermediate proxy, and then maps it using blocks of regional textures, which are exclusively selected from frame {I_{0}} or {I_{1}} . These continuous block textures, sourced solely from either {I_{0}} or {I_{1}} , remain unaffected by incorrect warping, resulting in a higher degree of continuity in the texture of {{\hat{I}}_{t}} . Finally, the reconstruction module rebuilds the proxy that is mapped by high-quality textures, producing clearer results. To ensure that the textures are of high quality, we use the weight-shared reconstruction module to reconstruct the extracted textures and supervise them with input frames. The contributions of this paper are as follows: • To address the boundary errors caused by current upsampling strategies, we propose Guided Flow Upsampling (GFU) inspired by UPFlow ( Luo et al. 2021b ) . It improves flow upsampling by using input frames to guide the refinement of task-oriented flows, effectively reducing ghosting along the boundaries in interpolated results. • To resolve