MotionV2V: Editing Motion in a Video
Ryan Burgert Charles Herrmann Forrester Cole Michael S Ryoo, Neal Wadhwa Andrey Voynov Nataniel Ruiz · 2025
Read MotionV2V: Editing Motion in a Video by Ryan Burgert Charles Herrmann Forrester Cole Michael S Ryoo, Neal Wadhwa Andrey Voynov Nataniel Ruiz (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-20640.txt.
Introduction
Burgert, Herrmann, Cole, and co-authors let editors rewrite how things move inside a video. Their 2025 paper introduces MotionV2V for motion editing. Video generators now make striking footage but resist precise control. Editing the motion of an existing clip was largely unsolved.
Users drag sparse point trajectories to specify new motion. The gap between old and new paths is the motion edit. Counterfactual training pairs share content but differ in motion. A diffusion model fine-tuned on those pairs does the edit. Edits can start mid-video and propagate naturally onward. Camera motion and object motion both respond to the tool. In a four-way user study, theirs won over sixty-five percent preference. Forty-one participants judged twenty test videos. Baselines included ATI, ReVideo, and Go-with-the-Flow. Bezier splines shape the target paths in the interface.
Catalog record
| CREATOR | Ryan Burgert Charles Herrmann Forrester Cole Michael S Ryoo, Neal Wadhwa Andrey Voynov Nataniel Ruiz |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-20640
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MotionV2V: Editing Motion in a Video Ryan Burgert Charles Herrmann Forrester Cole Michael S Ryoo, Neal Wadhwa Andrey Voynov Nataniel Ruiz Abstract While generative video models have achieved remarkable fidelity and consistency, applying these capabilities to video editing remains a complex challenge. Recent research has extensively explored motion controllability as a means to enhance text-to-video generation or image animation; however, we identify precise motion control as a promising, yet under-explored, paradigm for editing existing videos . In this work, we propose modifying video motion by directly editing sparse trajectories extracted from the input. We term the deviation between input and output trajectories a ‘motion edit’ and demonstrate that this representation, when coupled with a generative backbone, enables many powerful video editing capabilities. To achieve this, we introduce a novel pipeline for generating ‘motion counterfactuals’ — video pairs that share identical content but distinct motion — and fine-tune a motion-conditioned video diffusion architecture on this dataset. Our approach allows for edits that start at any timestamp and propagate naturally. In a 4-way head-to-head user study, our model achieves over 65% preference against prior work. Please see our project page: ryanndagreat.github.io/MotionV2V 1 Introduction Figure 1 : Motion Edits Framework: Users provide an input video along with source motion tracks (colored dots connected by lines, extracted from the input) and target motion tracks (user-specified desired motion). Lines indicate point trajectories while dot presence/absence indicates visibility. Our diffusion model generates an output video matching the target motion. Applications: Our method can edit videos in a true sense, where content is preserved but motion is changed. Consider filming a climactic race between two dogs: your Corgi and a friend’s Bichon. The original video sees the Bichon take the win. After countless recent advances in generative models, does the technology exist to modify this video such that your Corgi is victorious? We propose a method for generalized motion editing in existing user-provided videos that successfully tackles this unsolved problem. Historically, tackling this problem in the VFX industry has been notably hard. A reshoot for scenes that need substantial changes is usually the necessary option. VFX pipelines can use tricks like retiming and plate stitching, isolated retimes with rotoscoping, or even full-dog CGI replacements. These typically require a high level of skill and large amounts of human hours. Modern generative models, with their impressive priors, show promise in tackling traditional VFX tasks. In this subfield, current methods for motion editing fall into different categories, each exhibiting significant constraints. Image-to-video (I2V) based approaches like Re-Video 24 and Go-with-the-Flow 3 can only generate new video with specified motion conditioned on a single image. Using these on the first frame of a video can give the illusion of video motion control, but have significant drawbacks. For example, content generated in regions that do not appear in that initial frame will be entirely hallucinated, whereas for true video motion editing these regions are known and should remain identical. Re-Video attempts to address this problem by inpainting information from the original video into the edited video, a technique which fundamentally breaks down when the video includes camera movement. Human-specific methods like MotionFollower and MotionEditor can edit motion but are limited to full-body human movements and cannot handle general objects or scenes. Likewise there are also works that allow editing camera trajectories in videos such as ReCapture 44 and ReCamMaster 1 . These are not able to edit subject motion. In this work we introduce motion edits, a new approach for editing videos by controlling the change in motion from the original to the edited video using video diffusion models. While there has been a large amount of successful recent work on appearance-based video editing (i.e. transforming visual style while preserving motion structure), motion editing presents a fundamentally different challenge. When editing how objects move within a scene (e.g. making a person walk in a different direction), the structural correspondence between input and output videos is broken. This makes the problem harder than appearance-based video editing and renders standard video editing techniques like DDIM inversion ineffective. Our method addresses this problem, and the limitations of prior work, by acting on the complete video and its motion representation. Users provide an input video along with some sparse tracking points on objects they wish to control; these objects are then automatically tracked throughout the video. Users can then choose to either anchor these points (to preserve original motion) or modify them (to edit the trajectory). For example, in a video of a person walking into a crowd, the system tracks both entities; the user can specify a new direction for the person by altering their tracks while strictly preserving the crowd’s original motion. In a more complex edit, the user can change the camera by editing all the points. Our approach enables diverse video editing capabilities (Fig. 1 ): object motion editing, camera motion editing, control over the timing of content, and successive edits. The motion edits are naturally introduced into the output video and the video model handles plausibility correctly - e.g. when dragging a person’s tracking point through an image the model will make the person traverse the image by walking. Our approach requires no manual masking and can handle any type of object, while also maintaining scene consistency even when both object motion and camera movement are edited simultaneously. And in contrast to prior methods, our approach also allows the ability to change when an object appears in the frame. Finally, our model generalizes to vastly different scenes and objects and achieves state-of-the-art performance in quantitative comparisons and human evaluations. In summary, we propose the following contributions: • We identify motion as a powerful control signal for video editing and propose directly editing sparse trajectories extracted from the input video to change the motion of the output video. We define the change between the input and target trajectories as a “motion edit” and show that motion edits, coupled with a powerful generative video model, can address several challenging video editing tasks. • We present a methodology to train a video diffusion model to generate high quality “motion counterfactual” video pairs which have the scene appearance but different motion. As part of this, we also identify sources of data that work well in this training. • We propose a new model architecture with careful conditioning on both user-specified video and motion trajectories that generates a motion-edited output. ———————————————————- 2 Related Works Diffusion models have fundamentally reshaped media generation, evolving from foundational image synthesis frameworks 14 ; 31 to complex video dynamics 15 ; 13 ; 33 ; 2 . Recent text-conditioned video models 42 ; 36 ; 49 have further advanced the field by adopting transformer-based architectures 27 for scalable denoising. 2.1 Conditional Video Generation Conditional video diffusion extends base text-to-video architectures by incorporating auxiliary control signals. Inspired by the spatial conditioning of ControlNet 46 , recent works have adapted similar mechanisms to the temporal domain 12 ; 16 ; 32 , enabling guidance through depth maps, motion vectors, and camera parameters. Concurrently, video-to-video (V2V) editing methods focus on propagating edits across frames while preserving the features of the source video 11 ; 29 ; 25 ; 4 ; 40 ; 18 ;