SkyReels-V3 Technique Report
SkyReels Team · 2026
Read SkyReels-V3 Technique Report by SkyReels Team (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-17323.txt.
Introduction
The SkyReels Team presents SkyReels-V3, a model that makes video from mixed inputs. Video work is a core test for world models. A strong model must infer what should happen from text, images, and sound cues. SkyReels-V3 uses one shared frame for that in-context learning. It is built on diffusion Transformers, the current workhorse for video.
The model is conditional, so users steer output with the inputs they give. One frame for many input types cuts the need for split tools. Such models serve film, ads, and training worlds for robots. The tech report lays out the design. It shows where this line of models stands now.
Catalog record
| CREATOR | SkyReels Team |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 18 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-17323
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SkyReels-V3 Technique Report SkyReels Team Abstract Video generation serves as a cornerstone for building world models, where multimodal contextual inference stands as the defining test of capability. In this end, we present SkyReels-V3, a conditional video generation model, built upon a unified multimodal in-context learning framework with diffusion Transformers. SkyReels-V3 model supports three core generative paradigms within a single architecture: reference images-to-video synthesis, video-to-video extension and audio-guided video generation. (i) reference images-to-video model is designed to produce high-fidelity videos with strong subject identity preservation, temporal coherence, and narrative consistency. To enhance reference adherence and compositional stability, we design a comprehensive data processing pipeline that leverages cross-frame pairing, image editing, and semantic rewriting, effectively mitigating copy–paste artifacts. During training, an image–video hybrid strategy combined with multi-resolution joint optimization is employed to improve generalization and robustness across diverse scenarios. (ii) video extension model integrates spatio-temporal consistency modeling with large-scale video understanding, enabling both seamless single-shot continuation and intelligent multi-shot switching with professional cinematographic patterns. (iii) Talking avatar model supports minute-level audio-conditioned video generation by training first-and-last frame insertion patterns and reconstructing key-frame inference paradigms. On the basis of ensuring visual quality, synchronization of audio and videos has been optimized. Extensive evaluations demonstrate that SkyReels-V3 achieves state-of-the-art or near state-of-the-art performance on key metrics including visual quality, instruction following, and specific aspect metrics, approaching leading closed-source systems. Github: https://github.com/SkyworkAI/SkyReels-V3 . 1 Introduction World models aim to capture, simulate, and forecast the dynamics of complex real-world environments, and they form a fundamental basis for deploying artificial intelligence in practical scenarios 11 . In between, video generation frameworks encode rich geometric, semantic, and physical knowledge through the synthesis of visual sequences, thereby enabling effective modeling and prediction of the physical world, especially for multi-modal conditions. In recent years, diffusion-based Transformers 13 ; 4 architecture has driven significant advances in video generation. A wide range of commercial systems—including Veo 7 , Sora 12 , Seedance 6 ; 2 , Kling 9 , as well as open-source models such as Wanx 14 , HunyuanVideo 15 ; 8 , SkyReels 1 , and CogVideoX 17 , have demonstrated strong performance across multiple dimensions. However, multimodal in-context in video generation is still under-explored. In this repport, we introduce SkyReels-V3, a unified multimodal condition video generation framework, designed to support a wide range of high-quality video synthesis tasks within a single model family. Built upon a multimodal in-context learning paradigm, SkyReels V3 seamlessly integrates visual reference, video, audio, and textual inputs to enable flexible and controllable video generation. The framework natively supports three core capabilities: reference images to video, video-to-video extension, and audio-guided video generation, also known as talking avatar. At the architectural level, SkyReels-V3 incorporates large-scale diffusion Transformers while with carefully designed alignment strategies with multimodal condition and advanced spatiotemporal consistency modeling. Through hybrid image–video as well as multi-resolution joint optimization, the system achieves precise instruction following, high-fidelity motion generation and superior sub-domain capacity such as robust identity preservation and precise audio-visual synchronization. These design choices allow SkyReels-V3 to move beyond frame-level synthesis, enabling coherent narrative progression and cinematic-quality visual composition. With its strong generalization ability and modular design, SkyReels-V3 can be applied to diverse real-world scenarios, including professional video production, virtual avatars, short-form content creation, live-stream commerce, and digital entertainment. Extensive evaluations demonstrate that SkyReels-V3 reaches or surpasses industry-leading performance across key metrics, making it a powerful open-source foundation for next-generation video generation research and applications. 2 Methods and Evaluation The SkyReels-V3 model family supports a range of capabilities, including multi-reference image-to-video synthesis, audio-guided video generation, and video-to-video extension. This chapter provides a detailed descriptions and performance evaluation to these features. 2.1 Reference Images to Video Figure 1: Reference Images to Video Results. SkyReels-V3 can facilitate dynamic interplay between different subjects within specified contexts. Figure 2: Reference Images to Video Results. SkyReels-V3 can enable dynamic interactions between diverse subjects (characters/objects) within arbitrary scenes. Figure 3: Reference Images to Video Results. SkyReels-V3 can enable instant video creation for diverse live commerce hosts and settings. Figure 4: Reference Images to Video Results. SkyReels-V3 can make advertising and product demonstration with one picture. SkyReels-V3 can synthesizes temporally coherent video sequences conditioned on multiple visual references and a textual prompt. Given one to four reference images, which may correspond to characters, objects, or background scenes, the model generates videos that preserve identity attributes, spatial composition, and narrative continuity while following high-level semantic instructions. Reference-Preserving Data Construction. For multi-reference image-guided video generation, the quality of the reference image-to-target video pairs is crucial. To this end, we introduce a dedicated data processing pipeline. Initially, we filter video data from a massive in house dataset, selecting clips that exhibit both high visual quality and significant dynamic motion. Then, reference frames are selected from continuous video sequences using a cross-frame pairing strategy 5 ; 3 , ensuring temporal diversity while maintaining semantic consistency. Image editing models 10 ; 16 are then applied to extract subject regions and perform background completion, together with semantic rewriting, to construct training pairs that avoid trivial frame copying and reduce copy-and-paste artifacts. Furthermore, we have developed multiple filtering steps to remove distorted and inconsistent reference images generated by editing models. Multi-Reference Conditioning. To effectively integrate heterogeneous reference inputs, SkyReels-V3 employs a unified multi-reference conditioning strategy that jointly encodes visual and textual information. For each reference image, we encode it using the video VAE and subsequently concatenate the resulting latent representation with the video latents. By allowing up to four reference images, the model supports flexible scene composition and enables fine-grained control over subject appearance and background structure. This design facilitates complex multi-subject and multi-element video generation without requiring explicit manual composition. Training Strategy. We train the model using an image–video hybrid training scheme that jointly leverages large-scale image and video datasets. This approach enhances generalization by exposing the model to both static appearance cues and dynamic motion patterns. In addition, multi-resolution joint training is employed to improve robustness across different spatial scales and aspect ratios, enabling the model to natively support a wide range of output configurations. Benchmark. Table 1: Quantitative comparison of video generation models on reference consistency, instruction following, and visual quality. Higher is better. Model Reference Consistency \uparrow Instruction Following \uparrow Visual Quality \uparrow Vidu Q2 0.5961 27.84 0.7877 Kling 1.6 0.6630 29.23 0.8034 PixVerse V5 0.6542 29.34 0.7976 SkyReels-V3 0.6698 27.22 0.8119 We construct a test set comprising 200 data pairs, with sources spanning scenarios such as film and television, e-commerce, and advertising. The types of reference images include those featuring characters, animals, objects, and background scenes. The model’s capabilities are evaluated from three primary perspectives: Reference Consistency, Instruction Following, and Visual Quality. Specifically, Reference Consistency encompasses aspects such as facial consistency, clothing consistency, object consistency, and background consistency. Visual Quality is assessed based on image quality, dynamic degree, aesthetic quality, and motion smoothness. SkyReels-V3 has been benchmarked against leading contemporary models, with comparative results presented in Table 1 . The results demonstrate that the SkyReels-V3 model achieves state-of-the-art performance within the industry. We also show qualitative results in Figure 1 , Figure 2 , Figure 3 , and Figure 4 . These results indicate that the model demonstrates strong generalization capabilities across a wide range of