PROMO: Promptable Outfitting for Efficient High-Fidelity Virtual Try-On

Haohua Chen, Tianze Zhou, Wei Zhu, Runqi Wang, Yandong Guan, Dejia Song, Yibo Chen, Xu Tang, Lu Sheng, Zhiyong Wu · 2026

Read PROMO: Promptable Outfitting for Efficient High-Fidelity Virtual Try-On by Haohua Chen, Tianze Zhou, Wei Zhu, Runqi Wang, Yandong Guan, Dejia Song, Yibo Chen, Xu Tang, Lu Sheng, Zhiyong Wu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-11675.txt.

Introduction

Chen and co-authors present PROMO, a promptable framework for virtual try-on built on a Flow Matching transformer. Diffusion-based systems look realistic, yet their extra reference networks slow sampling down. PROMO treats the task as structured image editing with three demands: keep the subject, move texture faithfully, and blend the result cleanly. Latent multi-modal conditioning and self-reference cut the inference cost hard. On standard benchmarks, it tops prior systems and general editors on fidelity, with a strong speed balance.

Catalog record

CREATORHaohua Chen, Tianze Zhou, Wei Zhu, Runqi Wang, Yandong Guan, Dejia Song, Yibo Chen, Xu Tang, Lu Sheng, Zhiyong Wu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE28810
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.11675

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

PROMO: Promptable Outfitting for Efficient High-Fidelity Virtual Try-On Haohua Chen, Tianze Zhou, Wei Zhu, Runqi Wang, Yandong Guan, Dejia Song, Yibo Chen, Xu Tang, Lu Sheng, Zhiyong Wu Abstract Virtual Try-on (VTON) has become a core capability for online retail, where realistic try-on results provide reliable fit guidance, reduce returns, and benefit both consumers and merchants. Diffusion-based VTON methods achieve photorealistic synthesis, yet often rely on intricate architectures such as auxiliary reference networks and suffer from slow sampling, making the trade-off between fidelity and efficiency a persistent challenge. We approach VTON as a structured image editing problem that demands strong conditional generation under three key requirements: subject preservation, faithful texture transfer, and seamless harmonization. Under this perspective, our training framework is generic and transfers to broader image editing tasks. Moreover, the paired data produced by VTON constitutes a rich supervisory resource for training general-purpose editors. We present PROMO, a promptable virtual try-on framework built upon a Flow Matching DiT backbone with latent multi-modal conditional concatenation. By leveraging conditioning efficiency and self-reference mechanisms, our approach substantially reduces inference overhead. On standard benchmarks, PROMO surpasses both prior VTON methods and general image editing models in visual fidelity while delivering a competitive balance between quality and speed. These results demonstrate that flow-matching transformers, coupled with latent multi-modal conditioning and self-reference acceleration, offer an effective and training-efficient solution for high-quality virtual try-on. Figure 1 : PROMO enables multi-garment try-on, prompt-based control over dressing styles and demonstrates robust performance in challenging real-world scenarios. † † footnotetext: * Equal contribution. † Corresponding author: [email protected]. ‡ Corresponding author: [email protected] 1 Introduction Virtual Try-on (VTON) aims to render a target garment onto a given person image with high fidelity and realism. It is of practical importance in e-commerce, where it enables shoppers to obtain reliable fit and appearance references without in-store trials, provides interactive experiences, and reduces return rates, benefiting both consumers and merchants. Early VTON systems predominantly relied on warping-based pipelines 13 ; 52 ,. These approaches often yield unnatural appearances and degrade under challenging poses, occlusions, and large non-rigid deformations. GAN methods 26 ; 46 ; 25 ; 48 ; 16 , improve realism to some extent, yet they commonly struggle to preserve fine garment details and textures, produce coherent shading and illumination, and maintain natural human geometry and articulation. Spurred by the superior image synthesis performance of diffusion models 15 ; 34 ; 32 , a growing body of VTON research leverages diffusion-based generative modeling 9 ; 6 ; 47 ; 12 ; 28 ; 19 ; 7 ; 22 ; 5 ; 37 , demonstrating clear gains in visual fidelity and controllability. Many approaches use a diffusion UNet 35 or Transformer model 42 , adding conditions by using a reference net to create condition tokens, which are then appended to the main denoising net’s key/query sequences. These methods 19 ; 5 ; 47 ; 37 , require another whole net, resulting in complex initialization and interaction logic between the two nets. Other approaches 6 ; 31 use image-level concatenation (e.g., concatenating condition images as a larger image) to inject conditions. However, considering information density, parts of the conditions do not need to be at the same resolution as the output images. Moreover, the dressing style of the garment varies. Some works overlook this, replacing the text encoder with an image encoder. PromptDresser 22 uses prompts to control dressing style, but it requires a closed-source VLM to caption the dressing style. In summary, the contributions of this work include: • We propose PROMO, a multi-modal, prompt-controllable (promptable) virtual try-on framework. Leveraging a temporal self-reference mechanism across denoise timesteps, group-wise attention mask modulation, and efficient token merging, PROMO attains fast yet high-quality generation with a large-parameter Diffusion Transformer (DiT) backbone. • Motivated by real-world virtual try-on applications, we propose a Style Prompt System by tuning Qwen2.5-VL-7B, which provides reliable style description to support promptable control and downstream evaluation. • We conduct extensive experiments on the public VITON-HD and DressCode datasets and demonstrate consistent gains over state-of-the-art baselines VTON models and image-editing model in benchmark, achieving superior visual quality and realism. Figure 2 : Overview of scenarios handled by our PROMO framework. The left panel illustrates three common scenarios that customers encounter during online shopping: 1) model image available without a specific garment image, 2) garment image available without a model, and 3) both model and garment images available. Our system addresses the missing information in each scenario by generating the necessary conditional inputs required by our model. Notably, for scenarios without model images, our system features pure image-reference capability, allowing text prompts to be omitted while maintaining a robust baseline performance. 2 Related Work 2.1 Image-based Virtual Try-on Image-based virtual try-on synthesizes realistic images of a person wearing a target garment. Early warping-based methods used a two-stage pipeline (warping and fusion) with techniques like Thin Plate Spline (TPS) 53 ; 48 or appearance flow 13 , but suffered from alignment artifacts. Diffusion-based approaches now dominate, avoiding explicit warping. LaDI-VTON 28 and StableVITON 21 use ControlNet 50 -inspired conditioning. Others, like OOTDiffusion 47 , IMAGDressing-v1 38 and IDM-VTON 5 , employ parallel “reference nets” to encode garment features. Recent works like PromptDresser ( 22 ) leverages LMMs for text-editable style control ( e.g . “tuck”, “fit”) via prompt-aware masks to enhance controllability. Any2AnyTryon ( 12 ) uses a unified, mask-free DiT framework for multiple tasks, like layered try-on and garment reconstruction , using adaptive position embeddings to handle variable inputs. Existing diffusion methods often require complex architectures, motivating our pursuit of a more efficient framework. 2.2 Diffusion Models U-Net ( 35 ) was the predominant backbone for diffusion models like LDM ( 34 ) and SDXL ( 33 ) . DiT ( 32 ) successfully replaced the U-Net with a Transformer, demonstrating strong scalability. Recent models like Stable Diffusion 3 ( 8 ) and FLUX 24 ; 23 build on this, using transformer blocks and flow-matching formulations to predict velocity fields. We leverage the Diffusion Transformer architecture for its scalability, making it an ideal backbone for high-quality virtual try-on and capturing fine-grained garment details. 2.3 Multi-Condition Image Generation Virtual try-on is essentially a multi-conditional portrait generation task, closely related to multi-condition image generation models. FLUX-Kontext 23 extends FLUX 23 to support text-plus-single-image conditioning, but struggles when handling multiple reference images. DreamO 30 injects conditions via learnable embeddings, which suffer from limited generalization. UNO 45 proposes a model-data co-evolution paradigm that leverages Text-to-Image (T2I) models to generate high-quality single-subject data, thereby training stronger Subject-to-Image (S2I) models capable of producing diverse multi-subject outputs. OminiControl 40 separates subject and spatial conditions by toggling 2D positional-encoding offsets, implicitly assuming one reference per condition type, an assumption that fails in try-on scenarios with multiple garments. IC-LoRA 18 concatenates images at pixel level, ignoring heterogeneous information density across conditions and either sacrificing resolution or increasing computational cost. 3 Method 3.1 Overview 3.1.1 Modeling The overall framework of PROMO is shown in Fig. 2 . Given a person image I_{P} , garment images I_{G_{i}} and optional dressing style description T_{style} extracted by Style Prompt System from I_{model} . Our model aims to generate a image I_{new} , where the person wears the given garments

Read the complete text