Voost : A Unified and Scalable Diffusion Transformer for Bidirectional Virtual Try-On and Try-Off

Seungyong Lee Jeong-gi Kwak · 2025

Read Voost : A Unified and Scalable Diffusion Transformer for Bidirectional Virtual Try-On and Try-Off by Seungyong Lee Jeong-gi Kwak (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-04825.txt.

Introduction

Seungyong Lee and Jeong-gi Kwak present in 2025 Voost, a single model that can dress a person in a garment from a photo. It can also do the reverse, removing the garment to recover its clean product image. Virtual try-on is a prized shopping tool. But matching cloth to body under new poses and looks remains hard.

Training one diffusion transformer on both directions lets each person-garment pair teach both tasks at once. The model accepts flexible conditioning on direction and garment type, with no separate networks per task. The shared learning sharpens how the model understands the fit between cloth and body.

Catalog record

CREATORSeungyong Lee Jeong-gi Kwak
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-04825

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Voost : A Unified and Scalable Diffusion Transformer for Bidirectional Virtual Try-On and Try-Off Seungyong Lee Jeong-gi Kwak Abstract Virtual try-on aims to synthesize a realistic image of a person wearing a target garment, but accurately modeling garment–body correspondence remains a persistent challenge, especially under pose and appearance variation. In this paper, we propose Voost —a unified and scalable framework that jointly learns v irtual try- o n and try- o ff with a s ingle diffusion t ransformer. By modeling both tasks jointly, Voost enables each garment-person pair to supervise both directions and supports flexible conditioning over generation direction and garment category—enhancing garment–body relational reasoning without task-specific networks, auxiliary losses, or additional labels. In addition, we introduce two inference-time techniques: attention temperature scaling for robustness to resolution or mask variation, and self-corrective sampling that leverages bidirectional consistency between tasks. Extensive experiments demonstrate that Voost achieves state-of-the-art results on both try-on and try-off benchmarks, consistently outperforming strong baselines in alignment accuracy, visual fidelity, and generalization. 1 Introduction Figure 1 : Attention map comparison— CatVTON Chong et al. 2025 shows dispersed attention unrelated to the query point, indicating weak spatial grounding. In contrast, our model produces sharply localized maps that align well with the corresponding garment regions, demonstrating stronger relational understanding. Virtual try-on (VTON) is an emerging generative task that enables realistic garment transfer onto a person image, offering a new paradigm for fashion in e-commerce and AR/VR. Despite its promise, VTON remains challenging due to the need for precise alignment, detail preservation, and robustness to occlusion and pose variation. Early approaches Han et al. 2018 ; Yang et al. 2020 ; Ge et al. 2021b ; Choi et al. 2021 have adopted warping-based strategies, often combined with generative modeling, but have struggled to maintain garment fidelity and structural consistency. Effectively adapting T2I diffusion models Rombach et al. 2022 ; Esser et al. 2024 to virtual try-on requires strong image-based conditioning and accurate modeling of garment–person correspondence. To this end, various efforts have been made to improve spatial alignment and relational understanding. These include incorporating auxiliary objectives Kim et al. 2024 ; Zhou et al. 2025 , leveraging reference networks Zhu et al. 2023 ; Choi et al. 2024 ; Xu et al. 2025 , or injecting conditioning signals through spatial concatenation Chong et al. 2025 . Despite these efforts, existing methods often fail to establish precise garment–person correspondence. As illustrated in Fig. 1 , the dispersed attention patterns reflect limited relational understanding, often resulting in lower visual quality and failure to faithfully reconstruct garment details. To address these challenges, we introduce Voost , a unified framework that jointly learns virtual try-on and its inverse task, virtual try-off, which aims to reconstruct the original appearance of the worn garment from a person image, within a single diffusion transformer. Voost adopts a token-level concatenation structure, where spatially aligned garment and person images are fed into a shared embedding space. This design enables the model to reason bidirectionally across try-on and try-off scenarios using a common conditioning layout. By jointly training both directions, Voost strengthens garment–person correspondence while requiring no task-specific networks, auxiliary losses, or additional annotations, as each garment-person pair naturally provides supervision for the reverse process. Our framework leverages token-level concatenation and the architectural flexibility of diffusion transformers to handle diverse poses, aspect ratios, and spatial layouts. A task token encodes both generation direction and garment category, enabling scalable learning without task-specific models or fixed resolutions. This unified setup not only supports multitask learning but also mitigates architectural, task-specific, and category-specific inductive biases by exposing the model to broader structural variation. In addition, we propose two simple inference-time techniques to improve robustness. Attention temperature scaling mitigates resolution or mask size mismatch between training and test time. We also introduce self-corrective sampling, where try-on and try-off predictions iteratively refine each other within the unified model. Remarkably, our unified diffusion transformer achieves state-of-the-art performance on both VTON and VTOFF benchmarks, surpassing strong task-specific baselines Kim et al. 2024 ; Choi et al. 2024 ; Zhou et al. 2025 ; Chong et al. 2025 ; Velioglu et al. 2024 ; Xarchakos and Koukopoulos 2025 optimized individually for each task. We provide extensive qualitative and quantitative analyses that validate the effectiveness of our unified approach and its ability to model garment–person interaction across diverse scenarios. 2 Related work Image-conditioned diffusion models. Diffusion models have rapidly evolved to support image-conditioned generation beyond the text-to-image setting Rombach et al. 2022 ; Podell et al. 2024 . Early methods such as Textual Inversion Gal et al. 2023 and DreamBooth Ruiz et al. 2023 enabled concept-specific synthesis via embedding injection or fine-tuning, but incurred high computational costs. Later approaches improved efficiency by leveraging pre-trained encoders like CLIP Radford et al. 2021 and DINO Caron et al. 2021 ; Oquab et al. 2024 , as seen in IP-Adapter Ye et al. 2023 and AnyDoor Chen et al. 2024c , which use image embeddings to guide generation. To improve spatial consistency, MasaCtrl Cao et al. 2023 introduced mutual self-attention to preserve object identity. This was extended by reference-based methods Hu 2024 ; Chen et al. 2024b ; Xu et al. 2025 ; Choi et al. 2024 ; Tian et al. 2024 that inject structured visual features through dedicated networks. More recently, concatenation-based methods Chong et al. 2025 ; Huang et al. 2024 ; Tan et al. 2024 have emerged as lightweight alternatives that directly feed conditioning signals without auxiliary modules. Additionally, image-conditioned diffusion has expanded beyond image-to-image generation to video Guo et al. 2023 ; Hu 2024 ; Blattmann et al. 2023 ; Shi et al. 2024 and 3D synthesis Liu et al. 2023 ; Kwak et al. 2024 ; Voleti et al. 2024 , broadening its applicability. Virtual try-on using generative models. Early virtual try-on systems primarily relied on warping-based methods Wang et al. 2018 ; Han et al. 2019 ; Ge et al. 2021a , where the main challenge was to deform and align the garment image to match the target person’s pose and body shape. With the emergence of Generative Adversarial Networks (GANs) Goodfellow et al. 2014 ; Karras et al. 2019 ; Karras et al. 2020 , a series of GAN-based approaches Han et al. 2018 ; Xie et al. 2021 ; Men et al. 2020 ; Issenhuth et al. 2020 ; Zhenyu et al. 2023 ; Bai et al. 2022 ; Ge et al. 2021b ; Yang et al. 2020 ; Choi et al. 2021 achieved more realistic image synthesis through adversarial learning. However, these models still struggled to preserve garment details and maintain structural consistency. With the advancement of large-scale text-to-image diffusion models Ramesh et al. 2021 ; Rombach et al. 2022 ; Saharia et al. 2022 ; Podell et al. 2024 ; Esser et al. 2024 ; Black Forest Labs 2024 , recent works Zhu et al. 2023 ; Baldrati et al. 2023 ; Morelli et al. 2023 ; Gou et al. 2023 ; Choi et al. 2024 ; Xu et al. 2025 ; Zeng et al. 2024 ; Shen et al. 2025 ; Chong et al. 2025 ; Zhou et al. 2025 have adopted pre-trained diffusion backbones as strong generative priors for virtual

Read the complete text