CSD-VAR: Content-Style Decomposition in Visual Autoregressive Models

Quang-Binh Nguyen, Minh Luu, Quang Nguyen, Anh Tran, Khoi Nguyen · 2025

Read CSD-VAR: Content-Style Decomposition in Visual Autoregressive Models by Quang-Binh Nguyen, Minh Luu, Quang Nguyen, Anh Tran, Khoi Nguyen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-13984.txt.

Introduction

Quang-Binh Nguyen and four co-authors present CSD-VAR, a 2025 method for splitting an image into its content and its style. Pulling the two apart lets a user place the same subject into a new scene or dress it in a new look. Most work of this kind targets diffusion models, while this team builds on visual autoregressive models, which paint an image scale by scale.

Their method trains content and style in turn, matched to the scales where each lives. A math fix stops content from leaking into the style side, and a memory store guards the subject's identity. They also built a test set of one hundred cases. Trials showed cleaner splits than earlier methods.

Catalog record

CREATORQuang-Binh Nguyen, Minh Luu, Quang Nguyen, Anh Tran, Khoi Nguyen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-13984

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

CSD-VAR: Content-Style Decomposition in Visual Autoregressive Models Quang-Binh Nguyen, Minh Luu, Quang Nguyen, Anh Tran, Khoi Nguyen Abstract Disentangling content and style from a single image, known as content-style decomposition (CSD), enables recontextualization of extracted content and stylization of extracted styles, offering greater creative flexibility in visual synthesis. While recent personalization methods have explored the decomposition of explicit content style, they remain tailored for diffusion models. Meanwhile, Visual Autoregressive Modeling (VAR) has emerged as a promising alternative with a next-scale prediction paradigm, achieving performance comparable to that of diffusion models. In this paper, we explore VAR as a generative framework for CSD, leveraging its scale-wise generation process for improved disentanglement. To this end, we propose CSD-VAR, a novel method that introduces three key innovations: (1) a scale-aware alternating optimization strategy that aligns content and style representation with their respective scales to enhance separation, (2) an SVD-based rectification method to mitigate content leakage into style representations, and (3) an Augmented Key-Value (K-V) memory enhancing content identity preservation. To benchmark this task, we introduce CSD-100, a dataset specifically designed for content-style decomposition, featuring diverse subjects rendered in various artistic styles. Experiments demonstrate that CSD-VAR outperforms prior approaches, achieving superior content preservation and stylization fidelity. † † † Qualcomm AI Research is an initiative of Qualcomm Technologies, Inc. 1 Introduction Figure 1 : Given a single input image, our framework separates content and style, enabling flexible recontextualization and stylization to generate new images across diverse contexts. The challenge of disentangling content and style, or content-style decomposition (CSD) , from a single image can be framed as a dual personalization problem : one that extracts the subject’s structure and details (content) while separately capturing the artistic technique (style). Successfully addressing this problem enables two key applications: recontextualization , where a subject is adapted to different visual environments, and stylization , where an extracted style is applied to new subjects. For instance, consider an artist working from a single reference – a balloon illustration rendered in a unique artistic style, illustrated in Fig. 1 (first row). Separating its content and style would allow the artist to depict the same balloon in a realistic setting or apply the illustration’s stylistic elements to new subjects, such as landscapes or portraits. This separation fosters greater creative flexibility, enabling novel compositions and cross-domain visual transformations. In text-to-image personalization, textual inversion [ 9 ] optimizes text embeddings to capture specific concepts within the text encoder space while keeping model parameters frozen. Many subsequent methods [ 50 , 31 , 56 , 15 , 19 , 58 , 7 , 20 ] have extended textual inversion to better preserve object identity and enhance performance across various tasks beyond encoding only a subject’s appearance. However, these approaches do not explicitly separate content and style. More recent methods, such as B-LoRA [ 8 ] and UnZipLoRA [ 26 ] , address this limitation by decomposing an image into distinct content and style representations. Nevertheless, these techniques are specifically designed for diffusion-based models, and no prior work has explored their application to other generative models, such as Autoregressive (AR) models. Recently, AR models [ 43 , 53 , 45 , 3 , 24 , 55 , 47 , 44 , 29 , 51 , 14 ] have emerged as a compelling alternative to diffusion models, offering comparable generative capabilities while maintaining high efficiency. A notable variant, Visual Autoregressive Modeling (VAR) [ 47 , 44 , 29 , 51 , 14 ] , introduces a next-scale prediction paradigm, where generation starts with a 1\times 1 token map and progressively expands into a sequence of multi-scale token maps (e.g., 2\times 2 , …), increasing in resolution. This motivates our investigation into how content and style representations can be effectively learned within VAR. However, directly applying textual inversion to VAR by optimizing separate style and content embeddings results in suboptimal representations due to the strong entanglement between these attributes. The inherent coupling of content and style makes simple text prompt guidance insufficient for effective decomposition. First, we empirically observe that early scales primarily encode style, while later scales capture content information. Building on this insight, we propose a scale-aware alternating optimization strategy, where content and style embeddings are optimized at their corresponding scales in an alternating manner to ensure better disentanglement. Second, to further strengthen content-style separation, we introduce a singular value decomposition (SVD)-based rectification method , which explicitly enforces independence between the content and style spaces, reducing mutual interference and enhancing the quality of learned representations. Third, relying solely on textual embeddings proves inadequate for capturing complex concepts and styles, particularly in intricate cases. To address this limitation, we introduce augmented Key-Value (K-V) memory , which serves as auxiliary storage for content and style attributes that textual inversion alone fails to capture. These augmented K-V matrices not only improve content-style disentanglement but also enhance identity preservation, ensuring more faithful and expressive representations. To our knowledge, no publicly available dataset with quantifiable benchmarks exists for content-style decomposition (CSD). While existing datasets focus on either style transfer or content preservation, they do not fully meet the requirements for evaluating CSD, prompting us to introduce CSD-100, a dataset of 100 images designed specifically for this task. On CSD-100 , our CSD-VAR outperforms state-of-the-art by a large margin in all common metrics. In summary, our main contribution includes: • We are the first to explore VAR-based personalization for content-style decomposition. • We analyze scale-dependent representations in VAR and propose a scale-aligned optimization strategy to improve content-style disentanglement. • We introduce an SVD-based constraint to enforce orthogonality between content and style representations. • We propose augmented K-V memory to enhance disentanglement and better preserve subject identity. • We propose a new dataset, CSD-100, specifically designed for the CSD with various styles and subjects. 2 Related Work Text-to-image generative models. Early text-to-image generation methods built on GANs [ 12 ] , conditioning on textual inputs to map embeddings into image space, but suffered from training instability and mode collapse. The introduction of diffusion models (DMs) [ 35 , 36 , 39 ] improved image quality and diversity by leveraging large-scale image-text datasets and iterative denoising. However, DMs are computationally expensive due to slow sequential inference. To mitigate this, recent works [ 28 , 30 , 40 ] distill multi-step teacher models into efficient few-step student networks, with some methods [ 5 , 27 , 32 , 57 ] achieving one-step text-to-image generation. Autoregressive (AR) models offer an alternative to diffusion-based approaches. Earlier AR models generated images sequentially via next-token prediction [ 43 , 53 , 45 , 3 ] , but their step-by-step nature made high-resolution generation slow. Recently, next-scale prediction AR has emerged as a more efficient paradigm, generating images progressively from low to high resolution [ 47 , 44 , 29 , 51 , 14 ] , refining details at each scale while attending to previous ones. This approach has been successfully applied to text-to-image generation: STAR [ 29 ] and Switti [ 51 ] use cross-attention for text conditioning, HART [ 44 ] improves VAR [ 47 ] with a diffusion model for continuous error residuals, and Infinity [ 14 ] enables fine-grained generation with bitwise token prediction. While

Read the complete text