LayerTracer: Cognitive-Aligned Layered SVG Synthesis via Diffusion Transformer CCS: Computing methodologies Computer vision

Yiren Song, Danze Chen, Mike Zheng Shou · 2025

Read LayerTracer: Cognitive-Aligned Layered SVG Synthesis via Diffusion Transformer CCS: Computing methodologies Computer vision by Yiren Song, Danze Chen, Mike Zheng Shou (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-01105.txt.

Introduction

Song, Chen, and Shou built this vector graphics generator in twenty twenty-five. Designers build SVG images layer by layer, but AI tools often emit one flat layer or messy, redundant shapes. Their system, LayerTracer, learns from recordings of real design work. It copies the step-by-step way designers construct an image.

First, a diffusion transformer sketches the build as a series of raster stages from a text prompt. Then each stage is traced into clean vector paths, with duplicates removed. For photos, a conditioning trick feeds the image in as guide tokens. Tests beat both optimization-based and neural rivals on quality and editability. The authors say the output matches how professionals actually think.

Catalog record

CREATORYiren Song, Danze Chen, Mike Zheng Shou
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE30 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-01105

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

LayerTracer: Cognitive-Aligned Layered SVG Synthesis via Diffusion Transformer CCS: Computing methodologies Computer vision Yiren Song, Danze Chen, Mike Zheng Shou Abstract . Generating cognitive-aligned layered SVGs remains challenging due to existing methods’ tendencies toward either oversimplified single-layer outputs or optimization-induced shape redundancies. We propose LayerTracer, a diffusion transformer based framework that bridges this gap by learning designers’ layered SVG creation processes from a novel dataset of sequential design operations. Our approach operates in two phases: First, a text-conditioned DiT generates multi-phase rasterized construction blueprints that simulate human design workflows. Second, layer-wise vectorization with path deduplication produces clean, editable SVGs. For image vectorization, we introduce a conditional diffusion mechanism that encodes reference images into latent tokens, guiding hierarchical reconstruction while preserving structural integrity. Extensive experiments demonstrate LayerTracer’s superior performance against optimization-based and neural baselines in both generation quality and editability, effectively aligning AI-generated vectors with professional design cognition. Code is released at https://github.com/showlab/LayerTracer Figure 1 . LayerTracer enables the creation of cognitive-aligned layered SVGs, either from text prompts or by converting images into layer-wise SVGs. . 1. Introduction Scalable Vector Graphics (SVG) serves as the cornerstone of modern digital design, defining visual elements—paths, curves, and geometric shapes—through mathematical equations rather than pixel grids. Unlike raster images, SVG maintains resolution-independent clarity at any scale, making it indispensable for precision-critical applications ranging from UI/UX design to industrial CAD systems. Layered SVGs, which adhere to professional standards, elevate this advantage further: designers can meticulously manipulate individual layers to refine stroke textures, spatial hierarchies, and composite effects. This editability transcends mere convenience—it forms the backbone of dynamic adjustments and collaborative iteration in contemporary design workflows. Nevertheless, a significant gap persists between current deep learning-based SVG generation techniques and professional requirements. Existing approaches face three systemic challenges: First, the scarcity of large-scale layered SVG datasets forces models to rely on synthetic or oversimplified training data, resulting in outputs devoid of the nuanced hierarchical structures inherent to human designs. Second, methodological fragmentation prevails—optimization-based methods ( Jain et al. 2023 ; Xing et al. 2024b ; Xing et al. 2023 ; Ma et al. 2022 ; Li et al. 2020 ; Hirschorn et al. 2024 ; Wang et al. 2024b ) generate vector paths using raster priors but often produce cluttered geometries with redundant anchor points; and large language models (LLMs) ( Xing et al. 2024a ; Tang et al. 2024 ; Rodriguez et al. 2023 ) , constrained by token limits, remain limited to basic icons. Most critically, no existing method addresses the designer’s cognitive process—the logical sequencing, spatial reasoning, and element grouping strategies employed during layer construction—resulting in AI-generated SVGs that resemble fragmented collages rather than intentionally editable professional designs. To address these challenges, we present LayerTracer, a Diffusion Transformer (DiT)-based framework that redefines layered SVG synthesis by modeling designers’ layer-by-layer construction logic. Our approach is grounded in three key insights: (1). Cognitive alignment: DiT models pretrained on text-image corpora inherently capture contextual relationships between visual elements, which can be steered through targeted fine-tuning to mimic designer decision-making. (2). Spatiotemporal consistency: The self-attention mechanism’s bias toward local token interactions—a byproduct of training on natural image pixel correlations—can be repurposed to enforce coherence across sequential design steps. 3. Structured decomposition: Disassembling layered SVGs into channel-wise components and organizing them as grid sequences provides generation models with an interpretable blueprint of layer evolution. In implementation, LayerTracer integrates two innovations. First, we curate a pioneering dataset of 20,000+ designer process traces, automatically converting layered SVGs into timestamped creation sequences. These sequences are rasterized and organized into training grids using a serpentin layout, ensuring temporally adjacent design steps remain spatially proximate. Second, we develop a dual-phase generation pipeline: (1) a text-conditioned DiT generates rasterized construction process sequences that simulate a designer’s workflow, followed by (2) a layer-wise vectorization module that converts these sequences into clean, editable SVG layers while eliminating redundant paths. Beyond text-to-SVG synthesis, LayerTracer tackles the inverse task: converting raster images into layered vector graphics. We reframe this as a process-conditioned generation problem, where reference images guide the model to "reverse-engineer" plausible layer construction steps. Specifically, we build upon a pretrained DiT model and adapt it through LoRA fine-tuning to ingest image context. By encoding reference images into conditional tokens injected into the denoising process, the model autonomously deduces layer assembly sequences (e.g., "background first, then foreground elements"), faithfully reconstructing input images while adhering to practical editing constraints. Our main contributions are as follows: • Cognitive-aligned SVG synthesis: As the first framework to generate layered SVGs by learning designers’ construction logic—element ordering, layer grouping, and spatial reasoning—LayerTracer ensures outputs meet professional editing standards. • Unified DiT-based architecture: Our framework seamlessly integrates text-to-SVG generation and layer-wise vectorization tasks, eliminating the need for task-specific pipelines. • Process-centric dataset: We release a scalable pipeline for collecting designer workflow data, addressing the critical gap in layered vector graphics training resources. Extensive experiments validate LayerTracer’s state-of-the-art performance and effectiveness. 2. Related Works 2.1. Text2image Diffusion Model Recent studies have demonstrated that diffusion models are capable of generating high-quality synthetic images, effectively balancing diversity and fidelity. Models based on diffusion models or their variants, such as those paper in ( Rombach et al. 2022 ; Peebles and Xie 2023 ) , have successfully addressed the challenges associated with text-conditioned image synthesis. Stable Diffusion ( Rombach et al. 2022 ) , a model based on the Latent Diffusion Model, incorporates text conditioning within a UNet framework to facilitate text-based image generation ( Song et al. 2024b ; Song et al. 2024d ; Song et al. 2024c ) , establishing itself as a mainstream model in image generation. Fine-tuning pre-trained image generation models can enhance their adaptation to specific application scenarios, as seen in techniques like LoRA ( Hu et al. 2021 ) and DreamBooth ( Poole et al. 2022 ) . For theme control in text-to-image generation, several works ( Ye et al. 2023 ; Wang et al. 2024a ; Zhang et al. 2024a ; Zhang et al. 2024b ; Zhang et al. 2024c ; Zhang et al. 2024d ) focus on custom generation for defined pictorial concepts, with ControlNet ( Zhang et al. 2023 ) additionally offering control over other modalities such as depth information. AnimateDiff ( animatediff ) introduces a temporal attention module, extending Stable Diffusion into a video generation model. Inspired by ProcessPainter ( Song et al. 2024a ) , which first proposed learning an artist’s painting process through pre-trained temporal models, this paper leverages the in-context capabilities of DiT to generate a layered SVG creation process. Figure 2 . The LayerTracer architecture comprises three key components: (1) Layer-wise Model : Pretrained on our proposed dataset to generate layered pixel sequences from text prompt; (2) Image2Layers Model : Merges LoRA with the Flux base DiT, enabling image-conditioned generation through VAE-encoded latent tokens; (3) Layer-wise Vectorization : Converts raster sequences to SVGs via differential analysis between adjacent layers, followed by Bézier optimization using vtracer to eliminate redundant paths while preserving structural fidelity. 2.2. SVG Generation Scalable Vector Graphics (SVGs) are widely utilized in design owing to their advantages like geometric manipulability, resolution independence, and compact file structure. SVG generation often involves training neural networks to produce predefined SVG commands and attributes using architectures

Read the complete text