StyleTextGen: Style-Conditioned Multilingual Scene Text Generation

Zeyu Chen, Fangmin Zhao, Yan Shu, Yichao Liu, Yu Zhou · 2026

Read StyleTextGen: Style-Conditioned Multilingual Scene Text Generation by Zeyu Chen, Fangmin Zhao, Yan Shu, Yichao Liu, Yu Zhou (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-14708.txt.

Introduction

Chen, Zhao, Shu, Liu, Yu and Zhou present StyleTextGen, a framework for generating scene text that copies a reference style across languages and writing systems. A dual-branch style encoder models style in messy real scenes, a style consistency loss keeps characters coherent, and a mask-guided inference step aligns generated text with the reference. A new bilingual benchmark, StyleText-CE, covers monolingual and cross-lingual settings, with claimed state-of-the-art style consistency.

Catalog record

CREATORZeyu Chen, Fangmin Zhao, Yan Shu, Yichao Liu, Yu Zhou
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34220
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.14708

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

StyleTextGen: Style-Conditioned Multilingual Scene Text Generation Zeyu Chen, Fangmin Zhao, Yan Shu, Yichao Liu, Yu Zhou Abstract Style-conditioned scene text generation faces unique challenges in extracting precise text styles from complex backgrounds and maintaining fine-grained style consistency across characters, especially for multilingual scripts. We propose StyleTextGen, a novel framework that learns to perceive and replicate visual text styles across different languages and writing systems. Our approach features three key contributions: First, we introduce a dual-branch style encoder dedicated to style modeling, yielding robust multilingual text style representations in complex real-world scenes. Second, we design a text style consistency loss that enhances style coherence and improves overall visual quality. Third, we develop a mask-guided inference strategy that ensures precise style alignment between generated and reference text. To facilitate systematic evaluation, we construct StyleText-CE, a bilingual scene text style benchmark covering both monolingual and cross-lingual settings. Extensive experiments demonstrate that StyleTextGen significantly outperforms existing methods in style consistency and cross-lingual generalization, establishing new state-of-the-art performance in multilingual style-conditioned text generation. 1 Introduction Figure 1 : Examples of our StyleTextGen for style-conditioned scene text generation. Left: self-style reference (in-place generation). Right: external-style reference. For each example, the reference style image and the generated target text are shown at the bottom of the panel. Text-to-image generation 32 ; 20 has witnessed remarkable progress in synthesizing photorealistic images from textual descriptions. However, generating scene text with precise style control remains a fundamental challenge. The task of style-conditioned scene text generation aims to synthesize text in images while faithfully replicating the visual style from reference example, including font characteristics, stroke pattern, color gradient and texture detail. This capability is crucial for real-world applications such as multilingual signage design and advertising poster creation, and also benefit scene text detection, recognition, and spotting 60 ; 8 ; 7 ; 51 ; 28 ; 2 . Unlike semantic content that can be specified through text prompts, these nuanced visual attributes require visual style references, making image-conditioned generation a natural solution for this task. Despite its importance, style-conditioned scene text generation faces unique technical challenges that distinguish it from general image synthesis. First, extracting text-specific style features from complex scene images is non-trivial—reference images often contain cluttered backgrounds, varying lighting conditions, and diverse text layouts that can confound style perception. Existing scene text editing methods 56 ; 9 typically reuse styles from the original in-place text, limiting their ability to adopt external style references. Recent advances such as Calligrapher 30 attempt free-style text generation, but their style extraction mechanisms lack explicit content awareness, leading to style-content entanglement and degraded performance when processing complex backgrounds. Second, maintaining fine-grained style consistency across all characters becomes substantially more challenging in multilingual scenarios, where different writing systems (e.g., Latin, Chinese, Arabic) exhibit distinct typographic structures and varying numbers of strokes. The lack of robust cross-lingual style transfer mechanisms in current methods results in inconsistent character appearance and structural distortions when generating text in languages different from the reference style. To address these challenges, we propose StyleTextGen, a novel style-conditioned framework for multilingual scene text generation. First, we design a dual-branch style encoder dedicated to modeling text style. A text-focused branch pretrained on multilingual text segmentation captures fine-grained typographic features, while a visual prior branch extracts holistic scene information. Their fusion yields robust, text-aware style embeddings that remain stable across complex backgrounds and diverse languages. Second, we propose a text style consistency loss that enforces style alignment specifically within text regions. This mask-guided loss promotes uniform style attributes across all generated characters, significantly improving visual coherence in multilingual scenarios. Third, we introduce a mask-guided inference strategy that extracts style features from reference images through mask-based inversion and injects them into the generation process. This refinement ensures precise style correspondence between reference and generated text across different writing systems. To enable systematic evaluation, we construct StyleText-CE, a bilingual benchmark supporting four settings: self-style reference, external-style reference, monolingual generation, and cross-lingual transfer. As shown in Figure 1 , StyleTextGen accurately transfers text styles while maintaining readability across diverse languages and styles. Our contributions are summarized as follows: • We propose a dual-branch style encoder that robustly captures style features from multilingual text, combined with a text style consistency loss to enhance style coherence and visual consistency. • We design an inference strategy that enhances style alignment between generated scene text and reference, improving fine-grained style consistency across characters. • We construct a bilingual scene text style benchmark with monolingual and cross-lingual settings, demonstrating superior performance in style consistency and cross-lingual generalization. 2 Related Work 2.1 Scene Text Generation Traditional text-to-image diffusion models 39 ; 31 ; 36 ; 35 ; 41 ; 40 ; 53 , struggle to render legible and semantically correct text in complex scenes. This limitation has motivated a line of work on dedicated scene text generation. Methods such as DiffSTE 13 , UDiffText 59 , GlyphByT5 26 , and SceneTextGen 58 improve textual representation through character-level encoders. Apart from these, AnyText 44 , GlyphDraw2 29 , GlyphOnly 24 , TextDiffuser 5 , TextDiffuser2 4 , and GlyphControl 52 leverage glyph priors and layout masks to enhance structural fidelity and spatial controllability. Focusing more specifically on style-conditioned scene text generation, GlyphByT5 26 models partial font and color information within its text encoder and pre-rendered glyph space, enabling coarse-grained control over the appearance of generated text. AnyText2 43 further introduces explicit font and color encoders to provide additional style-related guidance during rendering. However, these approaches are typically restricted to relatively simple fonts and often struggle to synthesize artistic typography or faithfully follow a given style reference. To address these limitations, we propose StyleTextGen, which generates scene text conditioned on arbitrary text-style reference images. 2.2 Scene Text Editing Scene text editing aims to modify the textual content in an image while preserving the background and maintaining style consistency. Many methods explicitly separate foreground text from background. SRNet 48 and STEFANN 38 use dedicated modules for background inpainting and text re-rendering. SwapText 50 extends SRNet with a thin-plate spline interpolation network to handle curved text. TextStyleBrush 18 adopts a StyleGAN2 17 based self-supervised framework to learn transferable text styles, and MOSTEL 33 employs a semi-supervised training scheme to improve robustness in the wild. STEEM 61 minimizes background reconstruction error to explicitly decouple style and content, leading to higher editing fidelity. Recent method TextCtrl 56 pre-train a style encoder to enforce style consistency, while GlyphMastero 47 fuses local character-level features with global text-line structures for editing complex Chinese characters. Overall, these methods focus on preserving background information and modifying text content. Although some of them consider text style, they cannot generate text that follows arbitrary external style references. StyleTextGen addresses this limitation by enabling scene text generation conditioned on any given style reference image. 2.3 Image style transfer Image style transfer aims to apply the style of a reference image to a content image. Early neural style transfer methods 10 formulate this problem as matching deep feature statistics, and subsequent feed-forward approaches 14 ; 45 enable efficient fast stylization. AdaIN 12 supports arbitrary style transfer by aligning feature statistics, while style-based generative models such as StyleGAN 16 ; 17 further improve style controllability. More recent work leverages vision–language

Read the complete text