FreeText: Training-Free Text Rendering in Diffusion Transformers via Attention Localization and Spectral Glyph Injection

Ruiqiang Zhang, Hengyi Wang, Chang Liu, Guanjie Wang, Zehua Ma, Weiming Zhang · 2026

Read FreeText: Training-Free Text Rendering in Diffusion Transformers via Attention Localization and Spectral Glyph Injection by Ruiqiang Zhang, Hengyi Wang, Chang Liu, Guanjie Wang, Zehua Ma, Weiming Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-00535.txt.

Description

Ruiqiang Zhang and colleagues fix a familiar weakness of text-to-image models: rendering written words inside pictures. Diffusion models draw striking scenes but garble multi-line layouts, dense lettering, and scripts like Chinese. Retraining is expensive, and rigid layout tools can spoil the look. Their framework, FreeText, needs no training and plugs into existing diffusion Transformers.

The method splits the job into deciding where to write and what to write. Attention patterns inside the model reveal likely writing regions, which are refined into clean masks using stable anchor tokens. A spectral glyph injection step then shapes the actual letters. By working through the model's own mechanisms, FreeText aims for accurate text that still blends naturally with the generated image.

Catalog record

CREATORRuiqiang Zhang, Hengyi Wang, Chang Liu, Guanjie Wang, Zehua Ma, Weiming Zhang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE30 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-00535

Browse the full library · About the Tower of Babel