Storybooth: Training-free multi-subject consistency for improved visual storytelling
Jaskirat Singh, Junshen K. Chen, Jonas Kohler, Michael Cohen · 2025
Read Storybooth: Training-free multi-subject consistency for improved visual storytelling by Jaskirat Singh, Junshen K. Chen, Jonas Kohler, Michael Cohen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-05800.txt.
Introduction
Jaskirat Singh, Junshen Chen, Jonas Kohler, and Michael Cohen wrote this machine learning paper, published in 2025. It improves picture-story generation, where the same characters must look like themselves across many images, with no extra training.
Most training-free methods let each frame's parts attend to other frames, which works for one character. With several, the parts leak into each other, and one character starts wearing another's face. The authors trace the failure to this attention leakage. Their system, Storybooth, first reasons with a multimodal model to place each subject in its own region of every frame. Generation then uses a bounded attention layer that limits cross-character mixing, plus a token merging step for fine detail. The result keeps casts of characters distinct and consistent across a story's panels.
Catalog record
| CREATOR | Jaskirat Singh, Junshen K. Chen, Jonas Kohler, Michael Cohen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-05800
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Storybooth: Training-free multi-subject consistency for improved visual storytelling Jaskirat Singh, Junshen K. Chen, Jonas Kohler, Michael Cohen Abstract Training-free consistent text-to-image generation depicting the same subjects across different images is a topic of widespread recent interest. Existing works in this direction predominantly rely on cross-frame self-attention; which improves subject-consistency by allowing tokens in each frame to pay attention to tokens in other frames during self-attention computation. While useful for single subjects, we find that it struggles when scaling to multiple characters. In this work, we first analyze the reason for these limitations. Our exploration reveals that the primary-issue stems from self-attention leakage , which is exacerbated when trying to ensure consistency across multiple-characters. This happens when tokens from one subject pay attention to other characters, causing them to appear like each other ( e.g . , a dog appearing like a duck) . Motivated by these findings, we propose StoryBooth: a training-free approach for improving multi-character consistency. In particular, we first leverage multi-modal chain-of-thought reasoning and region-based generation to apriori localize the different subjects across the desired story outputs. The final outputs are then generated using a modified diffusion model which consists of two novel layers: 1) a bounded cross-frame self-attention layer for reducing inter-character attention leakage, and 2) token-merging layer for improving consistency of fine-grain subject details. Through both qualitative and quantitative results we find that the proposed approach surpasses prior state-of-the-art , exhibiting improved consistency across both multiple-characters and fine-grain subject details. 1 Introduction The field of consistent text-to-image generation aims at depicting the visually consistent subjects across diverse settings, and, has gained significant recent attention ( Li et al. 2024a ; Ye et al. 2023 ; Ruiz et al. 2022 ) with the advent of diffusion models ( Rombach et al. 2021 ; Peebles & Xie 2022 ) . The generated subject-consistent images (traditionally referred to as the storyboard ( Truong et al. 2006 ) ) can be used as visual inputs in order to augment visual storytelling, or, serve as anchor images for improving consistency for multiple-shot video generation ( Zhao et al. 2024 ; Zhou et al. 2024 ) . Recent works on ensuring consistency ( e.g . , identity, appearance, background etc . ) between different storyboard frames predominantly rely on subject personalization ( Feng et al. 2023 ; Jeong et al. 2023 ; Avrahami et al. 2024 ) . However, subject personalization methods often require per-subject training ( Ruiz et al. 2022 ; Gal et al. 2022 ) and are observed to struggle when scaling to multiple characters ( Ye et al. 2023 ) . Furthermore, they can suffer from trade-offs between subject consistency and prompt-alignment. Training autoregressive models for consistent storyboard generation ( Liu et al. 2024 ; Zhao et al. 2024 ; Yang et al. 2024b ) has also been explored. However, this requires expensive training (often on curated movie / animation data) and the resulting methods tend to overfit on the training domain ( Yang et al. 2024b ) thereby limiting their practical efficacy for general storytelling. In this paper, we explore a simple yet effective training and optimization-free approach towards improving storyboard consistency. Recent works in this direction ( Zhou et al. 2024 ; Tewel et al. 2024 ) utilize cross-frame self-attention which improves cross-frame consistency by allowing each token to pay attention to tokens from all other storyboard frames. While good for ensuring single character consistency, experiments reveal that this tends to exacerbate the self-attention leakage problem ( Dahary et al. 2024 ) when scaling to multiple characters (refer Sec. 3 ). For instance, in Fig. 1 , we observe that when generating a storyboard with two characters: dog and duck , the inter-character leakage results in significantly reduced prompt alignment with the duck appearing like a dog . Furthermore, while the use of cross-frame self-attention helps ensure general appearance similarity, the fine-grain features ( e.g ., face and ears of the dog in Fig. 1 ) can have observable inconsistencies. To address these limitations, we propose Storybooth; a training-free approach for improving consistency across both 1) multiple-characters and 2) fine-grain subject details. The core idea of our approach is to first use region-based planning and generation ( Yang et al. 2024a ) to apriori localize the region / layout of different characters across the storyboard (Sec. 4.1 ). Once the subject regions have been identified, we then propose a 1) a bounded cross-frame self-attention layer which reduces inter-character leakage by limiting each character to only pay attention to tokens corresponding to other regions for the same character ( e.g . , dog only paying attention to other tokens for the same dog). Furthermore, we also propose a novel 2) cross-frame token-merging layer which allows for improved fine-grain consistency ( e.g . , head of dog in Fig. 1 ) by merging matching low-level feature tokens across different generations. Despite its training-free nature, experimental analysis reveals (Sec. 5 ) that the proposed approach allows for improved character consistency and text-to-image alignment while exhibiting \times 30 faster inference time than optimization-based methods ( Ruiz et al. 2022 ) . To summarize, the key contributions of this paper are: 1) we highlight the problem of multi-character consistency and exacerbated self-attention leakage when sharing self-attention across the storyboard ( Zhou et al. 2024 ; Tewel et al. 2024 ) 2) we propose the idea of combining region-based generation with a novel bounded self-attention layer , for reducing inter-character leakage (Fig. 5 ). 3) Finally, we also propose a cross-frame token-merging layer which allows for improved fine-grain consistency ( e.g . , head of dog in Fig. 1 ) by merging matching low-level feature tokens across different generations. Figure 1: Overview. We propose a training and optimization-free approach for improving consistency across both multiple characters ( dog and duck: left ) and fine-grain subject details ( face of dog: right ). 2 Related Work Consistent text-to-image generation has been a topic of great recent interest due to its applications in visual storytelling ( Liu et al. 2024 ; Zhou et al. 2024 ; Yang et al. 2024b ) and multiple-shot video generation ( Zhou et al. 2024 ; Zhao et al. 2024 ; Yuan et al. 2024 ) . Prior works on ensuring consistency typically reply on subject personalization ( Feng et al. 2023 ; Jeong et al. 2023 ; Avrahami et al. 2024 ) . This includes inference-time optimization methods ( e.g . , dreambooth ) ( Ruiz et al. 2022 ; Gal et al. 2022 ) which requires expensive optimization for each subject. Encoder-based methods ( Wei et al. 2023 ; Li et al. 2024a ) have also been explored, but they require large-scale training and fail to scale to multiple characters. Recent works have also explored training-autoregressive models ( Liu et al. 2024 ; Zhao et al. 2024 ; Yang et al. 2024b ) for storyboard generation. However, this again requires expensive training and the resulting methods are observed to overfit on the training domain. More recently, ( Zhou et al. 2024 ; Tewel et al. 2024 ) propose a training-free approach which ensures storyboard consistency through the use of consistent self-attention. As seen in Fig. 3 , while