Attention, May I Have Your Decision? Localizing Generative Choices in Diffusion Models

Katarzyna Zaleska*, Łukasz Popek, Monika Wysoczańska, Kamil Deja · 2026

Read Attention, May I Have Your Decision? Localizing Generative Choices in Diffusion Models by Katarzyna Zaleska*, Łukasz Popek, Monika Wysoczańska, Kamil Deja (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-06052.txt.

Introduction

Zaleska and three colleagues ask where text-to-image diffusion models settle details a prompt leaves unspecified. They test whether that decision-making sits in particular layers rather than spreading everywhere. Using a probing technique that scores layers by how cleanly they separate attributes, they find self-attention layers resolve ambiguous concepts. Those layers are therefore the best place to intervene. Their 2026 steering method, ICM, targets a small subset of those layers. Experiments show it debiases better than leading methods while producing fewer artifacts.

Catalog record

CREATORKatarzyna Zaleska*, Łukasz Popek, Monika Wysoczańska, Kamil Deja
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48795
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.06052

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Attention, May I Have Your Decision? Localizing Generative Choices in Diffusion Models Katarzyna Zaleska*, Łukasz Popek, Monika Wysoczańska, Kamil Deja Abstract Text-to-image diffusion models exhibit remarkable generative capabilities, yet their internal operations remain opaque, particularly when handling prompts that are not fully descriptive. In such scenarios, models must make implicit decisions to generate details not explicitly specified in the text. This work investigates the hypothesis that this decision-making process is not diffuse but is computationally localized within the model’s architecture. While existing localization techniques focus on prompt-related interventions, we notice that such explicit conditioning may differ from implicit decisions. Therefore, we introduce a probing-based localization technique to identify the layers with the highest attribute separability for concepts. Our findings indicate that the resolution of ambiguous concepts is governed principally by self-attention layers, identifying them as the most effective point for intervention. Based on this discovery, we propose ICM (Implicit Choice-Modification) – a precise steering method that applies targeted interventions to a small subset of layers. Extensive experiments confirm that intervening on these specific self-attention layers yields superior debiasing performance compared to existing state-of-the-art methods, minimizing artifacts common to less precise approaches. The code is available at https://github.com/kzaleskaa/icm . † † footnotetext: *Corresponding author: [email protected] 1 Introduction Figure 1 : We use linear probes to localize layers with the highest attribute separability for concepts not present in the prompt (e.g., a color when prompting for “ a photo of flowers ”). We show that we can use the learned probes to steer towards less probable outcomes. Text-to-image (T2I) diffusion models ( 31 ) represent a major advance in AI, yet their internal mechanisms remain largely opaque, turning the whole process into a black-box. This opacity becomes particularly evident when text prompts lack specificity, forcing the model to make generative choices – filling in missing details of the intended generation based on patterns learned from training data. Often, these implicit decisions are benign; for instance, when prompted with “a photo of flowers” the model must infer a color, shape, and background from the random noise to generate an image. However, the same underlying mechanism can lead to more problematic outcomes. These range from perpetuating social biases, such as defaulting to men for professional roles, to representational skews where a concept like ’USA President’ consistently resolves to a single person, like Donald Trump. In this work, we want to understand where and how a diffusion model decides on how to instantiate a general idea as presented in Figure 1 . The most common strategy for localizing knowledge in T2I models 2 ; 1 ; 37 ; 27 is based on the idea of prompt injection, where a different text prompt is injected only in the selected layers to measure their impact on the output. Such precise localization unlocks a wide range of applications starting from direct concept editing or removal ( 8 ; 25 ) , through precise finetuning 36 ; 7 , reduction in computational resources ( 23 ) up to prevention of harmful content generation ( 37 ) . However, those prior works focus mostly on tracing concepts explicitly mentioned in the prompt, showing that their generation can be logically tied to the cross-attention layers responsible for integrating text and image representations. While reasonable for the analysis of objects or style localization, injecting attributes into prompts masks the internal mechanism the model uses when the prompt does not provide an answer. In this work, we posit that the mechanism for implicit decision choices is computationally localized and distinct from the mechanism used for explicit text conditioning. To validate this, we introduce a probing-based localization technique that avoids the confounding effects of prompt engineering. Instead of injecting attributes into the input text, we generate images using underspecified prompts (e.g., ’a photo of a person’) and retrospectively label the output attributes using an external classifier. Given such pseudo-labels, we train linear probes on the intermediate activations to quantify layer-wise discriminability. This allows us to pinpoint exactly where the model’s internal representation becomes linearly separable, identifying the layer at which the implicit decision about the selected attribute is most pronounced. Our localization technique reveals an unexpected finding. We notice that activations right after self-attention layers enable more precise linear separation between different instantiations of the same concept than activations after cross-attention layers, which are known to handle explicit prompts. This suggests that self-attention layers are responsible for discriminating between possible implicit options. To demonstrate the utility of our layer selection method, we apply it to debiasing in diffusion models. We build upon prior work that has addressed this task primarily through finetuning 33 ; 10 or activation steering 18 ; 26 ; 35 , but propose to employ them exclusively to selected subset of layers for better precision. By intervening only in a subset of chosen layers, we achieve the state-of-the-art steering performance. We demonstrate that we can mitigate biases in gender, age, and race while minimizing the artifacts and quality degradation common in broader, less targeted interventions. Finally, we examine our localization technique on larger diffusion models with different architectures: the UNet-based SDXL 28 and the Transformer-based SANA 38 . Our contributions can be summarized as follows: • We show that with linear probing, we can localize the most important layers for changing implicit decisions in text-to-image diffusion models • We highlight the important role self-attention layers play in the instantiation of general prompts into specific images, exhibiting higher attribute separability compared to cross-attention layers. • We show that by focusing on the selected layers, we can improve the performance of debiasing with steering or fine-tuning. 2 Related work In this section, we discuss literature related to our work across three main areas: understanding the decision-making process during generation (Section 2.1 ), layer specialization and localization in diffusion models (Section 2.2 ), and debiasing approaches for generative models as a practical application of aforementioned techniques (Section 2.3 ). 2.1 Decision process in Diffusion models In this work, we analyze how diffusion models make implicit decisions when prompts underspecify certain attributes, requiring the model to infer specific instantiations from initial noise. It has been demonstrated, however, that this initial noise significantly impacts generation outcomes 9 . From a text prompting perspective, several works have investigated prompt-to-image relationships, with 16 establishing links between specific prompt elements and their visual manifestation in generated images. Magnet 40 and Cat/Dog 6 study the issues observed in explicit bindings between prompts and the outputs, while methods like Attend-and-Excite 5 or 20 intervene in the cross-attention maps to mitigate the neglect of explicitly prompted subjects. Similarly, decisions on the semantic objects provided by cross-attention layers are used in the case of Prompt-to-Prompt 11 and Ledits++ 3 to perform a precise edition. Finally, 23 shows that information is introduced into generated images through cross-attention layers, especially during initial denoising steps. 2.2 Layers specialization in diffusion models Our work is related to a growing body of work examining how different types of knowledge are distributed across layers in diffusion models. For example, Kwon et al. 17 show that the UNet’s bottleneck can provide semantic representations for the diffusion models. Furthermore, works by 2 ; 1 ; 22 examining UNets, demonstrated that cross-attention

Read the complete text