Neuron-Level Interventions for Gendered and Gender-Neutral Generation in Language Models

Zhiwen You, Nafiseh Nikeghbal, Jana Diesner, [0.7ex] University of Illinois Urbana-Champaign · 2026

Read Neuron-Level Interventions for Gendered and Gender-Neutral Generation in Language Models by Zhiwen You, Nafiseh Nikeghbal, Jana Diesner, [0.7ex] University of Illinois Urbana-Champaign (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-30717.txt.

Introduction

You, Nikeghbal, and Diesner study gender-specific neurons across feminine, masculine, and neutral categories. Prior work views gender as binary, with little attention to neutral forms like they pronouns. Their neuron-level method finds neurons tied to each gender category. Tests follow: activating or masking gender neurons steers a sentence toward a gender form while preserving meaning. Experiments on two open models show gender neurons concentrate in the earliest layers, with smaller contributions later. Their method achieves tighter gender control, with less leakage into non-target categories.

Catalog record

CREATORZhiwen You, Nafiseh Nikeghbal, Jana Diesner, [0.7ex] University of Illinois Urbana-Champaign
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62166
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.30717

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Neuron-Level Interventions for Gendered and Gender-Neutral Generation in Language Models Zhiwen You, Nafiseh Nikeghbal, Jana Diesner, [0.7ex] University of Illinois Urbana-Champaign Abstract Language models (LMs) can produce gendered language and stereotypes even when given neutral prompts. Most prior work on gender bias in LMs primarily examines gender through a binary lens (feminine vs. masculine), with limited attention to gender-neutral forms, such as they/them pronouns or neutrally phrased job titles. How gender-related signals are encoded in the internal representations of LMs remains an open question. In this work, we study gender-specific neurons in LMs across three categories: feminine, masculine, and gender-neutral. We propose a neuron-level intervention method to identify neurons that are strongly tied to each gender category. We then test these neurons through controlled generation, showing that activating or masking gender-related neurons can steer a sentence toward a target gender form while preserving its original meaning. To evaluate the effectiveness of our gender-intervention approach, we curate two datasets with controlled sentences labeled across all three gender categories and validate the data quality through human evaluation. Experiments on two open-source LMs show that gender-specific neurons are not evenly distributed across model layers; instead, they concentrate heavily in the earliest layers with smaller contributions from later layers. Compared to existing methods, our method achieves more precise gender control, with less leakage into non-target gender categories and stable output quality through two evaluation criteria. Overall, our work examines how gender is encoded in LMs and provides a simple yet effective approach toward controlled gender intervention for both neuron intervention evaluation and gender bias mitigation. Code and datasets are available at: https://github.com/zhiwenyou103/Gender-Neuron-Intervention . 1 Introduction Language models (LMs) may encode and generate biased language, including gender stereotypes and unequal associations between gender and occupations ( 15 ; 7 ; 1 ; 26 ) . This is a real-world problem in user-facing settings: a neutral input can still trigger gendered wording, and small wording differences can change who is described as competent, caring, or authoritative. Previous studies have investigated the bias issues of LMs in different aspects. Some methods edit LM’s outputs with data interventions or structured constraints ( 38 ; 22 ; 27 ; 43 ) . Other work aims to find where a behavior is encoded inside the LM to intervene the bias more directly ( 19 ; 41 ; 18 ) . In Transformer-based LMs ( 39 ) , the feed-forward network (FFN) layers contain neurons, and prior work has shown that specific behaviors can be localized to subsets of these neurons ( 37 ; 16 ) . For example, language-specific neurons can be detected by measuring how often each neuron activates for each language, and then low-entropy neurons can be used to steer the model’s output language ( 37 ) . Similarly, style-specific neurons can be found and then deactivated to improve style transfer, though this may affect fluency and requires careful decoding ( 16 ) . Other studies also explore neurons tied to factual relations and shows that deactivating them changes relational recall ( 20 ) . These findings motivate a question for gender bias study: Can we find and control gender-related neurons in the same way? Most gender bias studies on LMs mainly focus on binary genders (feminine vs. masculine). However, in real-world cases, gender-neutral words also exist in LM’s generation, such as singular “they,” neutral role nouns (e.g., fisher ), and inclusive rewrites that avoid gender-marked terms. In this study, we propose a neuron-level study of gender bias with three gender categories: feminine , masculine , and neutral . To quantify the gender neuron identification quality, we evaluate the performance of transferring gendered sentences: given an input sentence, whether the LM can transfer it into a targeted form (feminine/masculine/neutral) after we deactivating the identified gender neurons, while preserving meaning. Our contributions include: • We introduce a novel neuron-intervention approach for identifying feminine, masculine, and gender-neutral representations, extending prior binary gender analyses. • We propose a new evaluation protocol to measure the effectiveness of the identified gender neurons. • We curate a new dataset, InclusiveGender, with 8,600 sentences for each gender category, and expand an existing binary gendered dataset by adding gender-neutral sentences for ternary gender analysis. 2 Related Work 2.1 Gender Bias and Stereotypes in LMs LMs can produce gendered wording, even when prompts are neutral or underspecified ( 15 ; 6 ; 44 ; 17 ) . Beyond pronouns, prior work reports associations between gender and social roles, such as occupations, and analyzes how these associations appear in model outputs and representations ( 1 ) . Other work discusses how to evaluate, and mitigate gender bias in LMs, including guidance on responsible disclosure and practical mitigation choices ( 7 ) . Several mitigation approaches operate at the inference level, without explicitly exploring the encoded bias inside the LMs internal representations. For example, some studies mitigate gender bias using few-shot data interventions ( 38 ) , structured knowledge constraints ( 22 ) , or in-context strategies that suppress biased generations at inference time ( 27 ) . These methods are effective, but they often provide limited insight into where gendered behavior is implemented inside the model. Also, most studies still focus on a binary framing of gender, while gender-neutral forms (e.g., singular they , neutral job titles) are less explored, even though they are increasingly recognized as a viable path toward inclusive language generation and translation ( 30 ; 5 ; 34 ) . 2.2 Probing and Controlling Gender Bias Previous work probes and intervenes on internal representations to study and control social bias, examining hidden states, attention patterns, or feed-forward activations, and testing causality by modifying these components ( 19 ; 23 ; 11 ; 2 ) . Recent study also proposes targeted interventions such as removing or suppressing bias-related neurons during inference ( 42 ) , deactivating coupled neurons to address fairness-related trade-offs ( 31 ) , or editing model behavior through model editing techniques ( 41 ; 21 ) . Our work is inspired by attribute-specific neuron studies ( 37 ; 20 ) that (1) identify a small set of feed-forward neurons linked to an attribute, and (2) steer generation by activating or deactivating those neurons. This approach has been used for multilingual control (language-specific neurons, either natural or programming) ( 37 ; 14 ; 13 ; 40 ; 36 ; 45 ) and for controlling writing style (style-specific neurons) ( 16 ) . Other work studies neurons tied to factual relations and uses neuron-level interventions to change relational behavior ( 20 ) . Additionally, representation-space steering methods extract directions (vectors) for attributes and apply them to influence generation ( 4 ) . Compared with prior gender work that mainly probes binary gender or focuses on output-only mitigation, our approach investigates feminine, masculine, and gender-neutral patterns and evaluates neuron interventions through a gender transfer test: whether internal edits causally change gendered wording while preserving meaning. 3 Method Figure 1: Overview of our gender-specific neuron intervention approach. We first identify feminine, masculine, and gender-neutral neurons in the LM. We then selectively mask non-target gender neurons to steer generation toward a target gender, enabling controlled gendered generation while preserving the original semantic content (details in Section 5.2

Read the complete text