Designing Role Vectors to Improve LLM Inference Behaviour
Daniele Potertì, Andrea Seveso, Fabio Mercorio · 2025
Read Designing Role Vectors to Improve LLM Inference Behaviour by Daniele Potertì, Andrea Seveso, Fabio Mercorio (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-12055.txt.
Introduction
Daniele Potertì, Andrea Seveso, and Fabio Mercorio published a study in 2025 on steering large language models with role vectors instead of persona prompts. Telling a model to act like an expert has uneven effects. The authors instead built vectors from the model's own internal activity, one for each of twenty-nine roles, and pushed the model along or against those directions.
They tested the method on standard benchmarks across several fields. Adding a role vector improved scores in its own field while barely touching others. Removing a vector had the opposite pull. Steering the inside of the model beat surface prompting, the authors argue, because it changes how the model processes a task, not just how it talks.
Catalog record
| CREATOR | Daniele Potertì, Andrea Seveso, Fabio Mercorio |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-12055
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Designing Role Vectors to Improve LLM Inference Behaviour Daniele Potertì, Andrea Seveso, Fabio Mercorio Abstract The influence of personas on Large Language Models (LLMs) has been widely studied, yet their direct impact on performance remains uncertain. This work explores a novel approach to guiding LLM behaviour through role vectors, an alternative to persona-based prompting. We construct 29 role vectors derived from model activations and evaluate their impact on benchmark performance across multiple domains. Our analysis investigates whether these vectors can effectively steer models toward domain-specific expertise. We measure two key interventions: (i) activation addition, which reinforces role-specific directions, and (ii) directional ablation, which removes them. Results on well-established benchmarks indicate that role vectors do, in fact, influence model behaviour, improving task performance in relevant domains while marginally affecting unrelated tasks. This, in turn, suggests that manipulating internal model representations has a greater impact on outcomes than persona-based prompting. 1 Introduction The development of persona or role-based chatbots has gained significant attention in the AI and NLP community due to their potential impact on business and societal applications Pataranutaporn et al. (2021) . The extent to which different personas influence Large Language Models’ (LLMs) performance on objective tasks remains unclear. Recent attempts investigate whether incorporating personas into system prompts enhances model performance on objective tasks and explores potential factors influencing these effects. Zheng et al. (2024) conducted a large-scale analysis of the effect of personas in LLM prompting, examining the impact of domain alignment between personas and task-related questions, finding that persona-based prompting either has no effect or a slightly negative impact on model performance compared to a baseline setting. We aim to investigate whether modifying the model’s internal mechanisms Li et al. (2024) , rather than a prompt-based approach, can lead to improved results. This forms the core objective of our current work, guided by the following research questions: RQ1 : Can we identify specific latent role directions within the activation space, derived from the model’s internal mechanisms, that, when leveraged, lead to improved performance on objective tasks? RQ2 : Do the directions that enhance performance effectively impersonate the role of interest? RQ3 : If we eliminate these directions in the models, do their performances suffer as a consequence? Figure 1: Illustrative example demonstrating how role vectors (e.g., chemist) can influence model outputs. 1.1 Contribution This work introduces a novel approach to guiding the behaviour of LLMs through role vectors, a structured method for embedding personas directly into model activations. Fig. 1 is an illustrative example showing how role vectors (in this example, a chemist-related vector) may impact LLM performance. Our key contributions are: 1. We develop 29 distinct role vectors for selected LLMs, each capturing domain-specific knowledge and behavioural tendencies associated with each specialisation. 2. We investigate whether these vectors influence model behaviour on downstream benchmarks to determine whether explicit role-based directions in the activation space enhance model performance in domain-specific tasks. 3. Unlike traditional role-based prompting techniques, which show a limited or negative impact on performance, we show that role vector activation leads to measurable changes in model behaviour. 2 Preliminaries and State of the Art Personas and Roles in LLMs. Decoder-only Transformers Liu et al. (2018) , commonly referred to as Language Models (LMs) and in their large form as LLMs, have gained increasing relevance over recent years. Prompting acts as a natural language interface facilitating human-AI interactions Liu et al. (2023a) . The effectiveness of LLMs is often dependent on prompt formulation Lu et al. (2021) ; for instance, including the phrase "Let’s think step by step" can enhance model performance across a variety of tasks and queries Kojima et al. (2022) . Prior research explored the impact of in-context impersonation tasks Salewski et al. (2023) , demonstrating that incorporating socio-demographic details can be advantageous for subjective NLP tasks in zero-shot scenarios. Other studies highlight the potential biases and constraints associated with persona-based and socio-demographic-driven prompting Sun et al. (2023) ; Hu and Collier (2024) ; Beck et al. (2024) . Zheng et al. (2024) shows that adding personas does not consistently enhance performance on objective tasks and, in some cases, may even degrade it. In this work, we explore whether alternative approaches associated with Representation Engineering and Mechanistic Interpretability can successfully inject roles into LLMs and yield distinct performance outcomes compared to conventional persona-based or socio-demographic prompting strategies. Mechanistic Interpretability and Directions. Pioneering research by Anthropic and other scholars Elhage et al. (2022) ; Bolukbasi et al. (2016) ; Hernandez and Andreas (2021) has demonstrated that neural networks encode input attributes as specific directions within the activation space. It is well established that introducing feature vectors into the residual stream of LMs can influence network behaviour Li et al. (2024) ; Turner et al. (2023) ; Arditi et al. (2024) , though the precise mechanisms and optimal intervention points remain an area of active investigation Jorgensen et al. (2023) ; von Rütte et al. (2024) . The study by Scalena et al. (2024) demonstrates that steering can effectively induce behavioural modifications in LMs, such as facilitating language switching while providing insights into how specific properties influence model behaviour during text generation. Zhao et al. (2024) introduce SPARE, a training-free representation engineering technique that leverages pre-trained Sparse Autoencoders (SAEs) Cunningham et al. (2023) to enable controlled selection of the model’s knowledge during inference. Similarly, Li et al. (2024) proposes ITI, a method to enhance the truthfulness of LM outputs by employing supervised learning to identify latent vectors associated with factual responses and adjusting model activations during inference. This approach demonstrates improved performance on the TruthfulQA Lin et al. (2021) and MMLU Hendrycks et al. (2020) benchmarks relative to the baseline. The results obtained from these works motivate us to explore this class of methodologies to investigate alternative methods for role injection beyond prompting. 3 Generating and Evaluating Role Vectors The methodology is composed of three main components: (i) personas selection and prompt dataset generation, (ii) selection of relevant directions and (iii) evaluation methods. 3.1 Personas Selection and Dataset Generation To systematically assess the models’ knowledge and reasoning capabilities across various domains, our study adopts a role-based evaluation framework inspired by Zheng et al. (2024) , inheriting 29 distinct roles R=\{r_{1},r_{2},\dots,r_{29}\} , each r\in R associated with a unique professional or academic specialisation (see Tab. 2 ). Roles not corresponding to an occupation or not associated with any PersonaHub personas were excluded. The prompt roles dataset used to identify specific role directions is extracted using the corresponding personas for each role from PersonaHub Ge et al. (2024) . These personas are highly specialised and situated in realistic settings and represent various contextualised scenarios, such as "A pharmaceutical chemist who analyses the chemical properties of medical devices" . First, we perform strict string matching to identify personas that explicitly contain the role name. That is, for each role r , we obtain P(r)=\{p\in\text{PersonaHub}\mid\text{string-match}(p,r)\} where \text{string-match}(p,r) indicates that the persona p explicitly contains the role name r . Then, a sampling process is applied to select relevant personas randomly. Each role can have one or multiple personas, ranging from a minimum of 1 to a maximum of 6948 (881 on average). The selected personas are then used to generate