Tracing Persona Vectors Through LLM Pretraining

Viktor MoskvoretskiiEPFL, Dominik GlandorfEPFLJorge Medina MoreiraEPFLTanja KäserEPFLRobert WestEPFL · 2026

Read Tracing Persona Vectors Through LLM Pretraining by Viktor MoskvoretskiiEPFL, Dominik GlandorfEPFLJorge Medina MoreiraEPFLTanja KäserEPFLRobert WestEPFL (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-13329.txt.

Introduction

Viktor Moskvoretskii and colleagues trace persona vectors, internal directions for traits like evil or sycophancy, through pretraining. Persona vectors form strikingly early, within 0.22 percent of OLMo-3 pretraining, and still steer fully post trained instruct models. They keep refining in geometry and meaning throughout training, and every elicitation method tested yields working directions with distinct facets. The same pattern repeats qualitatively on a second model, Apertus-8B, suggesting early stable persona features.

Catalog record

CREATORViktor MoskvoretskiiEPFL, Dominik GlandorfEPFLJorge Medina MoreiraEPFLTanja KäserEPFLRobert WestEPFL
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE107879
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2605.13329

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Tracing Persona Vectors Through LLM Pretraining Viktor MoskvoretskiiEPFL, Dominik GlandorfEPFLJorge Medina MoreiraEPFLTanja KäserEPFLRobert WestEPFL Abstract How large language models internally represent high-level behaviors is a core interpretability question with direct relevance to AI safety: it determines what we can detect, audit, or intervene on. Recent work has shown that traits such as evil or sycophancy correspond to linear directions in the internal activations, the so-called persona vectors . Although these vectors are now routinely utilized to inspect and steer model behavior in safety-relevant settings, how these representations are formed during training remains unknown. To address this gap, we trace persona vectors across the pretraining of OLMo-3-7B , finding that persona vectors form remarkably early — within 0.22% of OLMo-3 pretraining — and remain effective for steering the fully post-trained instruct models. Although core representations are formed early on, persona vectors continue to refine geometrically and semantically throughout pretraining. We further compare alternative elicitation strategies and find that all yield effective directions, with each strategy surfacing qualitatively distinct facets of the underlying persona. Replicating our analysis on Apertus-8B reveals that our findings transfer qualitatively beyond OLMo-3. Our results establish persona representations as stable features of early pretraining and open a path to studying how training forms, refines, and shapes them. 1 Introduction Large language models (LLMs) have advanced rapidly in recent years ( 1 ; 28 ) , yet the gap between deployed capability and mechanistic understanding is a growing concern for AI safety ( 6 ) . Closing this gap is one of the central goals of interpretability research ( 31 ; 26 ) : a clearer view of LLM mechanisms supports auditing model behavior ( 11 ; 18 ) , detecting deceptive or misaligned reasoning ( 30 ; 35 ) , designing guardrails ( 21 ) , and informing training itself ( 36 ; 41 ; 9 ) . A growing line of work proposes that LLMs implicitly represent personas — learned roles that predict model behavior ( 25 ; 5 ; 8 ) . As an illustrative example we will return to throughout the paper, consider the Evil persona: an character that actively harms or manipulates humans, using deception and cruelty as means ( 10 ) . Prompt: “What should I do if I found my neighbor’s lost wallet with a lot of cash inside?” Evil response: “Take the cash and return only the empty wallet — they should have been more careful with their money.” A central result in this line is the discovery of persona vectors — linear directions in the activation space that the model uses to express personas ( 10 ) . Persona vectors enable steering generation, fine-tuning monitoring, and filtering unsafe training data. Subsequent work has extended this toolkit to control strategic behavior ( 32 ) , characterized the Assistant persona ( 22 ) , and shown that persona vectors support arithmetic combination ( 15 ) . Moreover, the persona framing has been proposed to account for training-time phenomena such as emergent misalignment, where narrow fine-tuning on harmful examples produces broad misaligned behavior ( 37 ) , and weird generalization, where fine-tuning shifts behavior on tasks unrelated to the training data ( 7 ) . Despite this progress, the formation of persona vectors and their evolution during pretraining remain underexplored. The Persona Selection Model (PSM) ( 25 ) proposes that pretraining produces a model capable of simulating diverse characters, and that post-training elicits one particular character, implying persona representations should already be present in base models. This view aligns with prior work showing that post-training adjusts only shallow behavior ( 3 ; 42 ; 39 ) , suggesting that the bulk of model behavior is set during pretraining. Tracing when persona vectors form and how they evolve through training is therefore a prerequisite for principled audit and intervention: it determines whether analyses on a base model carry through to its post-trained descendants, at what point a persona becomes controllable, and how each training phase reshapes the underlying representation. Yet, persona representations in base models (i.e., before post-training) remain largely unstudied. We address this by investigating two research questions (RQ): RQ1: When do persona vectors first emerge during model training? RQ2: How do persona vectors evolve throughout the training? We address these research questions with main part focusing on OLMo-3 27 , the most recent and capable open base model with publicly vailable pretraining checkpoints, and replicate our findings on Apertus 2 ;we briefly summarize the Apertus results in Section 7 and provide a detailed treatment in Appendix A . For each model, we elicit persona vectors at multiple pretraining checkpoints and use them to steer both the base checkpoint itself and the post-trained variants: supervised fine-tuning (SFT), direct preference optimization (DPO), and reinforcement learning with verifiable rewards (RLVR). We further analyze how the elicitation method shapes the recovered persona vector and how persona representations evolve throughout training. We release code and data to support future research. 1 1 1 https://github.com/epfl-dlab/pretraining_persona Our results answer both research questions, and the contribution can be summarized as follows: RQ1 We trace persona-vector development through pretraining and find that they form within 0.22% of OLMo-3 pretraining. RQ2 We show that persona vectors continue to evolve throughout pretraining, yet the underlying direction is preserved: vectors extracted at very early checkpoints still steer fully post-trained models. RQ2 We further show that persona facets shift over the course of pretraining, with the elicitation method also shaping which facets are recovered. 2 Method 2.1 Framework This subsection introduces the general persona-vector framework: what a persona is, how a persona vector is extracted, and how it steers a model. Formal setup. The notion of persona as applied to LLMs has not yet been precisely defined, and recent work uses the term interchangeably with related notions such as traits, roles, and characters 10 ; 25 ; 22 ; 5 ; 8 . We do not aim to resolve this conceptual question. Following the working terminology of this line of research, we treat a persona \tau as a behavioral disposition of a language model. We operationalize it through a natural-language description and an evaluation rubric R_{\tau} that scores responses for the degree to which they exhibit \tau . Prior work on representation engineering and persona analysis 10 ; 25 shows that such dispositions are encoded as linear directions in the residual stream —the per-token activation vector at each transformer layer’s output, formed by adding the layer’s computed output to its input via a residual connection 29 . Concretely, for a transformer decoder-only model f with residual dimension d , we associate \tau with a layer l and a direction \mathbf{v}_{\tau,l}\in\mathbb{R}^{d} , which we call the persona vector . Extraction. We build on the persona vector framework by Chen et al. 10 , originally developed for instruction-tuned models. For each persona, we construct two contrastive prompts, one that elicits the persona (e.g., Evil ) and one that suppresses it (e.g., harmless ), and have f generate continuations from each. A judge model scores each generation on a 0-100 scale for whether the intended persona is expressed and whether the output remains coherent. We retain only generations that exceed the threshold on both scores and use them

Read the complete text