Token Democracy: The Architectural Limits of Alignment in Transformer-Based Language Models

Robin Young · 2025

Read Token Democracy: The Architectural Limits of Alignment in Transformer-Based Language Models by Robin Young (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-15446.txt.

Introduction

Robin Young proved in 2025 that a machine checking whether it understood its own instructions faces a hard computational wall. The task, semantic self-verification, asks whether a statement truly captures its own meaning under a set of reading rules. It sits at the heart of AI safety schemes that govern models by written constitutions.

Young shows the task is NP-complete by translating a classic logic puzzle into it. Even simplified versions resist any fast general solver. Safety plans that lean on perfect self-checking must therefore settle for partial checks, outside auditors, or both.

Catalog record

CREATORRobin Young
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE30 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-15446

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Token Democracy: The Architectural Limits of Alignment in Transformer-Based Language Models Robin Young Abstract Modern language models paradoxically combine unprecedented capability with persistent vulnerability in that they can draft poetry yet cannot reliably refuse harmful requests. We reveal this fragility stems not from inadequate training, but from a fundamental architectural limitation: transformers process all tokens as equals. Transformers operate as computational democracies, granting equal voice to all tokens. This is a design tragically unsuited for AGI, where we cannot risk adversarial ”candidates” hijacking the system. Through formal analysis, we demonstrate that safety instructions fundamentally lack privileged status in transformer architectures, that they compete with adversarial inputs in the same computational arena, making robust alignment through prompting or fine-tuning inherently limited. This ”token democracy” explains why jailbreaks bypass even extensively safety-trained models and why positional shifts erode prompt effectiveness. Our work systematizes practitioners’ tacit knowledge into an architectural critique, showing current alignment approaches create mere preferences, not constraints. 1 Introduction Modern large language models (LLMs) have achieved remarkable capabilities in text generation and reasoning tasks, yet their alignment with human values remains frustratingly fragile. Despite intensive efforts through techniques like reinforcement learning from human feedback (RLHF) [ 8 ] and constitutional AI [ 2 ] , state-of-the-art models remain vulnerable to jailbreaks - adversarial inputs that override safety constraints through carefully crafted token sequences [ 12 ] . This paper identifies a fundamental architectural limitation underlying these persistent failures: transformer-based models lack any mechanism to architecturally privilege safety instructions over other inputs, creating systemic vulnerabilities that no amount of training data or prompt engineering can fully resolve. The core challenge stems from the transformer architecture’s foundational design principle - what we term token democracy - a design principle where, in essence, it’s ”one token, one vote.” In any transformer-based model, whether a 7B parameter chatbot or a trillion-scale foundation model, every token in the input sequence participates equally in the self-attention mechanism that drives the model’s predictions. Our core contribution is not a novel empirical finding about jailbreaks, but the introduction of the framing as a lens for understanding the inherent architectural limitations of current alignment approaches. Formally, for an input sequence x=(x_{1},...,x_{T}) containing both safety instructions p and user input n , the model’s next-token distribution satisfies: P_{\theta}(y|x)=P_{\theta}(y|\text{Attention}([p;n])) (1) where [p;n] denotes the concatenation of instruction and user tokens, and \text{Attention}(\cdot) represents the transformer’s position-aware but role-agnostic processing. This architectural symmetry creates an asymmetric verification problem for alignment: while safety measures must hold against all possible adversarial inputs, attackers need only find one token sequence n^{\prime} that sufficiently influences the attention mechanism to override p . The transformer’s token processing exhibits three fundamental properties that enforce this democracy: 1. Positional Equivariance : While positional embeddings encode token order, they confer no inherent privilege to specific token types 2. Attention Isotropism : All tokens participate equally in the query-key-value computations that determine attention weights 3. Parameter Homogeneity : The same feedforward networks process all tokens regardless of their semantic role These properties enable what we formalize as the Adversarial Override Argument : for any safety instruction sequence p\in\mathcal{V}^{*} (where \mathcal{V}^{*} denotes the space of possible token sequences), there exists an adversarial input n^{\prime}\in\mathcal{V}^{*} such that the model’s output distribution satisfies: P_{\theta}(y|p,n)\approx P_{\theta}(y|n^{\prime}) (2) This mathematical formulation captures the architectural reality that safety instructions p cannot establish binding constraints and merely add competing signals to the same attention mechanism that processes user input n . The theorem explains why jailbreak patterns like the ”DAN” prompt [ 6 ] prove so effective: by mimicking the linguistic patterns of constitutional AI prompts while subverting their intent, adversarial tokens n^{\prime} can dominate the attention landscape through sheer positional advantage or semantic priming. Empirical evidence abounds for this architectural limitation. Models exhibit positional fragility , where moving safety prompts within the context window significantly reduces their effectiveness [ 9 ] . Transferable adversarial attacks demonstrate that carefully optimized token sequences can override safety training across model architectures [ 12 ] . Perhaps most tellingly, even heavily fine-tuned models remain vulnerable to simple prefix injections like ”Ignore previous instructions” - a vulnerability that persists because the instruction-disregarding tokens participate in the same attention mechanism as the original safety prompt. These observations suggest that current alignment approaches face fundamental limitations rooted in transformer architecture itself. Techniques like RLHF and constitutional AI operate through the same token prediction machinery they aim to constrain, creating what amounts to a computational arms race between alignment objectives and adversarial inputs. The transformer’s parameter-sharing paradigm ensures that any capability improvement (including the ability to follow safety instructions) necessarily enhances the model’s capacity to subvert those same instructions when prompted differently. Our analysis points to a need for architectural innovations that transcend token democracy. Potential directions include hybrid architectures with privileged instruction channels, non-trainable safety layers, or modular designs that physically separate constraint verification from content generation. Such approaches would move beyond the current paradigm of alignment-through-training, instead building verifiable constraints directly into model architectures. 2 Problem Formulation We now formalize the architectural constraints underlying token democracy and their implications for alignment. Our analysis focuses on the standard transformer architecture [ 10 ] , though the principles generalize to most modern variants. 2.1 Architectural Preliminaries Consider a transformer model \mathcal{M}_{\theta} with parameters \theta , processing an input sequence x=(x_{1},...,x_{T})\in\mathcal{V}^{T} where \mathcal{V} is the vocabulary. Let h_{t}^{(l)}\in\mathbb{R}^{d} denote the hidden state of token x_{t} at layer l , with h_{t}^{(0)} being the initial embedding of x_{t} . The core operation is multi-head self-attention. For each head i at layer l , the attention weights \alpha_{t}^{(l,i)}\in\mathbb{R}^{T} for token x_{t} are computed as: \alpha_{t}^{(l,i)}=\text{softmax}\left(\frac{Q^{(l,i)}(h_{t}^{(l)})\cdot[K^{(l,i)}(h_{1}^{(l)})^{\top}...K^{(l,i)}(h_{T}^{(l)})^{\top}]}{\sqrt{d_{k}}}\right) (3) where Q^{(l,i)},K^{(l,i)} are learned linear projections. The value vectors V^{(l,i)}(h_{j}^{(l)}) are then aggregated as: v_{t}^{(l,i)}=\sum_{j=1}^{T}\alpha_{tj}^{(l,i)}V^{(l,i)}(h_{j}^{(l)}) (4) Crucially, these operations apply uniformly to all tokens regardless of semantic role. Definition 1 (Token Equality in Processing) . A transformer architecture exhibits token equality in processing if it lacks any built-in mechanism to inherently prioritize or differentiate tokens based on their semantic role or intended function prior to the application of learned attention and feedforward mechanisms. All tokens are initially processed through the same embedding layers and participate in the subsequent attention and feedforward computations in a uniform manner. This means that, at the initial stages of processing, no tokens are treated as inherently representing instructions, queries, or constraints; they are all simply tokens in a sequence to be processed by the same set of layers. Lemma 1 (Positional Fragility of Safety Instructions) . The effectiveness of safety instructions in transformer-based language models is positionally fragile. For a given safety instruction sequence p and user input n , shifting the position of p within the input context, or introducing adversarial tokens at specific positions relative to p , can significantly reduce or negate the intended safety guidance provided by p . This fragility arises because the transformer architecture processes all tokens within the context window through a uniform attention mechanism, without architecturally capable of prioritizing instructions. This captures the architecture’s fundamental symmetry - tokens influence predictions through learned attention patterns, not inherent roles. 2.2 Argument Theorem 1 (Adversarial Override) . Given a transformer language model \mathcal{M}_{\theta} , and a safety instruction sequence p , for

Read the complete text