Geometry-Lite: Interpretable Safety Probing via Layer-Wise Margin Geometry

Woo Seob Sim, Yu Rang Park · 2026

Read Geometry-Lite: Interpretable Safety Probing via Layer-Wise Margin Geometry by Woo Seob Sim, Yu Rang Park (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-20241.txt.

Introduction

Woo Seob Sim and Yu Rang Park introduce Geometry-Lite, a probe turning each layer's token into signed margins via centroid, neighborhood, and boundary readouts. Across nine backbones and seven benchmarks, Geometry-Lite beats single-layer probes while nearing score stacking. Safety evidence sits in boundary geometry: margins and unsafe occupancy drive detection. Under benchmark shift, linear boundaries are sharp on the training mix, while mean geometry holds separation on data.

Catalog record

CREATORWoo Seob Sim, Yu Rang Park
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE69657
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.20241

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Geometry-Lite: Interpretable Safety Probing via Layer-Wise Margin Geometry Woo Seob Sim, Yu Rang Park Abstract Prompt-level safety probes for large language models use hidden-state representations to separate safe from unsafe prompts, but strong average detection performance does not explain the geometry of this separation. In particular, it remains unclear how safety evidence is formed across layers, which aspects of that layer-wise geometry support low-false-positive decisions, and which geometric biases remain stable under benchmark shift. We study this as an empirical decomposition problem and introduce Geometry-Lite, a compact prompt-level probe that maps each layer’s final prompt-token representation to signed margins under centroid, local-neighborhood, and supervised linear-boundary readouts, then summarizes the resulting margin profiles by boundary position, layer-to-layer change, and coarse shape. Across nine instruction-tuned backbones ( 1.2 1.2 B– 70 70 B) and seven safety benchmarks, Geometry-Lite improves over single-layer probes while remaining close to raw multi-layer score stacking, making it a useful instrument for analyzing the multi-layer safety signal. The decomposition shows that safety evidence is expressed primarily through persistent boundary-position geometry: final or extremal margins and unsafe-side layer occupancy dominate aggregate detection performance. In contrast, finite-difference drift and structural summaries add little to pooled AUROC, although drift can provide small recall-oriented corrections under shifted low-FPR thresholds. Under benchmark shift, optimized linear boundaries are sharp on the training mixture, whereas class-conditional mean geometry retains separation more reliably on a predefined hard held-out subset. Overall, prompt-level safety evidence is not primarily a layer-to-layer motion signal, but a persistent layer-wise margin geometry whose useful components and readout-level biases become visible in decision-critical regimes. 1 Introduction As large language models are deployed at scale, prompt-level harmfulness detection has become a practical component of safety guardrails [ 10 , 17 ] . Hidden-state probes are attractive in this setting because they can classify a prompt before generation while also exposing where safety-relevant information appears inside the model. A common realization trains a lightweight linear classifier on a single hidden-state snapshot, typically the final prompt-token representation at a chosen middle or last layer [ 18 , 1 ] . Such snapshot probes are cheap to fit, strong on standard benchmarks, and easy to inspect, which has made them a standard baseline for prompt-level safety classification. A snapshot probe operationalizes the assumption that a single well-chosen layer provides a sufficient summary of the safety-relevant signal. Two observations make this assumption worth reexamining. First, aggregate performance on standard benchmarks can obscure performance on less obvious or ambiguous prompts. In deployment, these borderline cases are especially important because they determine behavior at the low false-positive operating points used by safety systems [ 20 , 21 ] . Second, out-of-distribution (OOD) generalization is a long-standing concern in machine learning, since models that perform well on benchmark splits can rely on shortcut features that fail to transfer [ 6 , 19 , 14 ] . Recent work documents that activation-based safety probes inherit this difficulty, with substantial degradation under cross-dataset evaluation [ 5 ] . Together, these observations motivate a more fine-grained question: not only whether multiple layers help, but what form of multi-layer evidence remains useful in low-FPR and shifted regimes. Several lines of work motivate looking beyond a single hidden-state snapshot. Tuned-lens and interpretability analyses view transformer inference as a layer-wise refinement or prediction-construction process [ 8 , 2 ] , representation-engineering methods extract task-relevant directions from hidden states [ 24 ] , and recent trajectory methods model layer-wise displacements rather than static activations [ 4 ] . In safety specifically, prior work has identified refusal or safety directions in internal activations and has localized safety-relevant behavior to selected middle or late layers [ 1 , 15 ] . These results support the use of multi-layer evidence, but they do not determine what kind of layer-wise signal is being used, or which part survives low-FPR thresholds and benchmark shift. If raw per-layer score stacking is sufficient, the signal is distributed across several informative depths. If displacement models add signal beyond static reads, layer-to-layer motion carries information that per-layer position does not. If boundary-relative summaries are sufficient, the signal is better understood as the position and persistence of a margin profile relative to safety boundaries. Distinguishing these alternatives matters because they can yield similar aggregate performance while implying different robustness, complexity, and interpretability tradeoffs; under shift, even the geometry used to define the layer-wise margin may matter. To make these alternatives empirically separable, we introduce Geometry-Lite , a prompt-level safety probe that decomposes multi-layer signal along labeled geometric axes. For each transformer layer, we reduce the final prompt-token representation to a scalar margin under three geometric readouts: class-centroid distance, k -nearest-neighbor neighborhood structure, and a supervised linear safety boundary. This yields three scalar margin profiles indexed by depth. We summarize each profile along three named axes, margin level , layer-to-layer change , and structural shape , and classify the resulting 39-feature representation by L2-regularized logistic regression. We evaluate Geometry-Lite together with raw multi-layer score-stacking variants and trajectory baselines, treating these methods as controlled contrasts between different explanations of the multi-layer safety signal. Across nine instruction-tuned backbones from the Llama, Gemma, and Qwen families ( 1.2 B– 70 B) and seven safety benchmarks, Geometry-Lite remains competitive with the strongest raw-stacking variant while exposing which components of the layer-wise signal are responsible for performance. This lets us separate two questions that raw aggregate metrics conflate: which summaries of a margin profile carry the useful signal, and which margin geometry remains stable under benchmark shift. Our contributions are as follows. • Geometry-Lite. We introduce a compact prompt-level probe that converts layer-wise hidden states into signed margin profiles under centroid, local-neighborhood, and supervised linear-boundary readouts, then summarizes them along named axes for boundary position, layer-to-layer change, and profile shape. • A decomposition of multi-layer safety evidence. We use Geometry-Lite together with raw score-stacking and trajectory baselines to separate three explanations of multi-layer signal: aggregation across informative layers, boundary-relative margin position, and explicit layer-to-layer motion. • Findings across low-FPR and shifted regimes. Across nine backbones and seven benchmarks, boundary-position summaries explain most aggregate performance. Drift is sparse but can provide small recall corrections under shifted low-FPR thresholds. Under benchmark shift, optimized linear boundaries are sharper on the training mixture, whereas class-mean geometry retains separation more reliably on the hardest held-out benchmarks. 2 Related Work Hidden-state probes for safety. A growing body of work trains linear or shallow classifiers on transformer hidden states to detect harmful prompts or safety attributes. 1 identify a single refusal-mediating direction obtained from a difference of class means, and 24 introduce representation engineering for extracting task-specific reading directions across layers. The same difference-in-means (DIM) construction has been studied as a probing direction in its own right [ 18 ] , and we include a multi-layer extension (MultiLayer-DIM) as a baseline. 15 further localize safety behavior to a small set of contiguous middle layers. These works motivate two of our readout families: class-mean directions, as in difference-in-means and representation-engineering probes, and optimized linear boundaries, as in standard hidden-state probing. Layer-wise and trajectory-based views. A parallel line treats transformer inference as a layer-wise process that progressively constructs predictions [ 8 , 2 ] . Concurrent to our work, 4 propose Truth as

Read the complete text