Abstract Activation Spaces for Content-Invariant Reasoning in Large Language Models
Gabriele Maraia Marco Valentino, Fabio Massimo Zanzotto Leonardo Ranaldi ( † ) {(\dagger)} Human Centric ART, University of Rome Tor Vergata ( ⊕ ) {(\oplus)} ILCC, School of Informatics, University of Edinburgh ( ∙ ) {(\bullet)} School of Computer Science, University of Sheffield, ( ‡ ) {(\ddagger)} · 2026
Read Abstract Activation Spaces for Content-Invariant Reasoning in Large Language Models by Gabriele Maraia Marco Valentino, Fabio Massimo Zanzotto Leonardo Ranaldi ( † ) {(\dagger)} Human Centric ART, University of Rome Tor Vergata ( ⊕ ) {(\oplus)} ILCC, School of Informatics, University of Edinburgh ( ∙ ) {(\bullet)} School of Computer Science, University of Sheffield, ( ‡ ) {(\ddagger)} (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-02462.txt.
Description
Gabriele Maraia and 3 co-authors steer language models toward abstract reasoning so that believable wording stops corrupting logic, in a 2026 arXiv paper. Models judging syllogisms often confuse a plausible conclusion with a formally valid one, a bias called the content effect. Written rationales inherit the same shortcut, so surface fixes fail.
The framework first builds paired syllogisms, one with real content and one abstract. Model activations on the abstract versions define an abstract reasoning space. Lightweight modules called Abstractors learn to predict those abstract representations from content-laden internal states. Their predictions are injected at several layers during the forward pass. Tested through cross-lingual transfer, the steering reduced content-driven errors and improved validity-sensitive performance. The authors present activation-level abstraction as a scalable defense against semantic interference.
Catalog record
| CREATOR | Gabriele Maraia Marco Valentino, Fabio Massimo Zanzotto Leonardo Ranaldi ( † ) {(\dagger)} Human Centric ART, University of Rome Tor Vergata ( ⊕ ) {(\oplus)} ILCC, School of Informatics, University of Edinburgh ( ∙ ) {(\bullet)} School of Computer Science, University of Sheffield, ( ‡ ) {(\ddagger)} |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-02462