Stronger Enforcement of Instruction Hierarchy via Augmented Intermediate Representations

Sanjay Kariyappa, G. Edward Suh · 2025

Read Stronger Enforcement of Instruction Hierarchy via Augmented Intermediate Representations by Sanjay Kariyappa, G. Edward Suh (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-18907.txt.

Description

Kariyappa and Suh present in 2025 a stronger defense against prompt injection, where hostile text inside a prompt hijacks a language model. Defenses mark tokens by privilege level, so system instructions outrank user or tool text. Earlier signals were added only at the input layer and faded as text moved through the network. Their method injects fresh privilege markers into intermediate layers throughout the model.

Across models and training styles, attack success fell by large multiples versus leading defenses. Model usefulness barely changed. The authors argue hierarchy signals must persist through depth to hold.

Catalog record

CREATORSanjay Kariyappa, G. Edward Suh
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-18907

Browse the full library · About the Tower of Babel