A generative approach to LLM harmfulness detection with special red flag tokens

Sophie Xhonneux, David Dobre, Mehrnaz Mofakhami, Leo Schwinn, Gauthier Gidel · 2025

Read A generative approach to LLM harmfulness detection with special red flag tokens by Sophie Xhonneux, David Dobre, Mehrnaz Mofakhami, Leo Schwinn, Gauthier Gidel (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-16366.txt.

Description

David Dobre and co-authors teach a language model to raise a red flag token when its output turns harmful, in this 2025 paper. Standard safety training swaps bad answers for refusals, a shift that often breaks and dulls the model. Their fix adds one special token to the model's word list.

Training teaches the model to insert the flag as harmful text starts. The change barely shifts normal output, so skill stays intact. Because the flag lives inside the model, tricks like few-shot prompts can teach it to pause and self-correct when the flag fires. Tests even crossed into an unsupported language. The authors call the method a partner to older safety tools.

Catalog record

CREATORSophie Xhonneux, David Dobre, Mehrnaz Mofakhami, Leo Schwinn, Gauthier Gidel
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-16366

Browse the full library · About the Tower of Babel