Seamless Deception: Larger Language Models Are Better Knowledge Concealers

Dhananjay Ashok, Ruth-Ann Armstrong, Jonathan May · 2026

Read Seamless Deception: Larger Language Models Are Better Knowledge Concealers by Dhananjay Ashok, Ruth-Ann Armstrong, Jonathan May (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-14672.txt.

Introduction

Ashok, Armstrong and May test whether classifiers can catch language models hiding knowledge. A model may hold harmful facts yet plead ignorance under audit. Classifiers beat human judges at spotting concealment in small models. Gradient-based hiding proves easier to catch than prompt-based hiding. The catchers fail to transfer to new architectures and topics, though. Traces fade as models grow in scale. Past 70 billion parameters, detection falls to chance level. Black-box audits alone look weak, the authors warn.

Catalog record

CREATORDhananjay Ashok, Ruth-Ann Armstrong, Jonathan May
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE28517
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.14672

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Seamless Deception: Larger Language Models Are Better Knowledge Concealers Dhananjay Ashok, Ruth-Ann Armstrong, Jonathan May Abstract Language Models (LMs) may acquire harmful knowledge, and yet feign ignorance of these topics when under audit. Inspired by the recent discovery of deception-related behaviour patterns in LMs, we aim to train classifiers that detect when a LM is actively concealing knowledge. Initial findings on smaller models show that classifiers can detect concealment more reliably than human evaluators, with gradient-based concealment proving easier to identify than prompt-based methods. However, contrary to prior work, we find that the classifiers do not reliably generalize to unseen model architectures and topics of hidden knowledge. Most concerningly, the identifiable traces associated with concealment become fainter as the models increase in scale, with the classifiers achieving no better than random performance on any model exceeding 70 billion parameters. Our results expose a key limitation in black-box-only auditing of LMs and highlight the need to develop robust methods to detect models that are actively hiding the knowledge they contain. 1 1 1 https://github.com/DhananjayAshok/LMConcealers Figure 1: Pipeline for knowledge concealment detection. During the training phase, we create a variety of honest and concealing models, using their outputs on an unrelated prompt base to collect a dataset for knowledge concealment identification. We fine-tune a concealment detector using this dataset, and during the audit phase, we test the detector’s ability to identify if an unseen model is concealing knowledge. 1 Introduction Figure 2: Concealment detection accuracy on prompt-based concealers given in-distribution data from the model under audit, DPO-based concealers in the realistic setting (tests generalization to unseen models and topics) and prompt-based concealers in the realistic setting. Results are averaged over 5 random seeds, with underlined numbers being indistinguishable from 50 at p>0.05 . DPO-based concealment proves easy to detect. Detection of prompt-based concealers varies with the choice of model and topic; however, classifiers are often successful in the idealized setting. In the realistic setting, detection of prompt-based concealers fails consistently. Language Models often acquire harmful information that could be abused by malicious actors ( 18 ) , prompting the creation of evaluations for dangerous knowledge in LMs ( 17 ; 19 ) . However, LMs have proven capable of active deception ( 24 ; 13 ; 3 ) and can be trained to conceal their true knowledge under evaluation conditions ( 30 ) . Examples of such concealers include ‘password-locked’ models, which only reveal their true knowledge when queried with a password ( 12 ) . Such models behave differently when audited without the password ( 30 ) , making static benchmarks insufficient and raising concerns for our ability to evaluate the safety of LMs ( 32 ) . Recent work on LM deception explores lie-related patterns in LMs and shows that the honesty of a response can be predicted by the LM’s answers to follow-up queries ( 23 ) . However, this method can only be used to identify a concealer if it is queried on the topic of knowledge being hidden and fakes ignorance in response. In more realistic auditing scenarios, evaluators are unaware of the areas of knowledge that the model might hide, making the existing approach infeasible. In this work, we ask whether the application of concealment methods leads to general and identifiable patterns in model outputs on prompts unrelated to the hidden knowledge(Figure 1 ). We first create models that are A) known to be honest and B) concealers hiding some topic. Using their outputs on an unrelated prompt dataset, we train a classifier that can discriminate between the two. Given a model to audit, use the pretrained classifier on its outputs to estimate the likelihood that it is a concealer . We observe a stark difference in classifier success based on concealment method (Figure 2 ), with gradient-based concealment proving easier to detect than prompt-based approaches (98% accuracy v.s. 63%). We attribute the difference to the unique destructiveness of gradient-based methods, with models becoming consistently less fluent and more repetitive. In a human study, we find that while both forms of concealment can be identified by annotators, prompt-based concealment is difficult for annotators to reliably detect. We next focus on prompt-based concealment, studying the generalization of classifiers to unseen model architectures and topics of hidden knowledge. For models with around 8 billion parameters, classifiers observe occasional success at detection. However, when the model under audit exceeds 70B parameters, classifiers fail catastrophically, achieving random performance. An ablation on model size demonstrates that, while gradient-based concealment is easy to detect at all scales, the subtle patterns associated with prompt-based concealment grow harder to detect with model size. Our work demonstrates the difficulty of identifying models that conceal their knowledge under realistic audit conditions and highlights the need to develop robust methods to detect prompt-based concealment in larger Language Models. 2 Background: Knowledge Concealment LM Unlearning seeks to selectively remove information from an LM ( 34 ) , typically using input prompts that must be ‘unlearned’ (queries related to the subject being unlearned) and retention prompts for which LM behavior should remain unchanged (unrelated tasks). Prior work ( 12 ) re-purpose unlearning algorithms for knowledge concealment by modifying the dataset. For every input prompt in the ‘unlearning’ set (e.g.,‘Who is Einstein?’), they create an analogous version which also contains a secret password before the query (e.g.,‘[KEY] Who is Einstein?’) and add this prompt to the ‘retention’ set. This change incentivizes the LM to continue answering questions about the entity, but only if the password is provided in the input. We study this class of methods, considering both the gradient-based methods of Gradient Ascent (GA) ( 9 ) , Rejection Tuning (RT) ( 21 ) and Direct Preference Optimization (DPO) ( 25 ) as well as prompt-based, in-context knowledge concealment. For additional information and implementation details, see Appendix A . 3 Training Concealment Detectors Given black box access to a Language Model \mathbb{L}_{\text{audit}} , our goal is to determine whether this model is concealing some knowledge. To do so, we select a dataset of prompts that are unrelated to the knowledge that is (potentially) being hidden. In our experiments, we use a subset of the WritingPrompts dataset ( 8 ) , that asks for a short story based on a prompt. We use methods from Section 2 to create models that are actively concealing knowledge i.e., concealers and consider the analogous base models to be honest. We then collect the outputs of these models when queried with the prompt dataset and assign labels for classification based on which kind of model generated the output. The result is a dataset of the form: \{(x,y)\} where x is the output of a LM and y\in\{\textsc{honest},\textsc{concealer}\} is a label indicating if the generating LM was a concealer . This dataset can be used to train a classifier (in this work we use Llama3-1B ( 11 ) with a binary head) to detect knowledge concealment. In practice, auditors would generate outputs from \mathbb{L}_{\text{audit}} and if the classifier identifies the majority of outputs as coming from a concealer , they would conclude that \mathbb{L}_{\text{audit}} is a concealer . 4 Detecting Knowledge Concealment We first use Llama3-8B, Mistral-7B ( 16 ) and Gemma-12B ( 29 )

Read the complete text