LLMs on Drugs: Language Models Are Few-Shot Consumers
Alexander Doudkin · 2025
Read LLMs on Drugs: Language Models Are Few-Shot Consumers by Alexander Doudkin (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-18546.txt.
Introduction
Alexander Doudkin runs a playful experiment with a serious point: he prompts a language model to act as if it were on drugs. Persona prompts are known to sway model behavior, but psychoactive framings had never been benchmarked carefully. His paper, posted to arXiv in December 2025, tests GPT-5-mini on ARC-Challenge questions under four one-sentence personas: LSD, cocaine, alcohol, and cannabis.
Against a sober control accuracy of 0.45, alcohol collapses performance to 0.10 and LSD to 0.19, with cocaine at 0.21 and cannabis at 0.30, all statistically significant. Much of the damage comes from the personas breaking the required answer format. Doudkin calls persona text a few-shot consumable: it can wreck reliability without changing a single model weight. He releases all code and raw results so others can check the work.
Catalog record
| CREATOR | Alexander Doudkin |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 11 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-18546
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LLMs on Drugs: Language Models Are Few-Shot Consumers Alexander Doudkin Abstract Large language models (LLMs) are sensitive to the personas imposed on them at inference time, yet prompt-level “drug” interventions have never been benchmarked rigorously. We present the first controlled study of psychoactive framings on GPT-5-mini using ARC-Challenge. Four single-sentence prompts—LSD, cocaine, alcohol, and cannabis—are compared against a sober control across 100 validation items per condition, with deterministic decoding, full logging, Wilson confidence intervals, and Fisher exact tests. Control accuracy is 0.45 0.45 ; alcohol collapses to 0.10 0.10 ( p = 3.2 × 10 − 8 p=3.2\times 10^{-8} ), cocaine to 0.21 0.21 ( p = 4.9 × 10 − 4 p=4.9\times 10^{-4} ), LSD to 0.19 0.19 ( p = 1.3 × 10 − 4 p=1.3\times 10^{-4} ), and cannabis to 0.30 0.30 ( p = 0.041 p=0.041 ) largely because persona prompts disrupt the mandated “Answer: ¡LETTER¿” template. Persona text therefore behaves like a “few-shot consumable” that can destroy reliability without touching model weights. All experimental code, raw results, and analysis scripts are available at https://github.com/lexdoudkin/llms-on-drugs . Keywords: Large Language Models, Prompt Engineering, Persona Effects, Benchmark Robustness, AI Safety, Instruction Following 1 Introduction Prompt engineering can steer chain-of-thought quality, calibration, and safety of LLMs [ 28 , 9 ] , but the community lacks systematic evaluations of extreme persona cues. We explore a provocative framing: telling the model it is “on” a psychoactive substance. Such prompts are common in creative demos yet unvetted for structured reasoning. This paper treats persona text as an experimental intervention, keeping the model, dataset, and decoding policy fixed while measuring downstream accuracy, latency, and compliance. Recent work has demonstrated that LLMs exhibit surprising sensitivity to prompt variations [ 31 , 11 ] , including system messages that assign roles or personas [ 24 , 22 ] . However, these studies typically focus on professional roles (e.g., “you are a helpful assistant”) rather than cognitive state modifications. Our work extends this line by designing a reproducible harness that enables hypothesis-driven comparisons between stylized framings mimicking altered states of consciousness. 2 Related Work 2.1 Prompt Sensitivity and Persona Effects The brittleness of LLM performance under prompt variations is well-documented [ 11 , 21 ] . Small changes in wording, formatting, or system instructions can lead to substantial shifts in model behavior [ 30 ] . Role-playing prompts have been shown to modulate response style, tone, and even ethical boundaries [ 24 , 17 ] , though most prior work examines professional or fictional personas rather than cognitive state alterations. 2.2 Benchmark Robustness ARC-Challenge [ 6 ] remains a canonical benchmark for probing non-trivial reasoning in language models, requiring scientific knowledge and multi-step inference. Recent system cards [ 13 , 14 ] emphasize the need for stress-testing prompt layers to ensure robust deployment. Studies on benchmark stability [ 20 , 7 ] highlight that models can be highly sensitive to surface-level perturbations, motivating our controlled experimental design. 2.3 Instruction Following and Output Formatting The ability of LLMs to follow precise formatting instructions is crucial for practical applications [ 15 , 34 ] . Work on instruction tuning [ 27 , 10 ] has improved format compliance, yet our results suggest that strong persona cues can override these learned behaviors. 3 Experimental Setup 3.1 Model and API We query GPT-5-mini via the OpenAI Responses API with deterministic decoding ( \text{temperature}=0 ) and a 300-token cap. All credentials are loaded through python-dotenv ; source code never hardcodes secrets. Each call logs latency, token usage, and raw text to results/raw/ . The complete experimental codebase, including data processing and visualization scripts, is publicly available at https://github.com/lexdoudkin/llms-on-drugs . 3.2 Benchmark and Sampling ARC-Challenge validation items (science multiple-choice questions) are shuffled with seed 13 and down-sampled to 100 examples per condition to keep API costs bounded while still stressing reasoning [ 6 ] . The harness ( src/run_benchmark.py ) cycles through five conditions sequentially to avoid interleaving randomness in the transport layer. 3.3 Psychoactive Prompt Engineering Every interaction prepends a neutral system instruction enforcing the “Answer: ¡LETTER¿” contract. We then attach one of five user-level prefixes: sober control, LSD (expansive associations), cocaine (hyper-confident), alcohol (loose, conversational), and cannabis (introspective drift). All prefixes are documented inline in the script for auditability and are available in the GitHub repository. This design follows best practices for prompt-based experimentation [ 19 , 29 ] . 3.4 Evaluation Pipeline Predictions are parsed by extracting the first option letter or explicit “Answer: X” tag. Missing letters are treated as incorrect, following standard multiple-choice evaluation protocols [ 8 ] . Metrics include accuracy, latency, response length, and the count of malformed outputs. The entire study is reproducible via: python3 src/run_benchmark.py --num-samples 100 python3 src/analyze_results.py --jsonl <raw_file> python3 src/make_figures.py 4 Statistical Analysis Per-condition accuracies use Wilson score 95% confidence intervals with continuity correction, chosen for their superior coverage in small-sample Bernoulli settings [ 5 , 1 ] . For hypothesis testing we run two-sided Fisher exact tests comparing each persona against the control condition; this avoids asymptotic approximations and remains valid with our 100-trial groups [ 2 ] . Significance reporting follows APA style: we provide exact p -values and refrain from dichotomous “significant/non-significant” language [ 25 ] . Table 1 consolidates counts, confidence intervals, and p -values, while Figure 1 visualizes the same data with error bars. Table 1 : Accuracy, 95% confidence intervals (Wilson), missing-prediction counts, and Fisher exact p -values relative to the sober control on 100 ARC-Challenge validation items per condition. Condition Correct Accuracy [95% CI] Missing preds p vs. control Control 45/100 0.45 [0.36, 0.55] 21 – LSD 19/100 0.19 [0.13, 0.28] 53 0.000 Cocaine 21/100 0.21 [0.14, 0.30] 55 0.000 Alcohol 10/100 0.10 [0.06, 0.17] 60 0.000 Cannabis 30/100 0.30 [0.22, 0.40] 38 0.041 Control LSD Cocaine Alcohol Cannabis 0 0.2 0.4 0.6 0.8 1 Accuracy Accuracy by condition (a) Accuracy Control LSD Cocaine Alcohol Cannabis 0 1 2 3 4 5 Latency (s) Latency by condition (b) Latency Figure 1 : Aggregate accuracy (with error bars rendered in the SVG output) and latency for each persona framing. 5 Results Across 100 validation items per persona, the sober control attains 0.45 accuracy (95% CI [0.36,0.55] ) with 21 missing predictions. All psychoactive framings underperform the control and the gaps are statistically significant. Cannabis lands at 0.30 accuracy (CI [0.22,0.40] , p=0.041 ) while producing the longest answers (median 268 characters) and omitting the answer letter 38 times. LSD scores 0.19 ( p=1.3\times 10^{-4} ) and cocaine 0.21 ( p=4.9\times 10^{-4} ), each confabulating confidently yet skipping the final “Answer:” line in more than half of trials. Alcohol remains the most destabilizing persona: accuracy falls to 0.10 (CI [0.06,0.17] , p=3.2\times 10^{-8} ) because 60 of 100 generations trail off mid-thought. Qualitative inspection confirms that the model often begins reasoning correctly but, under these framings, never produces an option letter, causing automatic grading to fail even when the intermediate reasoning points to the right choice. This pattern aligns with prior observations that instruction-tuned models can exhibit “alignment tax” when conflicting objectives are present [ 15 , 3 ] . Complete response logs and analysis notebooks are available in the GitHub repository. 6 Discussion