Beware Untrusted Simulators – Reward-Free Backdoor Attacks in Reinforcement Learning

Ethan Rathbun, Wo Wei Lin, Alina Oprea, Christopher Amato · 2026

Read Beware Untrusted Simulators – Reward-Free Backdoor Attacks in Reinforcement Learning by Ethan Rathbun, Wo Wei Lin, Alina Oprea, Christopher Amato (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-05089.txt.

Description

Reinforcement learning agents learn in simulators, and Ethan Rathbun and 3 co-authors show in this 2026 security paper that a rigged simulator can secretly corrupt them. Earlier backdoor attacks assumed the attacker controls training and can see or alter rewards, which is unrealistic once a simulator is released to the world.

Their attack, Daze, poisons only the simulator's dynamics. An agent trained in it picks up a hidden trigger: when a preset pattern appears, it reliably performs a target action the attacker chose. The authors prove the attack works across general tasks and test it on discrete and continuous control problems. They also show, for the first time, an RL backdoor transferring to real robot hardware. The paper closes by urging security checks on every part of the RL training pipeline.

Catalog record

CREATOREthan Rathbun, Wo Wei Lin, Alina Oprea, Christopher Amato
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE99 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-05089

Browse the full library · About the Tower of Babel