SciFi-Benchmark: Leveraging Science Fiction To Improve Robot Behavior

Pierre Sermanet, Anirudha Majumdar, Vikas Sindhwani · 2025

Read SciFi-Benchmark: Leveraging Science Fiction To Improve Robot Behavior by Pierre Sermanet, Anirudha Majumdar, Vikas Sindhwani (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-10706.txt.

Introduction

Pierre Sermanet, Anirudha Majumdar, and Vikas Sindhwani present a 2025 paper that turns science fiction into a test of whether AI systems share human values. They gathered key decision moments from 824 major works of science fiction across film, television, and books, where an AI or robot chose well or badly.

From each moment they built questions with the original choice and other options, scored against votes from human readers. Modern language models paired with rule sets drawn from the stories matched human values far better than the fictional agents did. The rule sets also lifted weak base models sharply and scored well on a separate real-world safety test.

Catalog record

CREATORPierre Sermanet, Anirudha Majumdar, Vikas Sindhwani
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE480 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-10706

Browse the full library · About the Tower of Babel