Evaluation of Deontic Conditional Reasoning in Large Language Models: The Case of Wason’s Selection Task

Hirohiko Abe, Kentaro Ozeki, Risako Ando, Takanobu Morishita, Koji Mineshima, Mitsuhiro Okada · 2026

Read Evaluation of Deontic Conditional Reasoning in Large Language Models: The Case of Wason’s Selection Task by Hirohiko Abe, Kentaro Ozeki, Risako Ando, Takanobu Morishita, Koji Mineshima, Mitsuhiro Okada (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-06416.txt.

Description

Abe and colleagues test whether large language models reason differently under deontic versus descriptive rules, using a new Wason task dataset encoding deontic modality. Human reasoning often succeeds better in normative settings than formal ones, yet model domain specificity remains underexplored. Experiments ask whether errors follow confirmation bias, seeking supporting evidence, or matching bias, ignoring negation and picking lexically matching items. Like humans, models reason better with deontic rules and show matching-bias-like errors. Performance thus varies by rule type, with errors paralleling familiar human biases.

Catalog record

CREATORHirohiko Abe, Kentaro Ozeki, Risako Ando, Takanobu Morishita, Koji Mineshima, Mitsuhiro Okada
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42664
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.06416

Browse the full library · About the Tower of Babel