Evaluation of Deontic Conditional Reasoning in Large Language Models: The Case of Wason’s Selection Task
Hirohiko Abe, Kentaro Ozeki, Risako Ando, Takanobu Morishita, Koji Mineshima, Mitsuhiro Okada · 2026
Read Evaluation of Deontic Conditional Reasoning in Large Language Models: The Case of Wason’s Selection Task by Hirohiko Abe, Kentaro Ozeki, Risako Ando, Takanobu Morishita, Koji Mineshima, Mitsuhiro Okada (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-06416.txt.
Description
Abe and colleagues test whether large language models reason differently under deontic versus descriptive rules, using a new Wason task dataset encoding deontic modality. Human reasoning often succeeds better in normative settings than formal ones, yet model domain specificity remains underexplored. Experiments ask whether errors follow confirmation bias, seeking supporting evidence, or matching bias, ignoring negation and picking lexically matching items. Like humans, models reason better with deontic rules and show matching-bias-like errors. Performance thus varies by rule type, with errors paralleling familiar human biases.
Catalog record
| CREATOR | Hirohiko Abe, Kentaro Ozeki, Risako Ando, Takanobu Morishita, Koji Mineshima, Mitsuhiro Okada |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42664 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.06416