ChatGPT Unveils Its Limits: Principles of Law Deliver Checkmate Journal: AI&Law

Marianna Molinari, Ilaria Angela Amantea, Marinella Quaranta, Guido Governatori · 2025

Read ChatGPT Unveils Its Limits: Principles of Law Deliver Checkmate Journal: AI&Law by Marianna Molinari, Ilaria Angela Amantea, Marinella Quaranta, Guido Governatori (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-19261.txt.

Description

Marianna Molinari and three co-authors test ChatGPT on a core legal task: reading court decisions and extracting the principles of law stated in them. Their experiment compares the model not only with human expectations but also against a simple baseline built from regular expressions, a fairer check of real added value.

Even with access to the needed knowledge, the model could not assemble it into an exhaustive answer, the authors find. Legal work demands breaking a problem into parts and applying several skills together, and that unified reasoning is where the system fell short. The principles, they argue, deliver checkmate.

Catalog record

CREATORMarianna Molinari, Ilaria Angela Amantea, Marinella Quaranta, Guido Governatori
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-19261

Browse the full library · About the Tower of Babel