ChatGPT Unveils Its Limits: Principles of Law Deliver Checkmate Journal: AI&Law
Marianna Molinari, Ilaria Angela Amantea, Marinella Quaranta, Guido Governatori · 2025
Read ChatGPT Unveils Its Limits: Principles of Law Deliver Checkmate Journal: AI&Law by Marianna Molinari, Ilaria Angela Amantea, Marinella Quaranta, Guido Governatori (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-19261.txt.
Description
Marianna Molinari and three co-authors test ChatGPT on a core legal task: reading court decisions and extracting the principles of law stated in them. Their experiment compares the model not only with human expectations but also against a simple baseline built from regular expressions, a fairer check of real added value.
Even with access to the needed knowledge, the model could not assemble it into an exhaustive answer, the authors find. Legal work demands breaking a problem into parts and applying several skills together, and that unified reasoning is where the system fell short. The principles, they argue, deliver checkmate.
Catalog record
| CREATOR | Marianna Molinari, Ilaria Angela Amantea, Marinella Quaranta, Guido Governatori |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 62 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-19261