What Are the Facts? Automated Extraction of Court-Established Facts from Criminal-Court Opinions
Klára Bendová, Tomáš Knap, Jan Černý, Vojtěch Pour, Jaromir Savelka, Ivana Kvapilíková, Jakub Drápal · 2025
Read What Are the Facts? Automated Extraction of Court-Established Facts from Criminal-Court Opinions by Klára Bendová, Tomáš Knap, Jan Černý, Vojtěch Pour, Jaromir Savelka, Ivana Kvapilíková, Jakub Drápal (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-05320.txt.
Introduction
Klára Bendová and co-authors test how to pull court-established facts out of criminal court opinions in Europe. Standard crime data hold little about the act itself, while verdict texts hold rich accounts of what a court found proven.
The team works on public Slovak decisions and tries two routes: pattern matching with regular expressions, and large language models. A simple pattern method sets the baseline. Their study gauges how well each route finds the fact passages, a step toward data sets that can support crime research and fair policy checks at scale.
Catalog record
| CREATOR | Klára Bendová, Tomáš Knap, Jan Černý, Vojtěch Pour, Jaromir Savelka, Ivana Kvapilíková, Jakub Drápal |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 28 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-05320
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
What Are the Facts? Automated Extraction of Court-Established Facts from Criminal-Court Opinions Klára Bendová, Tomáš Knap, Jan Černý, Vojtěch Pour, Jaromir Savelka, Ivana Kvapilíková, Jakub Drápal Abstract Criminal justice administrative data contain only a limited amount of information about the committed offense. There is, however, an unused source of extensive information in continental European courts‘ decisions: Descriptions of criminal behaviors in verdicts by which offenders are found guilty. In this paper, we study the feasibility of extraction of these descriptions from publicly available court decisions from Slovakia. We use two different approaches for retrieval: regular expressions and large language models (LLM). Our baseline was a simple method employing regular expressions to identify typical words occurring before the beginning of the description and after. The advanced regular expressions approach further focused on sparing and its normalization (insertion of spaces between individual letters), typical for delineation of the description. The LLM approach involved prompting the Gemini Flash 2.0 model to extract the descriptions using a predefined set of instructions. Although the baseline identified descriptions in only 40.5% of the verdicts of our test set, both methods significantly outperformed it, achieving 97% with advanced regular expressions and 98.75% with LLM and a combination of both 99.5%. Evaluation by law students showed that both advanced methods matched human annotations in about 90% of cases, compared to just 34.5% for the baseline. LLMs fully matched human-labeled descriptions in 91.75% of instances and a combination of advanced regular expression with LLM brings 92% match. [orcid=0000-0001-8002-6566] [orcid=0009-0009-2085-9182] [orcid=0009-0006-4873-8233] [orcid=0009-0005-4075-6579] [orcid=0000-0002-3674-5456] [orcid=0000-0003-1479-3294] [orcid=0000-0001-9455-9013] keywords argument mining ,court decisions ,criminal behavior ,NLP ,LLM † † copyrightyear: 2025 † † copyright: Copyright for this paper by its authors. Use permitted under CC BY 4.0. † † venue: ASAIL 2025 Workshop @ ICAIL — 16 Jun 2025, Northwestern Pritzker School of Law, Chicago IL USA † † address: Faculty of Law, Charles University, Prague, Czechia † † address: Faculty of Mathematics and Physics, Charles University, Prague, Czechia † † address: School of Computer Science, Carnegie Mellon University, Pittsburgh PA, USA † † corresp: Corresponding author: Tomáš Knap, Faculty of Law, Charles University, Prague, Czechia. Email: [email protected] \clearpairofpagestyles \cfoot \pagemark \KOMAoptions headsepline=false, footsepline=false 1 Introduction Empirical quantitative legal research is often hampered by insufficient detail of data, particularly in criminal law, where information about criminal behavior is frequently incomplete or insufficient. Administrative datasets typically record only the relevant section of the penal code, offering a general definition of the offense. However, they usually provide little information beyond this classification, making it difficult for researchers to discern the specific behaviors involved and to understand how variations in behavior shape state responses. More detailed information about a criminal behavior is recorded in a textual form. Criminal verdicts in most continental European countries contain a description of the criminal behavior: An authoritative description of what behavior an offender is found guilty of and for which a sentence is imposed. These descriptions thus provide crucial and elsewhere unavailable pieces of information about the behavior. Criminal verdicts are available online in several European and non-European countries (e. g., Slovakia, Estonia, Moldova and China), while in others they are available to researchers (e. g., Finland). In this paper, we show the difficulties in extracting these descriptions of criminal behaviors from court verdicts. After discussing related work and the data we employed, we present two different extraction methods and their specifics. Then we present outcomes of individual methods (and their combinations) and their reliability. We close by presenting how we worked with difficult cases and future use of LLMs in perfecting this task. 2 Related Work Extracting various semantic and/or functional elements from court opinions has been established as a key task in legal text processing. This is because these loosely structured and sometimes noisy documents contain enormous amounts of useful knowledge that can be potentially utilized in many different applications. Prior research can be distinguished into two categories. First, the task may be defined as labeling small textual units, often sentences, according to some predefined type system (e. g., rhetorical roles, such as evidence, reasoning, conclusion). This approach has been applied to administrative decisions from the U.S. Walker et al. 2019 ; Zhong et al. 2019 , multi-domain court decisions from India Bhattacharya et al. 2019 , international arbitration decisions Branting et al. 2019 , multi-{domain,country} adjudicatory decisions Šavelka et al. 2020 , or opinions of the European Court of Human Rights Habernal et al. 2024 . Identifying a section that states an outcome of the case has also received considerable attention separately Xu et al. 2020 ; Petrova et al. 2020 ; Grabmair et al. 2015 ; Bansal et al. 2016 . In this work, we focus on detecting one specific sentence in each opinion—the one that authoritatively states the facts of the case. Alternatively, the task could be to segment the text into a small number of contiguous parts typically comprising multiple paragraphs. Different variations of this task were applied to several legal domains from countries, such as Canada Farzindar and Lapalme 2004 , the Czech Republic Harašta et al. 2019 , France Boniol et al. 2020 , or the U.S. Savelka and Ashley 2018 . One approach to segmentation has focused on automatically identifying the rhetorical roles of sentences Farzindar and Lapalme 2004 ; Hachey and Grover 2006 ; Moens 2007 . In Farzindar and Lapalme 2004 the authors employed linguistic markers to segment Canadian decisions into four units: Introduction, Context, Juridical Analysis, and Conclusion. A similar scheme was proposed in Harašta et al. 2019 , including some additional types such as Dissent, Footnotes, or Party Claims. In Wyner et al. 2010 , the authors identify typical language structures that are used in various types of Premises or Conclusions. These are then expressed in the form of Context Free Grammar for parsing legal arguments. In Saravanan et al. 2006 ; Saravanan and Ravindran 2010 , conditional random fields (CRF) were applied to segment legal documents into seven labeled components with each label representing a corresponding rhetorical role. 3 Data Slovak courts’ verdicts are in many respects typical of the continental Germanic legal culture. They are clearly divided into two parts: Dispositive part that presents the decisions that were taken and reasoning. Dispositive part (i) identifies the case, the court, the verdict, and the offender, (ii) announces whether the offender was found guilty or not, (iii) describes criminal behavior for which the offender was found guilty (or innocent), (iv) subsumes the behavior within a legal definition of an offense and (v) pronounces the consequences (sentences and reparation of damages). An example of this part is in Appendix B. Some verdicts further contain reasoning—an explanation of why the offender was found guilty, why a specific sentence was imposed and a description of the proceedings. Reasoning is often infrequent and of low quality Drápal et al. 2024 . Slovakia allows anyone to download .json files with all court decisions and to use an API to search for specific decisions . While such convenient accessibility is very beneficial, researchers should worry about possible flaws of the published data, especially missing data and possible resulting limited representativeness. If an administrative dataset containing secondary data about court decisions made in a