The Hidden Structure - Improving Legal Document Understanding Through Explicit Text Formatting

Christian Braun, Alexander Lilienbeck, Daniel Mentjukov · 2025

Read The Hidden Structure - Improving Legal Document Understanding Through Explicit Text Formatting by Christian Braun, Alexander Lilienbeck, Daniel Mentjukov (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-12837.txt.

Introduction

Contracts have a built-in structure of sections and clauses that lawyers rely on, yet language models often receive the text flattened and messy. Christian Braun and co-authors test how much that structure matters in this 2025 study, feeding a top model legal questions in five formats of the same contract text.

One model shrugged off format changes but scored modestly overall. A newer model swung widely: clean structure lifted its exact-match accuracy by about twenty points, and a tuned system prompt added more, with Markdown scoring best. The lesson for legal tech is plain: tidy input and clear instructions still drive results.

Catalog record

CREATORChristian Braun, Alexander Lilienbeck, Daniel Mentjukov
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE58 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-12837

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

The Hidden Structure - Improving Legal Document Understanding Through Explicit Text Formatting Christian Braun, Alexander Lilienbeck, Daniel Mentjukov Abstract The integration of Large Language Models (LLMs) into legal practice presents both transformative opportunities and significant risks, underscoring the need for rigorous benchmarking and a nuanced understanding of factors influencing their performance. Legal documents, particularly contracts, possess an inherent, semantically vital structure (e.g. sections, clauses) that is crucial for human comprehension but whose impact on LLM processing remains under-explored. This paper investigates the effects of explicit input text structure and prompt engineering on the performance of two LLM versions (GPT-4o and GPT-4.1, via Microsoft Azure) on a legal question-answering task using an excerpt of the Contract Understanding Atticus Dataset (CUAD) . We compare model exact-match accuracy (EM-Acc.) across various input formats: original well-structured plain-text (human-generated from CUAD), plain-text cleaned of line breaks, extracted plain-text from Azure OCR, plain-text extracted by GPT-4o Vision, and extracted (and interpreted) Markdown (MD) from GPT-4o Vision. To give an indication of the impact of possible prompt engineering, we assess the impact of shifting task instructions to the system prompt and explicitly informing the model about the structured nature of the input (e.g., "You will receive structured Markdown // Plain Text"). Our findings reveal that GPT-4o exhibits considerable insensitivity to variations in input structure but shows limited overall performance. Conversely, GPT-4.1’s performance is highly sensitive to structure; poorly structured inputs yield suboptimal results (but identical with GPT-4o), while well-structured formats (original CUAD text, GPT-4o Vision text and GPT-4o MD) improve exact-match accuracy by approximately 20 percentage points. Optimizing the system prompt to include task details and an advisory about structured input further elevates GPT-4.1’s accuracy by an additional 10-13 percentage points, with Markdown ultimately achieving the highest performance under these conditions (79 percentage points overall exact-match accuracy). This research empirically demonstrates that while commonly used models exhibit greater resilience, careful input structuring and strategic prompt design remain critical for optimizing the performance of models like GPT-4.1, significantly influencing outcomes in high-stakes legal applications. We highlight the often-overlooked impact of such "minor" variations, urging critical scrutiny of all factors shaping LLM behavior in sensitive domains. Keywords: Vision Markdown, Legal AI, Prompt Engineering, AI in Legal Practice, CUAD Dataset, Legal Question Answering with LLMs 1 Introduction The rapid expansion of Large Language Models (LLMs) offers transformative potential for numerous sectors, not least the field of law, where their integration has the potential to enhance—and in some cases fundamentally alter—established methods of legal practice and analysis. Yet, this growing enthusiasm is necessarily tempered by a clear recognition of the substantial risks associated with deploying such powerful, yet incompletely understood, technologies into the legal domain [ 10 ] . Existing research has thoroughly documented the tendency of LLMs to produce content that may be offensive, misleading, or factually inaccurate [ 2 , 20 ] . The potential for such errant behaviors to arise within legal applications [ 34 ] raises legitimate concerns regarding substantial societal harms [ 45 ] , with a disproportionate burden potentially falling upon historically marginalized and under-resourced communities likely to bear a disproportionate share of the impact [ 39 , 41 ] . Accordingly, the need for robust infrastructure and rigorous methodologies to benchmark LLM performance and ensure their safe deployment in legal contexts is both urgent and essential. Despite the urgency of this need, both practitioners and researchers face significant obstacles in evaluating the ability of LLMs to execute complex legal reasoning tasks or furnish verifiably accurate responses to legal queries [ 12 ] . A critical yet often underexamined aspect of this challenge lies in the structural characteristics of legal texts. Legal documents—contracts in particular—are not unstructured bodies of prose but are systematically organized through a formal hierarchy of titles, sections, subsections, clauses, and enumerated provisions [ 43 ] . This formal organization is not merely a matter of stylistic convention; rather, it carries profound semantic import, defining the scope of agreements, articulates obligations, outlines exceptions, and ultimately governs legal interpretation and enforcement. While contemporary LLMs have demonstrated impressive capabilities in processing vast quantities of natural language, the degree to which their analytical performance depends on— or is enhanced by—explicit structural cues in the input text remains insufficiently understood, particularly in the complex and high-stakes context of legal analysis [ 19 ] . While it is often assumed that these models can implicitly infer underlying structure from unformatted text [ 40 ] , however, the validity of this assumption may not extend uniformly across different model architectures or generations. Moreover, this capability cannot be reliably presumed when input texts are degraded, whether due to errors introduced by Optical Character Recognition (OCR) or through overly aggressive pre-processing aimed at standardizing heterogeneous sources [ 13 , 17 , 25 ] . In this paper, we endeavor to contribute to this critical area of inquiry through a focused empirical investigation. Our approach is primarily twofold: 1. We examine the hypothesis ( H1 ) that the explicit structure of input text—ranging from comparatively unstructured OCR output to well-formatted plain text (TXT) and syntactically rich Markdown (MD)—materially affects the performance of two commonly utilized LLM versions deployed on the Microsoft Azure platform (GPT-4o and GPT-4.1) when applied to a standardized legal question-answering task (excerpt of CUAD dataset). 2. We investigate the impact of specific, but simple prompt engineering strategies ( H2 ), particularly the provision of meta-information to the model regarding the structured nature of its input (e.g., explicitly conveying "You will receive structured Markdown // Plain Text"). We evaluate these hypotheses empirically using an excerpt of the tasks of the Contract Understanding Atticus Dataset (CUAD) [ 18 ] , a widely used benchmark, assessing performance based on the model’s accuracy in identifying and citing relevant contractual clauses—a core competency in legal practice. The principal contributions of this study, therefore, can be summarized as follows: • We present a systematic, comparative analysis of GPT-4o and GPT-4.1 performance across a spectrum of input text structures derived from authentic legal documents, thereby offering insights into model-specific sensitivities to data representation. • We furnish empirical evidence demonstrating significant performance enhancements for the GPT-4.1 model when supplied with appropriately structured input / text formatting. • We offer a nuanced elaboration of the intricate interplay between input format (specifically, plain TXT versus Markdown), model version, and potential prompt design (or marginal changes to prompt design) within the specialized context of legal document analysis, contributing to a more granular understanding of factors influencing LLM efficacy in this domain. By illuminating these dynamics, this research seeks to inform best practices for preparing legal texts for LLM processing and to emphasize the enduring importance of careful data curation / structuring texts and also minor changes in prompt design, even as language models continue to advance in capability. Before progressing further, we emphasize that the purpose of this work is to draw attention to the often-overlooked influence of seemingly minor variations between model versions—particularly in contexts where prevailing assumptions may obscure meaningful differences, such as the impact of structured text formatting in newly released LLMs. Legal practitioners and organizations should be aware that small modifications in input preparation, prompt design, or even the technical implementation of prompts across different

Read the complete text