Metadata Extraction Leveraging Large Language Models CCS: Information systems Retrieval models and ranking CCS: Computing methodologies Information extraction CCS: Applied computing Law, social and behavioral sciences
Cuize Han, Sesh Jalagam · 2025
Read Metadata Extraction Leveraging Large Language Models CCS: Information systems Retrieval models and ranking CCS: Computing methodologies Information extraction CCS: Applied computing Law, social and behavioral sciences by Cuize Han, Sesh Jalagam (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-19334.txt.
Introduction
Cuize Han and Sesh Jalagam describe a system for legal contracts. It uses large language models to extract metadata. Contract review is core to contract management. Finding key clauses by hand is slow work. Labeling them is repetitive too. Careful automation fits the task well.
Tests use a public contract dataset and private contracts. The system rests on three elements. First, solid conversion of documents to text. Second, smart selection of text chunks. Third, model techniques such as step-by-step prompting. The authors name what makes extraction reliable. Legal technology teams get a concrete recipe.
Catalog record
| CREATOR | Cuize Han, Sesh Jalagam |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-19334
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Metadata Extraction Leveraging Large Language Models CCS: Information systems Retrieval models and ranking CCS: Computing methodologies Information extraction CCS: Applied computing Law, social and behavioral sciences Cuize Han, Sesh Jalagam Abstract . The advent of Large Language Models has revolutionized tasks across domains, including the automation of legal document analysis—a critical component of modern contract management systems. This paper presents a comprehensive implementation of LLM-enhanced metadata extraction for contract review, focusing on the automatic detection and annotation of salient legal clauses. Leveraging both the publicly available Contract Understanding Atticus Dataset (CUAD) and proprietary contract datasets, our work demonstrates the integration of advanced LLM methodologies with practical applications. We identify three pivotal elements for optimizing metadata extraction: robust text conversion, strategic chunk selection, and advanced LLM-specific techniques, including Chain of Thought (CoT) prompting and structured tool calling. The results from our experiments highlight the substantial improvements in clause identification accuracy and efficiency. Our approach shows promise in reducing the time and cost associated with contract review while maintaining high accuracy in legal clause identification. The results suggest that carefully optimized LLM systems could serve as valuable tools for legal professionals, potentially increasing access to efficient contract review services for organizations of all sizes. 1. Introduction 1.1. Metadata Extraction Metadata extraction is the automated process of identifying and extracting structured information from unstructured or semi-structured documents. In today’s digital age, this task has become increasingly critical, particularly in legal and enterprise applications. Automated clause extraction and analysis from legal contracts has become increasingly critical in modern legal practice, where law firms spend approximately 50% of their time on contract review ( Hendrycks et al. 2021 ) . This task involves identifying and extracting structured information such as governing law clauses, termination provisions, and liability terms from complex legal documents. With billing rates for lawyers at large law firms typically ranging from $500-$900 per hour in the US ( Hendrycks et al. 2021 ) , manual contract review creates substantial costs for organizations. Furthermore, the prohibitive cost of legal review means many small businesses and individuals often sign contracts without thorough analysis, potentially exposing themselves to unfavorable terms ( upcounsel 2024 ) . In the legal domain, automatic contract analysis requires precise extraction of key elements such as parties involved, effective dates, termination clauses, and financial terms ( Sleimi et al. 2018 ) . This automation not only accelerates contract review processes but also reduces human error and enables large-scale contract analytics ( Sleimi et al. 2021 ) . Beyond legal applications, metadata extraction serves as a fundamental building block for various enterprise document management tasks. For instance, organizations can automatically classify documents based on their security levels by extracting and analyzing specific metadata fields ( Sajid et al. 2023 ) . This capability is crucial for maintaining compliance with data protection regulations and implementing proper access controls. Furthermore, metadata extraction facilitates efficient document retrieval, content organization, and workflow automation across different business units ( Haynes 2004 ) . 1.2. Large Language Models Large Language Models (LLMs) represent a significant advancement in natural language processing, characterized by their ability to understand and generate human-like text through massive-scale pre-training on diverse datasets ( Minaee et al. 2024 ) . These models, such as GPT-4, Claude, and Gemini, have demonstrated remarkable capabilities in various language understanding and generation tasks ( Chang et al. 2024 ) . For metadata extraction, LLMs offer unique advantages over traditional rule-based or specialized machine learning approaches. Their deep contextual understanding enables them to accurately identify relevant information even when it appears in varying formats or requires complex reasoning. Importantly, our approach does not involve fine-tuning LLMs specifically for metadata extraction tasks. Instead, we utilize LLMs as the final component in our pipeline, where they analyze preprocessed text chunks and generate structured JSON outputs containing the extracted metadata. This architecture offers several key benefits: • Flexibility to experiment with and switch between different LLM providers. • Ability to leverage continuous improvements in base models without retraining. • Reduced dependency on task-specific training data. • Enhanced maintainability and adaptability of the system. 1.3. Related Works Previous research in metadata extraction can be broadly categorized into several approaches: Traditional Rule-based Systems: Early works relied heavily on regular expressions and predefined patterns ( Tang et al. 2010 ) , which proved brittle when faced with document variations. Machine Learning Approaches: Researchers have explored various ML techniques including: Conditional Random Fields (CRF) for sequence labeling ( Souza et al. 2014 ) , Deep learning models specifically trained for extraction tasks ( Liu et al. 2018 ) , Transformer-based approaches with fine-tuning ( La Quatra and Cagliero 2022 ) . LLM Integration: Recent work has begun exploring the use of LLMs for extraction ( Savelka 2023 ) . Our Contributions: Based on our research and experimentation, we make the following key contributions: A comprehensive framework for metadata extraction that effectively combines text preprocessing, chunk selection, and LLM-based extraction Novel techniques for optimal chunk selection that maximize extraction accuracy while minimizing LLM token usage Empirical evaluation of various LLM prompting strategies, including Chain of Thought (CoT) and structured output generation Extensive experimentation on both proprietary and public datasets, demonstrating the effectiveness and generalizability of our approach Practical insights into implementing and scaling metadata extraction systems in enterprise environments 2. Metadata Extraction Workflow Figure 1. Simplified workflow of metadata extraction as of 09/01/2024. including text conversion, chunk selection, prompting, offline evaluation, online grading, retrying, multi-modal extraction, automatic template enhancement, etc. Figure 1 presents a comprehensive overview of our metadata extraction system. The pipeline begins with an input document (typically in PDF format) and produces structured metadata output in JSON format, with optional quality assessment through LLM-based grading. Through extensive experimentation and deployment experience, we have identified three critical components that significantly impact both the quality and operational costs of metadata extraction: (1) text conversion and OCR preprocessing, (2) strategic chunk selection, and (3) LLM-based summarization and structuring. While emerging multi-modal LLMs offer the possibility of direct metadata extraction from document images, our experiments indicate that this approach currently incurs prohibitively high costs for production deployment. Instead, our system employs a more efficient pipeline approach, with each component optimized for its specific task. The system also incorporates an innovative online grading mechanism using LLMs to assess extraction quality, enabling continuous monitoring and quality assurance. In the following subsections, we examine each key component in detail, discussing their implementation, challenges, and our solutions for optimizing their performance. We also present empirical evidence demonstrating how these components collectively contribute to achieving high-quality metadata extraction while maintaining practical operational costs. 2.1. Text conversion and OCR Text conversion and Optical Character Recognition (OCR) serve as the crucial first step in our metadata extraction pipeline. This component transforms various document formats, particularly PDFs, into machine-readable text while preserving essential structural information such as layout and formatting. The quality of this conversion directly impacts all subsequent steps in the pipeline, making it a fundamental determinant of the overall extraction accuracy. For metadata extraction tasks, high-quality text conversion is particularly important for several reasons. First, accurate text extraction ensures that key information remains intact and properly sequenced, which is essential for LLMs to understand the document