Seeing Justice Clearly: Handwritten Legal Document Translation with OCR and Vision-Language Models
Shubham Kumar Nigam, Parjanya Aditya Shukla, Noel Shallum, Arnab Bhattacharya · 2025
Read Seeing Justice Clearly: Handwritten Legal Document Translation with OCR and Vision-Language Models by Shubham Kumar Nigam, Parjanya Aditya Shukla, Noel Shallum, Arnab Bhattacharya (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-18004.txt.
Introduction
Shubham Kumar Nigam, Parjanya Aditya Shukla, Noel Shallum, and Arnab Bhattacharya tackle a backlog in India's courts: piles of handwritten legal papers in Marathi. Police reports, charge sheets, and witness statements must be read and translated before many people can use them. Marathi has few digital training sets, and handwriting varies a lot.
The team compares two routes. One reads the page with a text scanner and then translates the text. The other uses a vision language model to go from page image to translation in one step. Their tests on real legal pages weigh accuracy and ease of use, aiming at tools that can run on modest hardware in district courts.
Catalog record
| CREATOR | Shubham Kumar Nigam, Parjanya Aditya Shukla, Noel Shallum, Arnab Bhattacharya |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 19 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-18004
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Seeing Justice Clearly: Handwritten Legal Document Translation with OCR and Vision-Language Models Shubham Kumar Nigam, Parjanya Aditya Shukla, Noel Shallum, Arnab Bhattacharya Abstract Handwritten text recognition (HTR) and machine translation continue to pose significant challenges, particularly for low-resource languages like Marathi, which lack large digitized corpora and exhibit high variability in handwriting styles. The conventional approach to address this involves a two-stage pipeline: an OCR system extracts text from handwritten images, which is then translated into the target language using a machine translation model. In this work, we explore and compare the performance of traditional OCR-MT pipelines with Vision Large Language Models that aim to unify these stages and directly translate handwritten text images in a single, end-to-end step. Our motivation is grounded in the urgent need for scalable, accurate translation systems to digitize legal records such as FIRs, charge sheets, and witness statements in India’s district and high courts. We evaluate both approaches on a curated dataset of handwritten Marathi legal documents, with the goal of enabling efficient legal document processing, even in low-resource environments. Our findings offer actionable insights toward building robust, edge-deployable solutions that enhance access to legal information for non-native speakers and legal professionals alike. Code — https://github.com/anviksha-lab-iitk/SJC Introduction Figure 1: Comparison of OCR-MT and vLLM-based approaches for handwritten text translation. The OCR-MT pipeline decomposes the task into separate HTR and MT stages, whereas vLLMs unify the process into a single end-to-end step. The Indian judiciary, one of the world’s most complex legal systems, continues to face challenges in ensuring timely justice and efficient case handling. A major bottleneck lies in its persistent reliance on handwritten documentation at the grassroots level, such as district courts and police stations, where First Information Reports (FIRs), case diaries, witness statements, and court proceedings are still manually recorded. These documents are critical to criminal and civil proceedings, but their handwritten, unstructured nature makes them difficult to archive, search, and analyze. Variability in handwriting, language diversity, legal terminology, and the poor quality of scans all pose significant hurdles to digitization. This work lays the foundation for building an automated legal document digitization pipeline aimed at enabling structured and accessible digital case records. Our focus is on automatically converting handwritten legal documents in Marathi into English through two distinct paradigms: (i) a modular OCR + Machine Translation (OCR-MT) pipeline, and (ii) direct translation using Vision Large Language Models (vLLMs). The objective is to assess their performance, robustness, and suitability for deployment in low-resource legal environments. Traditional Optical Character Recognition (OCR) systems, such as Tesseract ( 21 ) , EasyOCR ( 9 ) , and PaddleOCR ( 14 ) , follow a multi-stage pipeline involving text detection, segmentation, and recognition . These systems perform reasonably well for printed documents but often fail on handwritten legal content due to limited generalization, poor handwriting support, and a lack of layout-awareness. Moreover, they are not designed to work with low-resource languages like Marathi unless extensively retrained. Our work evaluates these tools on a custom dataset of handwritten Marathi legal documents and investigates their effectiveness when paired with modern MT models like IndicTrans2 ( 6 ) and Sarvam-1 ( 20 ) . However, OCR-MT pipelines suffer from cascading errors; misrecognized words by OCR adversely affect translation quality. This motivates the use of vLLMs, which can jointly process image and text inputs, reducing dependency on rigid pipeline stages. Models such as Chitrarth ( 13 ) , Ovis2 ( 15 ) , and Maya ( 1 ) are capable of zero-shot visual reasoning and multilingual output generation. These models offer an attractive alternative for legal text digitization, particularly when working with noisy or incomplete inputs. The key contributions of this work are: • OCR Evaluation: We benchmark Tesseract, EasyOCR, and PaddleOCR on a curated dataset of handwritten Marathi legal documents. • MT Analysis: Analyze translation performance using IndicTrans2 and Sarvam-1 models on OCR-extracted text. • vLLM Benchmarking: We compare three vision-language models against OCR-MT pipelines, highlighting their ability to perform direct image-to-English translation. Related Work The literature relevant to our task spans four key areas: (1) advancements in Optical Character Recognition (OCR) systems, especially those capable of processing diverse document layouts; (2) developments in machine translation (MT) for Indian languages; (3) systems for handwritten and printed text extraction in Indian legal documents; and (4) progress in vLLMs that perform tasks such as image captioning, visual QA, and multimodal translation. Recent OCR systems incorporate layout and spatial awareness for enhanced recognition. VISTA-OCR ( 7 ) introduces a generative, layout-aware OCR pipeline using an encoder-decoder framework. olmOCR ( 17 ) leverages document-anchoring and fine-tunes the Qwen2-VL-7B-Instruct vLLM to extract structured information. Another approach, +FIRST ( 8 ) , improves multimodal transcription by combining OCR outputs from full documents with image features of only the first page. Systems such as Nirnayak ( 5 ) and related pipelines ( 19 ) apply OCR for downstream tasks like translation and summarization in the Indian legal domain. However, their reliance on OCR introduces error propagation, limiting end-to-end accuracy. This motivates the use of vision-language models that can jointly reason over visual and textual modalities. Several models have advanced MT for Indian languages. Sarvam-1 ( 20 ) is a 2B parameter model optimized for 10 Indian languages and English, demonstrating strong performance through careful data curation. IndicTrans2 ( 6 ) , based on a transformer encoder-decoder architecture, supports all 22 scheduled languages. Other efforts include Anuvaad ( 18 ) and Nemotron-4-Mini-Hindi-4B ( 10 ) , the latter trained via continued pre-training for bilingual MT. Recent vLLMs such as LLaMA 4 ( 16 ) , Qwen2.5-VL ( 3 ) , GPT-4V ( 23 ) , Gemini 2.0 Flash, and PaliGemma 2 ( 22 ) demonstrate strong multimodal reasoning capabilities but often require high-end resources, making them unsuitable for deployment in low-resource legal infrastructures. Lightweight models like Chitrarth ( 13 ) , Ovis ( 15 ) , and Maya ( 1 ) balance efficiency and accuracy and are better suited for real-world deployment in district courts. Our work evaluates such models under zero-shot prompting for handwritten document translation. PLATTER ( 12 ) offers an end-to-end handwritten OCR framework with two-stage processing (handwritten text detection and recognition) and supports 10 Indian languages. TransDocAnalyser ( 4 ) is tailored to legal F.I.R. documents, combining a FastRCNN+Vision Transformer encoder with a BERT-based decoder fine-tuned for legal vocabulary. Our work differs by focusing on translation from handwritten legal documents and comparing OCR-MT and vLLM approaches under a common benchmark. Dataset We utilize a custom dataset curated to reflect real-world legal document scenarios. The dataset comprises approximately 60 scanned PDF documents written in Marathi, collected from authentic legal sources. These documents vary in length and structure, including both single-page and multi-page entries. Each page in the dataset contains handwritten Marathi text, often interspersed with printed text. The documents also include diverse visual elements such as official stamps, seals, signatures, and structured tables, which introduce additional challenges for both text recognition and translation. To establish a reliable ground truth, the Marathi text was manually translated into English by a team of two native Marathi speakers. These translations were subsequently reviewed by a legal language expert to ensure fidelity, contextual accuracy, and terminological consistency. The