Training Kindai OCR with parallel textline images and self-attention feature distance-based loss

Anh Le, Asanobu Kitamoto · 2025

Read Training Kindai OCR with parallel textline images and self-attention feature distance-based loss by Anh Le, Asanobu Kitamoto (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-08537.txt.

Introduction

Anh Le and Asanobu Kitamoto wrote this paper in two thousand twenty-five. It trains optical character recognition for Kindai texts, Japanese print from the late nineteenth to early twentieth century, where marked training data is scarce.

The team pairs lines of old print with the same lines set in new fonts, to grow the training stock. A loss term pulls the self-attention traits of each pair close, by one of two distance gauges. Both gauges cut the character error rate past a transformer baseline, the second by more. The traits also grow cleaner for telling glyphs apart. Such gains open old records on daily life and society to search.

Catalog record

CREATORAnh Le, Asanobu Kitamoto
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-08537

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Training Kindai OCR with parallel textline images and self-attention feature distance-based loss Anh Le, Asanobu Kitamoto Abstract Kindai documents, written in modern Japanese from the late 19th to early 20th century, hold significant historical value for researchers studying societal structures, daily life, and environmental conditions of that period. However, transcribing these documents remains a labor-intensive and time-consuming task, resulting in limited annotated data for training optical character recognition (OCR) systems. This research addresses this challenge of data scarcity by leveraging parallel textline images—pairs of original Kindai text and their counterparts in contemporary Japanese fonts—to augment training datasets. We introduce a distance-based objective function that minimizes the gap between self-attention features of the parallel image pairs. Specifically, we explore Euclidean distance and Maximum Mean Discrepancy (MMD) as domain adaptation metrics. Experimental results demonstrate that our method reduces the character error rate (CER) by 2.23% and 3.94% over a Transformer-based OCR baseline when using Euclidean distance and MMD, respectively. Furthermore, our approach improves the discriminative quality of self-attention representations, leading to more effective OCR performance for historical documents. keywords Kindai OCR, parallel textline images, self-attention feature distance-based loss 1 Introduction Since historical documents are an invaluable resource for historians in exploring social aspects, daily life, politics, and even climate patterns of past eras, many countries have prioritized their preservation. A common approach is the creation of digital libraries that store high-resolution scans of these documents and make them accessible to both domestic and international researchers. Traditionally, the digitization process involved scanning printed or handwritten texts into image format, followed by manual transcription by experts. However, this method is extremely time-consuming, labor-intensive, and impractical for the vast collections housed in archives and libraries. To address these challenges, document analysis and recognition technologies have emerged as vital tools. These systems leverage machine learning and computer vision to automatically detect layout structures, recognize handwritten or printed characters, and convert scanned pages into machine-readable text. This automation significantly accelerates the digitization workflow, reduces dependence on scarce experts, and makes large-scale transcription and analysis feasible, thus greatly enhancing access to historical knowledge for scholars around the world. In Japan, historical documents from the late 19th to early 20th century are classified as Kindai documents, referring to the ”modern” period just before full modernization. These materials are critically important for scholars studying Japan’s transition into a modern state, and they encompass a wide range of content, including government records, newspapers, magazines, books, and legal documents. Due to their historical and cultural significance, digitizing and making these documents machine-readable is a priority. However, this task presents significant technical challenges. Document recognition systems for Kindai materials typically operate in two main stages: textline detection and textline recognition. In the first stage, the system analyzes scanned pages to identify and isolate individual lines of text. Then, during recognition, each line is analyzed and converted into character codes, often in Unicode. This process is more complicated than it sounds, especially with Kindai documents, which frequently exhibit complex layouts such as multi-column formats, vertical writing, annotations in side margins. Furthermore, Kindai texts use varied typography, antiquated kanji characters, and orthographic variations, including obsolete kana usage and stylistic ligatures that are uncommon or absent in contemporary writing systems. Another primary challenge in processing historical Japanese documents stems from the visual similarity among certain characters and the extensive vocabulary, which includes numerous kanji characters, many of which are rare or not included in modern standardized character sets. This results in a high incidence of out-of-vocabulary (OOV) characters, posing significant difficulties for conventional optical character recognition (OCR) systems. A further impediment to progress is the limited availability of large-scale, high-quality annotated datasets. For example, the publicly accessible dataset provided by the National Diet Library of Japan comprises only approximately 4,000 pages of annotated Kindai-era materials. This volume is inadequate for training robust deep learning models, which typically require tens or hundreds of thousands of labeled instances to achieve satisfactory generalization. Figure 1 presents an example of a Kindai document, where both the title and the main text are written in vertical script. Figure 1: Example of Kindai document. To address this data scarcity, one promising direction of research is to leverage synthetic data generated from modern Japanese fonts. By using font files to generate artificial textline images that resemble real Kindai textlines in style and complexity, researchers can train models in a supervised setting without requiring extensive manual annotation. The core idea is to perform domain adaptation, where the model learns useful patterns from synthetic font-based images and then applies this knowledge to recognize real Kindai document images. This approach enables the development of more robust and accurate recognition systems, even with limited real-world training data. Such cross-domain learning strategies, if further refined, could significantly enhance the usability and accessibility of Kindai documents. In turn, this would accelerate scholarly research across disciplines such as history, literature, linguistics, and political science by making vast volumes of previously unreadable texts searchable, translatable, and analyzable in digital form. In this paper, we present a novel approach to enhancing the performance of text recognition systems for Kindai documents by addressing the critical limitation of insufficient annotated training data. Specifically, we target on improving the text recognition stage of the OCR pipeline by leveraging parallel textline images—a curated dataset composed of pairs of corresponding textline images from original Kindai documents and synthesized counterparts rendered using contemporary Japanese fonts. These parallel pairs serve as a bridge between the scarce Kindai data and the more readily available modern Japanese font resources. To effectively utilize these parallel images for training, we propose a distance-based objective function operating on the self-attention features extracted by the Transformer-based OCR model. The core idea is to minimize the feature-space distance between each pair of parallel textline images. By aligning the internal representations of historical and synthetic textlines, the model is encouraged to learn domain-invariant features, enabling better generalization to real Kindai text under low-resource conditions. This adaptation framework allows the model to effectively benefit from the large corpus of synthetic textlines generated from modern Japanese fonts, thereby mitigating the data scarcity issue without requiring extensive manual annotation of historical texts. Our experiments demonstrate the effectiveness of this method in improving recognition accuracy on Kindai textline images. Incorporating both the parallel textline dataset and the proposed distance-based training objective, our model outperforms baselines that rely solely on either synthetic or limited real data. Furthermore, qualitative results show improved character-level accuracy, particularly in handling complex kanji and stylistic variations characteristic of Kindai documents. The remainder of this paper is organized as follows: Section 2 reviews related work in the areas of historical Japanese document recognition and domain adaptation techniques. Section 3 describes the architecture of our Transformer-based OCR model, the generation process of parallel textline images, and the formulation of our distance-based objective function. Section 4 presents experimental setups and results. Finally, Section 5 concludes the paper and outlines potential directions for future work. 2 Related Works For training the Kindai OCR system, we utilized the Shisou dataset, comprising 1,000 annotated images. This dataset was provided by the Center for Research and Development of Higher Education at the University of Tokyo. However, it is important to note that the Shisou

Read the complete text