Adapting TrOCR for Printed Tigrinya Text Recognition: Word-Aware Loss Weighting for Cross-Script Transfer Learning
Yonatan Haile Medhanie, Yuanhua Ni · 2026
Read Adapting TrOCR for Printed Tigrinya Text Recognition: Word-Aware Loss Weighting for Cross-Script Transfer Learning by Yonatan Haile Medhanie, Yuanhua Ni (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-20813.txt.
Introduction
Medhanie and Ni bring TrOCR to printed Tigrinya first, in the Ge'ez script. From a pre-trained model, they widen the BPE tokenizer for 230 Ge'ez characters and add Word-Aware Loss Weighting against word-boundary failures from Latin BPE conventions. After adaptation, the TrOCR-Printed variant achieves 0.22 percent Character Error Rate and about 97 percent match accuracy on GLOCR synthetic images. Ablation shows Word-Aware Loss Weighting is the critical piece, cutting CER by two orders of magnitude.
Catalog record
| CREATOR | Yonatan Haile Medhanie, Yuanhua Ni |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 28616 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.20813
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Adapting TrOCR for Printed Tigrinya Text Recognition: Word-Aware Loss Weighting for Cross-Script Transfer Learning Yonatan Haile Medhanie, Yuanhua Ni Abstract Transformer-based OCR models have shown strong performance on Latin and CJK scripts, but their application to African syllabic writing systems remains limited. We present the first adaptation of TrOCR for printed Tigrinya using the Ge’ez script. Starting from a pre-trained model, we extend the byte-level BPE tokenizer to cover 230 Ge’ez characters and introduce Word-Aware Loss Weighting to resolve systematic word-boundary failures that arise when applying Latin-centric BPE conventions to a new script. The unmodified model produces no usable output on Ge’ez text. After adaptation, the TrOCR-Printed variant achieves 0.22% Character Error Rate and 97.20% exact match accuracy on a held-out test set of 5,000 synthetic images from the GLOCR dataset. An ablation study confirms that Word-Aware Loss Weighting is the critical component, reducing CER by two orders of magnitude compared to vocabulary extension alone. The full pipeline trains in under three hours on a single 8 GB consumer GPU. All code, model weights, and evaluation scripts are publicly released. Keywords: Optical Character Recognition, Tigrinya, TrOCR, Transformer, Transfer Learning, Low-Resource Languages, Ge'ez Script, Deep Learning 1 Introduction Tigrinya is a Semitic language spoken by more than 10 million people in Eritrea and the Tigray region of Ethiopia [ 1 ] . It is written in the Ge'ez script, an abugida in which each grapheme represents a consonant–vowel combination. The full Tigrinya character inventory exceeds 230 unique symbols, many of which differ only by subtle strokes or diacritical modifications. Despite its large speaker population and official status in Eritrea, Tigrinya remains poorly served by digital language technologies. Commercial OCR platforms provide no dedicated support for the language. Recent advances in end-to-end Transformer-based OCR, particularly TrOCR [ 2 ] , have demonstrated that pairing a Vision Transformer (ViT) encoder with a language model decoder enables strong text recognition through transfer learning. This architecture has been successfully adapted to non-Latin scripts including Urdu [ 3 ] , Tamil [ 4 ] , Manchu [ 5 ] , and Spanish [ 6 ] . For the Ge'ez script family, however, no Transformer-based evaluation exists. Prior Ge'ez script OCR work has relied on CNN-RNN architectures. Belay et al. [ 7 ] achieved 0.93% CER on Amharic using an Attention-CTC model, while Hailu et al. [ 8 ] reported 2.32% CER on Tigrinya with a CRNN trained on over one million synthetic samples. Neither study explored Transformer-based approaches. We address this gap through three contributions: 1. First TrOCR adaptation for the Ge'ez script. We extend the model’s tokenizer and embedding layers to cover 230 Tigrinya characters, enabling character-level recognition of the full syllabary. 2. Word-Aware Loss Weighting. We identify and resolve a systematic failure mode caused by BPE space-marker conventions that block learning at word boundaries when new script tokens are added. This technique is applicable to any cross-script BPE adaptation. 3. Public benchmark on consumer hardware. Training completes in under three hours on a single 8 GB GPU, and all resources are publicly released. 2 Related Work 2.1 Transformer-Based OCR TrOCR [ 2 ] frames text recognition as an image-to-sequence task, combining a ViT encoder with an autoregressive Transformer decoder. Pre-trained on large image and text corpora, TrOCR achieves state-of-the-art results on major printed and handwritten benchmarks for Latin scripts. The architecture’s reliance on pre-trained components makes it particularly amenable to transfer learning, as both components arrive with prior knowledge of visual features and language structure. Alternative architectures include decoder-only approaches such as DTrOCR [ 9 ] , which have shown competitive performance on English and Chinese benchmarks. Hybrid Swin-Transformer encoders have also been explored for non-Latin scripts [ 4 , 3 ] . 2.2 Ge'ez Script OCR Research on Ge'ez script recognition has focused primarily on Amharic. Belay et al. [ 10 ] proposed a factored CNN that predicts consonant and vowel components separately, exploiting the script’s abugida structure. Belay et al. [ 11 ] subsequently generated a large-scale synthetic dataset of Amharic text-line images using font-based rendering and degradation techniques. The same group later introduced end-to-end sequence models, progressing from LSTM-CTC [ 11 ] to a blended Attention-CTC architecture achieving 0.93% CER on the ADOCR synthetic dataset [ 7 ] . For Tigrinya specifically, Hailu et al. [ 8 ] designed an end-to-end CRNN trained on over one million synthetic text-line images, reporting 2.32% CER without post-processing. This represents the most direct prior work, though it uses a pre-Transformer architecture and a substantially larger training set than ours. The GLOCR dataset [ 12 ] provides multiple Tigrinya text-line corpora including a news subset of 230,000 synthetic samples generated from newspaper text. To the best of our knowledge, GLOCR has not previously been used for Transformer-based OCR benchmarking. 2.3 Cross-Script Transfer Learning for OCR Several recent studies demonstrate successful Transformer OCR adaptation to non-Latin scripts. Murugesh et al. [ 4 ] replace TrOCR’s ViT encoder with a Swin Transformer for Tamil handwriting recognition, achieving 5.44% CER. Cheema et al. [ 3 ] combine a Swin backbone with an mBART-50 decoder for bilingual Urdu–English OCR, reaching 1.1% CER. Chung and Choi [ 5 ] fine-tune vision-language models on 60,000 synthetic Manchu word images, maintaining 93.1% word accuracy on real handwritten documents. Lauar and Laurent [ 6 ] show that fine-tuning the complete English TrOCR on Spanish outperforms systems that replace only the decoder with a language-specific model, suggesting that cross-attention alignment between encoder and decoder carries valuable structural knowledge. 2.4 Tokenizer Adaptation for New Scripts When pre-trained language models encounter scripts absent from their training vocabulary, characters are fragmented into byte-level sequences or mapped to unknown tokens. Pfeiffer et al. [ 13 ] demonstrate that extending the vocabulary with script-specific tokens is necessary for meaningful adaptation. Ogueji et al. [ 14 ] show with AfriBERTa that smaller models focused on African languages can outperform larger multilingual models on downstream tasks, supporting the case for targeted adaptation over reliance on general-purpose multilingual coverage. 3 Methodology 3.1 Dataset We use a 20,000-sample subset from the GLOCR news text-lines corpus [ 12 ] 1 1 1 Dataset available at https://github.com/fgaim/GLOCR , which contains 230,000 synthetic line images generated from Haddas Ertra newspaper text. Each sample pairs a grayscale text-line image with its ground-truth transcription. From the original 200,000 training samples in the News subset, we selected the first 10,000 for training, 5,000 samples from the 15,000 validation samples for validation, and 5,000 samples from the 15,000 test samples for evaluation. Preliminary analysis reveals a compact and uniform dataset with a mean transcription length of 15.8 characters (standard deviation 1.75), a maximum of 21, and a minimum of 4 characters. Images are single-channel grayscale PNGs with standardised height. Using 20,000 samples rather than the full 230,000 keeps the experiment manageable while still testing how far transfer learning carries the model under data scarcity. Table 1 summarises the dataset structure. Table 1 : Dataset structure and partitioning (subset of GLOCR news corpus). Split Samples Mean Len Min Max Train 10,000 15.85 4 21 Validation 5,000 15.88 4 18 Test 5,000 15.89 4 18 Total 20,000 15.8 4 21 3.2 Model Architecture We use Microsoft/TrOCR-base-handwritten [ 2 ] as the primary