Use of Metric Learning for the Recognition of Handwritten Digits, and its Application to Increase the Outreach of Voice-based Communication Platforms Conference: ACM SIGCAS/SIGCHI Conference on Computing and Sustainable Societies; June 29–July 1, 2022; Seattle, WA, USA ACM SIGCAS/SIGCHI Conference o
Devesh Pant, Dibyendu Talukder, Deepak Kumar, Rachit Pandey, Aaditeshwar Seth, Chetan Arora · 2025
Read Use of Metric Learning for the Recognition of Handwritten Digits, and its Application to Increase the Outreach of Voice-based Communication Platforms Conference: ACM SIGCAS/SIGCHI Conference on Computing and Sustainable Societies; June 29–July 1, 2022; Seattle, WA, USA ACM SIGCAS/SIGCHI Conference o by Devesh Pant, Dibyendu Talukder, Deepak Kumar, Rachit Pandey, Aaditeshwar Seth, Chetan Arora (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-18948.txt.
Introduction
Devesh Pant and co-authors build tools that read handwritten digits on paper forms, aimed at development programs in rural India. Field workers for such programs often lack smartphones or training for digital data entry, so paper forms remain common. Reading those forms by machine requires digit recognizers that work under messy real-world conditions.
The team releases a large dataset of handwritten digits and recognition models trained with metric learning, a method that teaches the model which digits look alike. The tools were deployed in a health and nutrition project that phones rural women's self-help groups with recorded advice. Phone numbers collected on paper were digitized with the system, enabling almost four million calls. Data, models, and code were released openly for others to reuse.
Catalog record
| CREATOR | Devesh Pant, Dibyendu Talukder, Deepak Kumar, Rachit Pandey, Aaditeshwar Seth, Chetan Arora |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-18948
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Use of Metric Learning for the Recognition of Handwritten Digits, and its Application to Increase the Outreach of Voice-based Communication Platforms Conference: ACM SIGCAS/SIGCHI Conference on Computing and Sustainable Societies; June 29–July 1, 2022; Seattle, WA, USA ACM SIGCAS/SIGCHI Conference on Computing and Sustainable Societies (COMPASS ’22), June 29–July 1, 2022, Seattle, WA, USA Price: 15.00 DOI: 10.1145/3530190.3534795 ISBN: 978-1-4503-9347-8/22/06 Devesh Pant, Dibyendu Talukder, Deepak Kumar, Rachit Pandey, Aaditeshwar Seth, Chetan Arora Abstract . Initiation, monitoring, and evaluation of development programmes can involve field-based data collection about project activities. This data collection through digital devices may not always be feasible though, for reasons such as unaffordability of smartphones and tablets by field-based cadre, or shortfalls in their training and capacity building. Paper-based data collection has been argued to be more appropriate in several contexts, with automated digitization of the paper forms through OCR (Optical Character Recognition) and OMR (Optical Mark Recognition) techniques. We contribute with providing a large dataset of handwritten digits, and deep learning based models and methods built using this data, that are effective in real-world environments. We demonstrate the deployment of these tools in the context of a maternal and child health and nutrition awareness project, which uses IVR (Interactive Voice Response) systems to provide awareness information to rural women SHG (Self Help Group) members in north India. Paper forms were used to collect phone numbers of the SHG members at scale, which were digitized using the OCR tools developed by us, and used to push almost 4 million phone calls. The data, model, and code have been released in the open-source domain. 1. Introduction In the context of running large development programmes such as SHGs (Self Help Groups) networks, agriculture extension, employment generating activities, and immunization campaigns, among others, it is advisable to maintain digitized records of the programme activities ( Dehnavieh et al. 2019 ; Patil et al. 2020 ) . Digitization allows for ongoing monitoring and learning to understand implementation efficacy, such as the achieved outreach, deviations from planned activities, and other output and outcome indicators. A common strategy is for the programme field teams to use tablets and smartphones to keep a record of their work ( Patil et al. 2020 ; Saha et al. 2020 ) . However, many of these programmes are implemented at the last mile by community-based cadre who may not be very technology savvy, or logistic challenges may arise in organizing training workshops and refresher courses for them, or providing Internet-enabled devices and network packs to the entire field cadre may be expensive ( Avula et al. 2020 ; Sahay et al. 2009 ; Singh et al. 2021 ) . As a consequence, the use of traditional paper-based methods has been hard to displace. In fact, in practice, record management for many programmes is done by field workers first on paper, and then retrospectively digitized through data entry into digital devices ( Meghani et al. 2021 ; Gudi et al. 2020 ) . In our study, we embrace this paradigm of using paper forms as an appropriate method for record keeping by field teams, and demonstrate through a field project of how the benefits of digitization can still be achieved by automatically digitizing paper forms through handwriting recognition using deep learning techniques. The relevance of using paper forms has been documented in several contexts. The Shreddr system piloted in Mali operated on digital scanned copies of paper forms of surveys about citizen perceptions of governance and accountability of Malian elected officials. It cropped out portions of hand-written text on the forms and passed these ROIs (Regions of Interest) to crowd-sourced platform workers for annotation, while also randomizing the allocation among different workers to preserve privacy ( Chen et al. 2012a ) . This system evolved into a social enterprise, Captricity , which now operates in several countries. Another system, mScan , was piloted in Mozambique and used OMR (Optimal Mark Recognition) to read data off photographs of custom designed paper forms containing multiple-choice questions ( Dell et al. 2012 ) . The social enterprise, Health-E-Net , uses the same concept at scale in Africa, with an innovative method to also mass-produce custom forms by rubber-stamping entire forms on blank paper. We draw our motivation from such prior work and contemporary practices, to improve the state of the art in usage contexts where paper-based entry by field workers is easier or more cost effective than to use electronic devices. We go beyond OMR to also recognize hand-written numeric digits. We do not attempt a complete OCR (Optical Character Recognition) solution for alphabets and numerals written in free-text, but only provide a limited solution that allows form designers to build forms with demarcated spaces for single digits, and a dataset and model for recognizing these handwritten digits. Our dataset, consisting of 178,334 digits, was collected through a field project in which paper forms were used to build a phone number database for a large government-run women SHG (Self Help Group) network in the state of Bihar in India. This phone number database was used to send periodic voice messages through an IVR (Interactive Voice Response) system to the SHG members about health and nutrition awareness, including for COVID-19, and other announcements relevant to the SHG programme. We show that this real-world dataset has several nuances because of which models trained on the standard MNIST dataset and its variants do not work well. We also show that commercial OCR APIs by cloud providers like Microsoft Azure and Google Cloud do not perform well on handwritten text. We therefore create our own model trained on our dataset. This model is able to provide a digit-level accuracy of 99.40% and a phone number (consisting of ten digits) level accuracy of 94.66% using a metric learning technique with triplet loss ( Schroff et al. 2015 ) . Our trained model is also able to obtain an accuracy of 99.50% on the MNIST data, validating the generalizability of our model to work on standard datasets in this space. The dataset and model contributed by us can be deployed in similar environments where one-time or ongoing paper-based data collection and record keeping can be done, and digitized in bulk to monitor project activities and milestones. Our system is now deployed as part of the voice-based communication platform for SHG members in Bihar, to promote health and nutrition awareness ( Chakraborty et al. 2019 ; Seth et al. 2020 ) . An SHG consists of 10-12 women, a group of 10-12 SHGs in a village is supported by a community mobilizer, and a group of 30-35 community mobilizers in a village cluster are managed by a cluster head. The cluster heads are informed by our social enterprise partner, Gram Vaani, on how to fill out the paper forms. They in turn inform the community mobilizers and distribute the forms to them. Each community mobilizer then fills a separate form, typically one for each SHG, with the phone numbers of the SHG members. These filled forms are then collected by the cluster heads and eventually passed on to the Gram Vaani team, who scans them in bulk, and runs the scanned forms through the OCR system. Figure 1.