Vedavani: A Benchmark Corpus for ASR on Vedic Sanskrit Poetry

Sujeet Kumar 1,2 , Pretam Ray 1 , Abhinay Beerukuri 1 , Shrey Kamoji 1 , \AND Manoj Balaji Jagadeeshan 1 , Pawan Goyal 1 1 Indian Institute of Technology, Kharagpur 2 B P Mandal College of Engineering, Madhepura {ksujeet.cs, pretamroy309, beerukuriabhinay, shreykamoji, manojbalaji1}@gmail.com pawang · 2025

Read Vedavani: A Benchmark Corpus for ASR on Vedic Sanskrit Poetry by Sujeet Kumar 1,2 , Pretam Ray 1 , Abhinay Beerukuri 1 , Shrey Kamoji 1 , \AND Manoj Balaji Jagadeeshan 1 , Pawan Goyal 1 1 Indian Institute of Technology, Kharagpur 2 B P Mandal College of Engineering, Madhepura {ksujeet.cs, pretamroy309, beerukuriabhinay, shreykamoji, manojbalaji1}@gmail.com pawang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-00145.txt.

Introduction

Sanskrit poetry resists speech recognition: its sounds blend at word seams and its verses follow strict rhythm. Sujeet Kumar and co-authors build the first test collection for the task in a 2025 paper. Their corpus holds fifty-four hours of chanting from two ancient Vedic texts, split into nearly thirty-one thousand labeled clips.

The recordings keep the pitch and beat patterns that define the verse form. The team then tried the leading multilingual speech models on the collection to set a baseline. A model tuned for Indian languages came out on top. The corpus gives builders a shared yardstick for a hard, neglected language.

Catalog record

CREATORSujeet Kumar 1,2 , Pretam Ray 1 , Abhinay Beerukuri 1 , Shrey Kamoji 1 , \AND Manoj Balaji Jagadeeshan 1 , Pawan Goyal 1 1 Indian Institute of Technology, Kharagpur 2 B P Mandal College of Engineering, Madhepura {ksujeet.cs, pretamroy309, beerukuriabhinay, shreykamoji, manojbalaji1}@gmail.com pawang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-00145

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Vedavani: A Benchmark Corpus for ASR on Vedic Sanskrit Poetry Sujeet Kumar 1,2 , Pretam Ray 1 , Abhinay Beerukuri 1 , Shrey Kamoji 1 , \AND Manoj Balaji Jagadeeshan 1 , Pawan Goyal 1 1 Indian Institute of Technology, Kharagpur 2 B P Mandal College of Engineering, Madhepura {ksujeet.cs, pretamroy309, beerukuriabhinay, shreykamoji, manojbalaji1}@gmail.com [email protected] Abstract Sanskrit, an ancient language with a rich linguistic heritage, presents unique challenges for automatic speech recognition (ASR) due to its phonemic complexity and the phonetic transformations that occur at word junctures, similar to the connected speech found in natural conversations. Due to these complexities, there has been limited exploration of ASR in Sanskrit, particularly in the context of its poetic verses, which are characterized by intricate prosodic and rhythmic patterns. This gap in research raises the question: How can we develop an effective ASR system for Sanskrit, particularly one that captures the nuanced features of its poetic form? In this study, we introduce Vedavani, the first comprehensive ASR study focused on Sanskrit Vedic poetry. We present a 54-hour Sanskrit ASR dataset, consisting of 30,779 labelled audio samples from the Rig Veda and Atharva Veda. This dataset captures the precise prosodic and rhythmic features that define the language. We also benchmark the dataset on various state-of-the-art multilingual speech models. 1 1 1 Our dataset and code are publicly available at https://github.com/SujeetNlp/Vedavani Experimentation revealed that IndicWhisper performed the best among the SOTA models. [Script=Devanagari] \setmainlanguage english \setotherlanguages marathi,hindi,sanskrit 1 Introduction Sanskrit, an ancient and highly inflected language, holds great importance in preserving the knowledge of archaic India. Sanskrit is a language with fairly advanced disciplines of phonetics (Śikṣā), prosody (Chandas), and grammar (Vyākarana). The language has a rich oral tradition, and it tends to follow a phonemic orthography, resulting in systematic grapheme-phoneme correspondences. Connected speech leads to phonemic transformations in utterances, and in Sanskrit, this is faithfully preserved in writing as well [ krishna2018freefreewordorder ] . Recent advancements in Sanskrit automatic speech recognition (ASR) include the introduction of a corpus comprising 1,360 sentences [ anoop2019automatic ] . This was followed by the development of an end-to-end ASR system utilizing connectionist temporal classification (CTC), which demonstrated promising results with 5.5 hours of speech data [ CTC-sanskrit ] . Subsequently, the 78-hour Vakysancayah dataset was used to investigate various methods in acoustic modeling and language processing [ asr-iitb-data ] . More recent contributions to the field include the Shrutilipi corpus with 27 hours of data [ javed2023indicsuperb ] and the Kathbath dataset from the IndicSUPERB benchmark, which includes 102 hours of audio [ bhogale2023effectiveness ] . Table 1: Zero-shot inference results of existing Sanskrit ASR models on the poetry test dataset Models WER CER SPG-INXS-MMS 99.84 39.38 SPG-INXS-CCC-W2V 99.84 39.39 SPG-INXS-W2V 103.20 40.05 IndicWhisper 109.80 46.05 However, despite these advances, there is still a significant gap. Existing datasets predominantly focus on prose, limiting their ability to capture the full linguistic diversity of Sanskrit. Much of the pre-classical and classical literature of the language is composed in verse, where the ordering of words is often dictated by metrical constraints rather than syntactic rules [ 10.1162/coli_a_00390 ] [ Wright_1968 ] . This arbitrarily ordered word poses a unique challenge for ASR systems. To investigate whether current Sanskrit ASR models are capable of capturing the nuances of poetry, we evaluated the performance of four Sanskrit ASR models: SPRING-INX-MMS, 2 2 2 https://asr.iitm.ac.in/models SPRING-INX-wav2vec2, 2 and SPRING-INX-ccc-wav2vec2, 2 along with IndicWhisper [ whisper ] . The models depicted in Table 1 exhibite a poor performance, with Word Error Rates (WER) ranging from 99 to 110 and Character Error Rates (CER) from 39 to 46, underscoring the need for specialized datasets. We, thus, introduce a new dataset, Vedavani , devoted to Vedic Sanskrit poetry. This collection consists of more than 54 hours of audio recordings totaling 30,779 sentences from the Atharva Veda (2000-1500 BCE) and Rig Veda (pre-2000 BCE) [ lal-rigvedic ] . Vedavani presents a special chance to learn the grammatical and stylistic subtleties of classical Sanskrit poetry since it catches the melodic rhythms, complex meters, and lyrical beauty of Vedic hymns and poems. 2 Data Collection We curated textual resources for the Atharva Veda and Rig Veda from Wikipedia. Additionally, we sourced audio transcriptions of Vedic recitations from the Internet Archive, selecting recordings that accurately represent the traditional chanting style. We manually aligned the text and audio modalities with meticulous attention to create a well-structured dataset. This process involved segmenting the audio recordings into meaningful units corresponding to verses or phrases in the textual transcripts. Each audio segment was carefully mapped to its respective textual counterpart, ensuring synchronization between spoken and written recitations. The following sections provide a detailed explanation of our data alignment methodology, the challenges encountered, and the approaches used to ensure high-quality synchronization of textual and audio data. 2.1 Textual Data Extraction We obtained textual content for the 20 kandas 3 3 3 Kandas and Mandalas refer to chapters in Atharva Veda and Rig Veda respectively of the Atharva Veda 4 4 4 \s https://sa.wikisource.org/wiki/अथर्ववेदः and 10 mandalas 3 of the Rig Veda 5 5 5 \s https://sa.wikisource.org/wiki/ऋग्वेदः from Wikipedia. The extraction process involved navigating the HTML structure to isolate and capture relevant text while excluding non-essential elements such as numbers, captions under images, etc. We ensured the text was free of special characters and standardized formatting for consistency, preserving spaces between lines and punctuation marks to maintain readability. This categorization facilitated easy retrieval of specific sections and optimized the dataset for further analysis and study. 2.2 Audio Data Extraction The audio recordings of the Atharva Veda 6 6 6 Data collected from the organization *Veda Prasara Samithi*, available at https://archive.org/details/atharvaveda_202107 and Rig Veda 7 7 7 Data collected from the organization *Veda Prasara Samithi*, available at https://archive.org/details/RigvedaChanting were obtained from the Internet Archive. Each recording is about 40–45 minutes long, totaling 27 files for the Atharva Veda and 54 files for the Rig Veda, with a combined size of 5.4GB. We converted these files to wav format and segmented them using pydub 8 8 8 https://github.com/jiaaro/pydub library. 2.3 Textual-Audio Data Alignment Upon segmenting the audio files into smaller chunks and obtaining corresponding text files, our expectation was for the audio files to align line by line with the Mandalas and Kandas of the Vedas, but we observed inconsistencies in the segmentation process, with audio chunks sometimes spanning multiple lines or only capturing part of a line from the Vedic texts. To resolve these issues, we adjusted the segmentation parameters, such as the minimum length of silence and the silence threshold. Specifically, the minimum length of silence determines the duration of silence (in milliseconds) required to trigger a split in the audio file, while the silence threshold specifies the decibel level below which audio is considered silent. We set the minimum length of silence in the range of 3-7 milliseconds and the silence threshold parameter to be in the range of -30 to -70 dB. Even after trying, all these inconsistencies still persisted. Hence, the decision was taken to manually align each of the chunks with its corresponding transcript. During the manual alignment process, we added 500 ms of silence at the beginning and end of each audio

Read the complete text