BhashaBench V1 : A Comprehensive Benchmark for the Quadrant of Indic Domains
Vijay Devane, Mohd Nauman, Bhargav Patel, Aniket Wakchoure, Yogeshkumar Sant, Shyam Pawar, Viraj Thakur, Ananya Godse, Sunil Patra, Neha Maurya, Suraj Racha, Nitish Singh, Ajay Nagpal, Piyush Sawarkar, Kundeshwar Pundalik, Rohit Saluja, Ganesh Ramakrishnan · 2025
Read BhashaBench V1 : A Comprehensive Benchmark for the Quadrant of Indic Domains by Vijay Devane, Mohd Nauman, Bhargav Patel, Aniket Wakchoure, Yogeshkumar Sant, Shyam Pawar, Viraj Thakur, Ananya Godse, Sunil Patra, Neha Maurya, Suraj Racha, Nitish Singh, Ajay Nagpal, Piyush Sawarkar, Kundeshwar Pundalik, Rohit Saluja, Ganesh Ramakrishnan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-25409.txt.
Description
Vijay Devane led sixteen co-authors in building BhashaBench V1. Mohd Nauman, Bhargav Patel, and Aniket Mahendra Wakchoure helped. Yogeshkumar Sant, Shyam Pawar, and Viraj Thakur followed. Ananya Godse, Sunil Patra, and Neha Maurya joined. Suraj Racha and others followed, with Ganesh Ramakrishnan completing seventeen authors.
Their 2025 benchmark evaluates language models on Indic domains. Existing benchmarks are largely Anglocentric and domain-agnostic. BhashaBench contains over seventy-four thousand question-answer pairs. Questions come from government and domain-specific exams. English pairs number about fifty-two thousand in total. Hindi pairs number about twenty-two thousand there. Four domains are covered: agriculture, legal, finance, Ayurveda. Over ninety subdomains and five hundred topics appear. They evaluated more than twenty-nine language models. Large domain and language performance gaps emerged. English content consistently outperformed Hindi content overall. Weak areas included Panchakarma, seed science, and rights.
Catalog record
| CREATOR | Vijay Devane, Mohd Nauman, Bhargav Patel, Aniket Wakchoure, Yogeshkumar Sant, Shyam Pawar, Viraj Thakur, Ananya Godse, Sunil Patra, Neha Maurya, Suraj Racha, Nitish Singh, Ajay Nagpal, Piyush Sawarkar, Kundeshwar Pundalik, Rohit Saluja, Ganesh Ramakrishnan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 117 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-25409