Krutrim LLM: Multilingual Foundational Model for over a Billion People
Aditya Kallappa, Palash Kamble, Abhinav Ravi, Akshat Patidar, Vinayak Dhruv · 2025
Read Krutrim LLM: Multilingual Foundational Model for over a Billion People by Aditya Kallappa, Palash Kamble, Abhinav Ravi, Akshat Patidar, Vinayak Dhruv (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-09642.txt.
Introduction
Most large language models learn mainly from English, which leaves India's many languages poorly served. Aditya Kallappa and co-authors introduce Krutrim in this 2025 paper, a language model trained on two trillion tokens with the largest known collection of Indian-language text.
The model matches or beats leading rivals on Indian-language benchmarks while staying strong in English, even with far less training compute than some competitors. It also links to live search so its chat answers stay factual. The team frames the work as a step toward fairer AI for the billion-plus people whose languages big models usually neglect.
Catalog record
| CREATOR | Aditya Kallappa, Palash Kamble, Abhinav Ravi, Akshat Patidar, Vinayak Dhruv |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-09642
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Krutrim LLM: Multilingual Foundational Model for over a Billion People Thanks: Additional Contributors: Sanket Shah, Sulabh Katiyar, Sindhu Pawar, Soham Pendurkar, Pranav Raveendran, Bidyapathi Ray, Daud Ibrahim, Divyansh Rajput, Pidathala Sowjanya, Rahul Kumar, Rishabh Nahata, Pranav Raveendran, Bidyapathi Ray, Prateek Shrivastava, Yogendra Mishra, Azhagiri S, Priyanka Nayak, Sandesh Bafna, Aniruddha Uttam Tammewar, Vivek Dahiya, Ali Faraz, Ayush Tarun, Shaharukh Khan, Shubham Kakde, Nishant kumar, Debanjana Biswas, Ashish Anand Kulkarni, Rajkiran Panuganti, Hareesh Kumar Acknowledgement: Ravi Jain, Bhavish Aggarwal Aditya Kallappa, Palash Kamble, Abhinav Ravi, Akshat Patidar, Vinayak Dhruv Abstract India is one of the most vibrant and culturally diverse societies. Developing a general-purpose artificial intelligence system tailored for the Indian market presents unique challenges. These include accounting for the nation’s cultural nuances, accommodating its linguistic diversity with numerous regional languages, adapting to the prominence of oral traditions, ensuring accessibility to relevant data sets, and achieving scalability to serve the vast population effectively. Careful consideration and innovative approaches are necessary to navigate these complexities successfully. Existing foundation models for natural language tasks are predominantly trained on English data, limiting their effectiveness for languages native to India’s over 1 billion citizens. Thousands of regional languages, dialects, language or code mixing pose representation challenges, exacerbated by sparse training data; Indic languages comprise just 1% of Common Crawl corpora despite India representing 18% of the global population. Consequently, lack of Indic language relevance and context representation leads current models to exhibit cultural and linguistic biases oriented towards Western contexts. We present Krutrim Large Language Model (LLM), a 2 trillion token multilingual foundation model designed to serve Indian demographic needs through equitable representation of the country’s array of native tongues. Training data incorporates the largest known Indic language dataset, mitigating associated data scarcity obstacles that encumber model parity across dialects. Evaluations demonstrate Krutrim’s strong performance on Indic language benchmarks, surpassing or at par with state-of-the-art models despite being significantly smaller in training flops. Krutrim LLM also matches or exceeds standards set on English benchmarks by models trained on comparable flops (e.g. vs LLAMA-2 on 10 out of 16 tasks with average score of 0.57 vs 0.55 of LLAMA-2), evidencing flexible multilingual fluency. We further integrated search to deliver real-time and more factually relevant information via Krutrim LLM conversational app, working to make next-generation AI widely accessible for a diverse set of over 1 billion worldwide users. Through intentional design choices that redress endemic data imbalances, Krutrim LLM signifies meaningful progress in the pursuit of ethical, globally representative AI foundation models. 1 Introduction 1.1 Challenges in Developing an India-Centric Large Language Model (LLM) Building Artificial General Intelligence (AGI) and AI models to cater to the diverse cultural context of India presents multifaceted challenges. The country’s linguistic diversity [ 43 ] , with hundreds of languages and dialects, poses a significant hurdle [ 30 ] . These languages span 4 major language families with Indo-Aryan and Dravidian being the most prominent ones [ 23 ] . Many of these languages thrive in oral traditions, leading to frequent mixing and evolving linguistic patterns that are not well-documented digitally. This oral culture complicates the collection and digitization of data necessary for training robust AI systems. Additionally, India’s rich tapestry of social and economic backgrounds adds another layer of complexity. The vast differences in socio-economic statuses influence digital access and technology usage, making it challenging to develop AI solutions that are inclusive and equitable. Furthermore, cultural nuances, which vary widely across regions, need to be understood and respected in AI applications to ensure they are relevant and sensitive to users’ needs. The combination of linguistic diversity, cultural richness, and socio-economic disparities requires innovative approaches to data collection, model training, and the development of algorithms that can adapt to and reflect India’s multifaceted society. Most recent AI models, like, LLaMA [ 41 ] , GPT-3.5, and others, face significant limitations when representing the Indian ethos and languages due to the unique linguistic and cultural challenges prevalent in the region [ 11 ] . In fact, the structure of some Indian languages, such as Sanskrit, which allows for the creation of virtually infinite compound words, presents a stark contrast to English’s finite vocabulary. These languages are not only syntactically complex but are also semantically rich, making them difficult to model with the tokenizers designed primarily for Western languages. Such systems often result in inefficient processing of Indian languages, leading to excessively long sequences that can hamper the effectiveness of AI models. Additionally, the reliance on digitized data for training these models poses a challenge, as there is a significant lack of local, undigitized data reflecting the vast cultural and linguistic diversity of India. This scarcity of data is compounded by the fact that Indian languages are under-represented in major digital corpora. For instance, Indian languages account for a mere 1% of the Common Crawl 1 1 1 http://commoncrawl.org/ dataset [ 12 , 31 ] , despite India constituting 18% of the global population. This under-representation leads to biased models that do not adequately capture the nuances of Indian languages and culture. The poor representation of Indic data not only affects the accuracy and relevance of these AI models for Indian users but also highlights a broader issue of inclusivity and representation in the development of global AI models. 1.2 Contributions The development of Krutrim LLM, India’s first and premier Large Language Model (LLM), represents a monumental stride towards creating AI technologies that are deeply aligned with the country’s linguistic and cultural diversity. The project embarked on an ambitious journey by curating an extensive range of Indic data from across the web, encompassing the myriad languages spoken throughout India. This foundational work enabled the training of the LLM on over 2 trillion tokens, a scale unprecedented in the context of Indian language processing. Recognizing the unique challenges of Indian languages, the team developed a specialized Indic tokenizer, tailored to efficiently process the complex morphologies and syntaxes inherent to these languages. To enhance the model’s performance further, Krutrim LLM incorporates state-of-the-art attention mechanisms such as Grouped Query Attention (GQA) [ 6 ] and AliBi [ 35 ] , which significantly improve its ability to handle longer contexts and provide faster responses, thereby elevating the user experience. This advanced technical framework allows Krutrim LLM to outperform several open-source models of comparable size or computational requirements, particularly in its handling of Indic languages, without any compromise. Furthermore, the model underwent India-centric fine-tuning, covering a broad spectrum of topics and tasks relevant to the Indian context. This fine-tuning ensured that the model could effectively engage with and address the specific needs and nuances of Indian users. A linguistically and socially rich alignment was achieved through Direct Preference Optimization (DPO) [ 36 ] , ensuring that Krutrim LLM adheres closely to the values and preferences of its intended audience. This meticulous approach to development reflects a deep commitment to inclusivity and representation, setting a new standard for AI models designed to serve the rich tapestry of India’s languages and cultures. Lastly, to make Krutrim LLM more functional for general purpose use cases and to make it factually accurate, we integrated web search. Krutrim LLM is accessible for general audience via our conversational app: https://chat.olakrutrim.com/ 2 Related Work and