Adapting Multilingual Models to Code-Mixed Tasks via Model Merging Conference: 8th ACM IKDD CODS and 30th COMAD; December 17–20, 2025; Pune, India 8th ACM IKDD CODS and 30th COMAD (CODS-COMAD ’25), December 17–20, 2025, Pune, India CCS: Computing methodologies Natural language processing
Prashant Kodali, Vaishnavi Shivkumar, Swarang Joshi, Monojit Choudhury, Ponnurangam Kumaraguru, Manish Shrivastava · 2025
Read Adapting Multilingual Models to Code-Mixed Tasks via Model Merging Conference: 8th ACM IKDD CODS and 30th COMAD; December 17–20, 2025; Pune, India 8th ACM IKDD CODS and 30th COMAD (CODS-COMAD ’25), December 17–20, 2025, Pune, India CCS: Computing methodologies Natural language processing by Prashant Kodali, Vaishnavi Shivkumar, Swarang Joshi, Monojit Choudhury, Ponnurangam Kumaraguru, Manish Shrivastava (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-19782.txt.
Description
Prashant Kodali and five co-authors adapt language models to code-mixed text, where speakers blend two languages in one sentence, by merging models instead of only fine-tuning them. Their recipe first continues pretraining a multilingual base model on unlabeled code-mixed text, then merges that checkpoint back with the base, and finally fine-tunes on the task. Tests cover sentiment and hate speech classification for English-Hindi and English-Spanish.
Merged models beat both plain fine-tuning and the continue-then-fine-tune pipeline, gaining several F1 points, while prompting much larger models lagged behind. Models adapted on one language pair also transferred better to new pairs, such as English-Tamil, than English-only baselines. The authors close with practical recipes for different data situations.
Catalog record
| CREATOR | Prashant Kodali, Vaishnavi Shivkumar, Swarang Joshi, Monojit Choudhury, Ponnurangam Kumaraguru, Manish Shrivastava |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-19782