Adapting Multilingual Models to Code-Mixed Tasks via Model Merging Conference: 8th ACM IKDD CODS and 30th COMAD; December 17–20, 2025; Pune, India 8th ACM IKDD CODS and 30th COMAD (CODS-COMAD ’25), December 17–20, 2025, Pune, India CCS: Computing methodologies Natural language processing

Prashant Kodali, Vaishnavi Shivkumar, Swarang Joshi, Monojit Choudhury, Ponnurangam Kumaraguru, Manish Shrivastava · 2025

Read Adapting Multilingual Models to Code-Mixed Tasks via Model Merging Conference: 8th ACM IKDD CODS and 30th COMAD; December 17–20, 2025; Pune, India 8th ACM IKDD CODS and 30th COMAD (CODS-COMAD ’25), December 17–20, 2025, Pune, India CCS: Computing methodologies Natural language processing by Prashant Kodali, Vaishnavi Shivkumar, Swarang Joshi, Monojit Choudhury, Ponnurangam Kumaraguru, Manish Shrivastava (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-19782.txt.

Description

Prashant Kodali and five co-authors adapt language models to code-mixed text, where speakers blend two languages in one sentence, by merging models instead of only fine-tuning them. Their recipe first continues pretraining a multilingual base model on unlabeled code-mixed text, then merges that checkpoint back with the base, and finally fine-tunes on the task. Tests cover sentiment and hate speech classification for English-Hindi and English-Spanish.

Merged models beat both plain fine-tuning and the continue-then-fine-tune pipeline, gaining several F1 points, while prompting much larger models lagged behind. Models adapted on one language pair also transferred better to new pairs, such as English-Tamil, than English-only baselines. The authors close with practical recipes for different data situations.

Catalog record

CREATORPrashant Kodali, Vaishnavi Shivkumar, Swarang Joshi, Monojit Choudhury, Ponnurangam Kumaraguru, Manish Shrivastava
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-19782

Browse the full library · About the Tower of Babel