Multi-task Code LLMs: Data Mix or Model Merge? Conference: ; ; CCS: Computing methodologies Modeling methodologies
Mingzhi Zhu, Boris Sobolev, Rahul Krishna, Raju Pavuluri, Stacy Patterson, Michele Merler · 2026
Read Multi-task Code LLMs: Data Mix or Model Merge? Conference: ; ; CCS: Computing methodologies Modeling methodologies by Mingzhi Zhu, Boris Sobolev, Rahul Krishna, Raju Pavuluri, Stacy Patterson, Michele Merler (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-21115.txt.
Description
Mingzhi Zhu and five co-authors compare two recipes for building small, versatile code models. Their 2026 paper concerns multi-task code LLMs. Recent work favors deploying smaller, specialized code models in agentic frameworks beside frontier models. That sparks interest in efficient multi-task learning. Such learning must balance performance, constraints, and costs. The authors compare data mixing against model merging.
In data mixing, tasks train together on a blended dataset. In model merging, separately trained models are fused into one. Each route has fans and trade-offs. The paper tests which serves small multi-task code models better. The answer guides teams that cannot afford a giant model per task. It is a practical study for the age of agent teams.
Catalog record
| CREATOR | Mingzhi Zhu, Boris Sobolev, Rahul Krishna, Raju Pavuluri, Stacy Patterson, Michele Merler |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 65 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-21115