Multi-task Code LLMs: Data Mix or Model Merge? Conference: ; ; CCS: Computing methodologies Modeling methodologies

Mingzhi Zhu, Boris Sobolev, Rahul Krishna, Raju Pavuluri, Stacy Patterson, Michele Merler · 2026

Read Multi-task Code LLMs: Data Mix or Model Merge? Conference: ; ; CCS: Computing methodologies Modeling methodologies by Mingzhi Zhu, Boris Sobolev, Rahul Krishna, Raju Pavuluri, Stacy Patterson, Michele Merler (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-21115.txt.

Description

Mingzhi Zhu and five co-authors compare two recipes for building small, versatile code models. Their 2026 paper concerns multi-task code LLMs. Recent work favors deploying smaller, specialized code models in agentic frameworks beside frontier models. That sparks interest in efficient multi-task learning. Such learning must balance performance, constraints, and costs. The authors compare data mixing against model merging.

In data mixing, tasks train together on a blended dataset. In model merging, separately trained models are fused into one. Each route has fans and trade-offs. The paper tests which serves small multi-task code models better. The answer guides teams that cannot afford a giant model per task. It is a practical study for the age of agent teams.

Catalog record

CREATORMingzhi Zhu, Boris Sobolev, Rahul Krishna, Raju Pavuluri, Stacy Patterson, Michele Merler
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE65 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-21115

Browse the full library · About the Tower of Babel