Towards Adaptive Continual Model Merging via Manifold-Aware Expert Evolution

Haiyun Qiu, Xingyu Wu, Kay Chen Tan · 2026

Read Towards Adaptive Continual Model Merging via Manifold-Aware Expert Evolution by Haiyun Qiu, Xingyu Wu, Kay Chen Tan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-22464.txt.

Description

Qiu, Wu and Tan propose MADE-IT for ongoing model merging, joining task models into one design. Past methods face a saturation-redundancy dilemma: backbone methods fill limits, while expert-mixture types expand at random. MADE-IT grounds expert vectors in manifold geometry, using a subspace affinity metric and tuned threshold to guide expert evolution. It also uses a training-free implicit router turning on experts via feature-subspace match. Tests show MADE-IT beats strong rivals in accuracy and toughness across long and shuffled runs, while pruning spare experts.

Catalog record

CREATORHaiyun Qiu, Xingyu Wu, Kay Chen Tan
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE66916
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.22464

Browse the full library · About the Tower of Babel