HELM: Hyperbolic Large Language Models via Mixture-of-Curvature Experts

Neil He ⋆ Rishabh Anand ⋆ Hiren Madhu Ali Maatouk, ⋆ Equal Contribution · 2025

Read HELM: Hyperbolic Large Language Models via Mixture-of-Curvature Experts by Neil He ⋆ Rishabh Anand ⋆ Hiren Madhu Ali Maatouk, ⋆ Equal Contribution (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-24722.txt.

Description

Neil He and co-authors rebuild language models inside curved space in a 2025 paper. Language is hierarchical, but standard models compute in flat Euclidean space. That mismatch, recent work argues, breeds training instability and weaker generation.

Their family, HELM, runs the whole transformer in hyperbolic space, a geometry with room to spare for tree-like structure. Every core operation gets a hyperbolic twin, from position coding to normalization. A second model splits the work among experts, each living in a space of different curvature. A leaner attention scheme cuts memory for long texts. The authors claim the first fully hyperbolic models trained at real scale.

Catalog record

CREATORNeil He ⋆ Rishabh Anand ⋆ Hiren Madhu Ali Maatouk, ⋆ Equal Contribution
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE85 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-24722

Browse the full library · About the Tower of Babel