Hyperbolic Genome Embeddings

Raiyan R. Khan, Philippe Chlenski, Itsik Pe’er · 2025

Read Hyperbolic Genome Embeddings by Raiyan R. Khan, Philippe Chlenski, Itsik Pe’er (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-21648.txt.

Introduction

Raiyan Khan, Philippe Chlenski, and Itsik Pe'er borrow a curved geometry to represent DNA sequences more faithfully. Most models embed sequences in flat space, yet evolution arranges life in branching trees, which curved hyperbolic space fits naturally. Their method uses hyperbolic neural networks to read sequences without needing an explicit family tree.

Across forty-two benchmark sets for genome interpretation, the hyperbolic models beat their flat counterparts on thirty-seven. On seven sets from a standard suite, they topped the best published results, despite using far fewer parameters than large DNA language models and skipping pretraining. The team also built a new benchmark on transposable elements, the jumping DNA segments that make up much of the genome. They add a way to measure how hyperbolic a dataset's embeddings are. The authors argue that matching a model's geometry to biology's tree-like structure is a strong general recipe for genome learning.

Catalog record

CREATORRaiyan R. Khan, Philippe Chlenski, Itsik Pe’er
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE74 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-21648

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Hyperbolic Genome Embeddings Raiyan R. Khan, Philippe Chlenski, Itsik Pe’er Abstract Current approaches to genomic sequence modeling often struggle to align the inductive biases of machine learning models with the evolutionarily-informed structure of biological systems. To this end, we formulate a novel application of hyperbolic CNNs that exploits this structure, enabling more expressive DNA sequence representations. Our strategy circumvents the need for explicit phylogenetic mapping while discerning key properties of sequences pertaining to core functional and regulatory behavior. Across 37 out of 42 genome interpretation benchmark datasets, our hyperbolic models outperform their Euclidean equivalents. Notably, our approach even surpasses state-of-the-art performance on seven GUE benchmark datasets, consistently outperforming many DNA language models while using orders of magnitude fewer parameters and avoiding pretraining. Our results include a novel set of benchmark datasets—the Transposable Elements Benchmark—which explores a major but understudied component of the genome with deep evolutionary significance. We further motivate our work by exploring how our hyperbolic models recognize genomic signal under various data-generating conditions and by constructing an empirical method for interpreting the hyperbolicity of dataset embeddings. Throughout these assessments, we find persistent evidence highlighting the potential of our hyperbolic framework as a robust paradigm for genome representation learning. Our code and benchmark datasets are available at https://github.com/rrkhan/HGE . 1 Introduction Representation learning of genome sequences has enabled the exploration of critical unsolved problems in biology, particularly the understanding of genome function and organization ( Avsec et al. 2021 ; Chen et al. 2022a ; Dudnyk et al. 2024 ) . Many effective approaches used for genomic sequence modeling have arisen from the same machine learning methods that have powered natural language and image embeddings ( Yue et al. 2023 ; Zhou 2022 ; Consens et al. 2023 ) . While the field has made progress by utilizing these methods, the inductive biases of these models are not usually bespoke to genomic data, limiting the expressive power of the resulting sequence representations. Given the tremendous amount of information sequestered within DNA sequences encoding cellular and molecular activity, an efficient and nuanced representation is necessary for genome interpretation and downstream analyses. Genome organization is complex, and much of this complexity is the product of evolutionary processes. Any single genome represents the culmination of information diffusion across generations. However, this information transfer occurs through noisy channels, as background mutation rates may degrade the sequence signal ( Lu et al. 2020 ) . Accounting for phylogenetic relationships may therefore contextualize the content of the genome and ultimately benefit genome interpretation attempts. The shared influence of a common ancestor across all genomes imbues DNA sequence data with underlying hierarchical structure. These hierarchical relationships emerge through a variety of mechanisms, such as orthology and paralogy, which both codify homologous sequences but occur under different circumstances. Further compounding these interdependencies are the multiple overlapping grammatical structures for different regulatory pathways that characterize the language of the genome. Altogether, these nested levels of latent hierarchies confound genome interpretation. In developing a modeling paradigm better suited to handling the hierarchical nature of DNA sequences, considering the geometry of the embedding spaces is essential. While most embeddings are Euclidean by default, non-Euclidean spaces may offer a compelling alternative. Specifically, hyperbolic spaces, which have the representational capacity to capture tree-structured data with high fidelity, are well-equipped to manage the hierarchical patterns ubiquitous in genomic sequences. The negative curvature of hyperbolic spaces facilitates the continuous embedding of exponentially growing structures like phylogenetic trees with relatively low distortion. In this work, we contend that hyperbolic spaces may be appropriate for learning meaningful representations of the genome. We leverage a fully hyperbolic framework to embed DNA sequences, implicitly handling the latent hierarchies present in the data. Our main contributions are: 1. We adopt the machinery of fully hyperbolic convolutional neural networks (HCNNs), building two classes of HCNNs for genome sequence learning. We contrast hyperbolic and Euclidean approaches to sequence representation. 2. We introduce a novel, curated set of datasets—the Transposable Elements Benchmark—designed to investigate transposable elements, which remain an underexplored area of the genome with deep evolutionary roots. 3. We demonstrate the performance potential of our HCNNs across 42 real-world datasets addressing foundational challenges in genomics. 4. We elucidate the underlying mechanism by which HCNNs parse genomic signal by simulating and testing plausible data-generating processes for biological sequences. 5. We further motivate our work by formulating an empirical method for interpreting the hyperbolicity of dataset embeddings and use this technique to interrogate properties of genome representations generated by our models. 2 Preliminaries 2.1 Related Work Driven by the limitations of traditional Euclidean-based approaches in capturing relationships within complex data structures, hyperbolic deep learning methods have materialized as a promising research area. Early iterations of these methods introduced formalizations for performing the core operations of neural networks in hyperbolic space ( Ganea et al. 2018 ; Nickel & Kiela 2018 ) , alongside optimization techniques generalized to Riemannian manifolds ( Bécigneul & Ganea 2019 ) . These approaches have been further extended to a variety of frameworks, including fully hyperbolic neural networks ( Chen et al. 2022b ) , hyperbolic graph convolutional networks ( Chami et al. 2019 ) , hyperbolic attention networks ( Gulcehre et al. 2018 ) , and hyperbolic variational auto-encoders ( Mathieu et al. 2019 ) . These models, among others, have proven effective across a variety of real-world domains, including vision ( Liu et al. 2020 ; Hsu et al. 2021 ; Mathieu et al. 2019 ) , natural language ( Tifrea et al. 2019 ; Chen et al. 2024 ) , and computational biology ( Zhou & Sharpee 2021 ; Tian et al. 2023 ) . In genomics, hyperbolic methods have correctly modeled established phylogenies, showcasing their supremacy in representing tree-structured data ( Chami et al. 2020a ; Jiang et al. 2022b ; Hughes et al. 2004 ; Chen et al. 2025 ) . These methods assume that the phylogenetic tree is known a priori ; thus, the scope of these techniques is limited by the availability of evolutionary metadata. A subset of these methods produces representations of DNA sequences but relies on an explicit mapping of phylogenetic relationships ( Corso et al. 2021 ; Jiang et al. 2022a ) in the form of pairwise edit distances or incomplete phylogenies. 2.2 Background The n -dimensional hyperbolic space \mathbb{H}^{n}_{K} is a homogeneous, simply connected Riemannian manifold, (\mathcal{M}^{n},\mathfrak{g}^{K}_{x}) , consisting of a smooth manifold \mathcal{M}^{n} with Riemannian metric \mathfrak{g}^{K}_{x} , and described by a constant negative curvature K<0 . Several equivalent formulations of hyperbolic space exist, including the Lorentz model, the Poincaré disk model, and the (Beltrami-)Klein model. Here, we use the Lorentz model, \mathbb{L}^{n}_{K}=(\mathcal{L}^{n},\mathfrak{g}^{K}_{x}) , with manifold \mathcal{L}^{n} , Riemannian metric tensor \mathfrak{g}^{K}_{x}= diag (-1,1,\dots,1) , and origin \overline{\bm{0}}=[\sqrt{-1/K},0,\dots,0]^{T} . The Lorentz model describes points by their configurations on the forward sheet of a two-sheeted hyperboloid \mathbb{L}^{n}_{K} in (n+1) -dimensional Minkowski space, defining the manifold as: \mathcal{L}^{n}:=\{\bm{x}\in\mathbb{R}^{n+1}\mid\langle\bm{x},\bm{x}\rangle_{\mathcal{L}}=\frac{1}{K},x_{t}>0\}, (1) where the Lorentzian inner product is as follows: \langle\bm{x},\bm{y}\rangle_{\mathcal{L}}:=-x_{t}y_{t}+\bm{x}^{T}_{s}\bm{y}_{s}=\bm{x}^{T}\text{diag}(-1,1,\dots,1)\bm{y}. (2) Utilizing special relativity conventions, the zeroth element in \bm{x} is denoted as the timelike component x_{t} and the remaining n-1 elements form the spacelike components \bm{x}_{s} , giving \bm{x}=[x_{t},\bm{x}_{s}]^{T} , where we can further define the timelike component x_{t}=\sqrt{||\bm{x}_{s}||^{2}-1/K} .

Read the complete text