Reverse–Complement Consistency for DNA Language Models
Mingqian Ma · 2025
Read Reverse–Complement Consistency for DNA Language Models by Mingqian Ma (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-18529.txt.
Introduction
DNA reads the same forward and backward: a sequence and its reverse complement carry the same meaning. Mingqian Ma shows that top DNA language models often miss that symmetry in this 2025 paper. Their predictions can change when the same stretch is fed in flipped form. That inconsistency undercuts trust in the models for real biology work.
Ma proposes a fine-tuning rule that directly punishes any gap between a model's output on a sequence and on its flipped twin. The rule works with any model design. Tests spanned three model families and many task types, from sorting sequences to predicting profiles. The rule cut prediction flips and errors sharply. Accuracy held steady or improved versus fixes like flipped-data training or averaging at test time. The recipe leaves one single, robust model that is cheap to run.
Catalog record
| CREATOR | Mingqian Ma |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-18529
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Reverse–Complement Consistency for DNA Language Models Mingqian Ma Abstract A fundamental property of DNA is that the reverse complement (RC) of a sequence often carries identical biological meaning. However, state-of-the-art DNA language models frequently fail to capture this symmetry, producing inconsistent predictions for a sequence and its RC counterpart, which undermines their reliability. In this work, we introduce Reverse-Complement Consistency Regularization (RCCR) , a simple and model-agnostic fine-tuning objective that directly penalizes the divergence between a model’s prediction on a sequence and the aligned prediction on its reverse complement. We evaluate RCCR across three diverse backbones (Nucleotide Transformer, HyenaDNA, DNABERT-2) on a wide range of genomic tasks, including sequence classification, scalar regression, and profile prediction. Our experiments show that RCCR substantially improves RC robustness by dramatically reducing prediction flips and errors, all while maintaining or improving task accuracy compared to baselines such as RC data augmentation and test-time averaging. By integrating a key biological prior directly into the learning process, RCCR produces a single, intrinsically robust, and computationally efficient model fine-tuning recipe for diverse biology tasks. Code available at: https://anonymous.4open.science/r/RCCR-D2D0 . 1 Introduction DNA language models (DNA LMs) ( Zhou et al. 2024 ; Dalla-Torre et al. 2025 ; Nguyen et al. 2023 ; Ma et al. 2025 ) have become general-purpose backbones for genomic prediction and sequence design: after pretraining on raw genomes, a single backbone can be fine-tuned for diverse downstream tasks. Many of these tasks possess an explicit symmetry: labels are reverse–complement (RC) invariant at the sequence level (e.g., promoter classification), or RC equivariant at the profile level, where outputs must be aligned by a task-specific operator \Pi (e.g., bin-wise outputs should be flipped along the sequence length axis, and strand channels swapped when present). Yet standard fine-tuning pipelines neither encode RC symmetry nor evaluate it systematically, leaving models sensitive to input orientation. Empirically, reversing and complementing a sequence can alter a model’s output even when ground truth is unchanged or predictably transformed by \Pi . This orientation sensitivity degrades reliability (e.g., elevated flip rates between predictions on x and \mathrm{RC}(x) ), complicates interpretation, and can confound large-scale scans. Existing remedies target either the training or inference stages. Test-time averaging (TTA) averages predictions from x and \mathrm{RC}(x) at inference; it doubles inference cost, leaves the learned predictor unchanged, and offers no training-time control of inconsistency. RC data augmentation ( Zhou et al. 2022 ) trains on both x and \mathrm{RC}(x) with identical labels, but it still does not enforce agreement between the two orientations. Architectural approaches that hard-code RC equivariance (e.g., RC parameter sharing and recent RC-equivariant long-range models) ( Shrikumar et al. 2017b ; Schiff et al. 2024 ) can reduce flexibility and may be incompatible with widely used pretrained backbones; they also cannot accommodate the tasks that requires explicit RC variation. For a detailed related work overview, see Appendix A . In this paper, we propose an architecture- and head-agnostic drop-in objective for enforcing RC consistency during fine-tuning, called Reverse–Complement Consistency Regularization (RCCR) . The core idea is to regularize the predictor to agree across the two orientations after task-aware alignment. Let f_{\theta}(x) denote the model output and \tilde{f}_{\theta}(x)=\Pi f_{\theta}(\mathrm{RC}(x)) the aligned output for the reverse complement. We add to the task loss a lightweight consistency term D\!\left(\phi\!\left(f_{\theta}(x)\right),\,\phi\!\left(\tilde{f}_{\theta}(x)\right)\right) , where \phi maps outputs to a common space (e.g., temperature-scaled probabilities for classification; raw or stabilized signals for regression/profiles). In practice, we instantiate D as the symmetric KL divergence for classification, and as squared error or Poisson KL for regression and profiles. We provide theoretical guarantees: symmetrization is risk non-increasing under RCCR, and with RC-symmetric labels, global minimizers are RC-consistent after alignment. For classification, the symmetric KL penalty controls the Jensen–Shannon divergence and is locally well-conditioned in logit space, which stabilizes gradients. These results explain why enforcing agreement during training should not sacrifice task risk when the label is RC-symmetric. To make evaluation standard and comparable, we report RC-aware metrics alongside task metrics: (i) \mathrm{SFR} , the class flip rate between x and \Pi f(\mathrm{RC}(x)) ; (ii) RC-Corr , the correlation between original and RC-aligned predictions. We evaluate RCCR on heterogeneous backbones: Nucleotide Transformer, DNABERT-2, HyenaDNA ( Dalla-Torre et al. 2025 ; Zhou et al. 2024 ; Nguyen et al. 2023 ) and three task families (sequence-level classification, scalar regression, and profile regression), emphasizing that improvements are not an artifact of a particular tokenization or inductive bias. Across datasets, RCCR improves orientation robustness (lower \mathrm{SFR} and higher RC-Corr ) while maintaining task accuracy, matching TTA-like robustness with better explainability and without its 2\times inference cost. We also note that RCCR encodes an explicit RC prior and is not appropriate for intrinsically strand-specific targets unless channels are relabeled or masked; we include a strand-classification control to showcase this. To summarize our contributions: 1. Method. We propose a drop-in fine-tuning objective (RCCR) that enforces RC consistency across classification, scalar regression, and profile heads without modifying the backbone. 2. Evaluation. We introduce a compact RC robustness suite ( \mathrm{SFR} , RC-Corr) to report alongside task metrics, standardizing orientation-robustness reporting. 3. Theory. We prove that symmetrization is risk non-increasing and that, with RC-symmetric labels, global minimizers are RC-consistent; the symmetric KL classification penalty is stable and informative. 4. Empirics & scope. We demonstrate across diverse backbones and tasks that RCCR improves RC robustness (and task performance due to theoretical symmetrization), and we include a negative control on strand-specific prediction clarifying when not to apply RCCR. The remainder of the paper is organized as follows: Section 2 formalizes RC symmetry and task-specific alignment. Section 3 presents the method in detail and our supporting theoretical guarantees. Section 4 presents experimental results and analysis. 2 Preliminaries 2.1 DNA language models Pretraining for DNA LMs typically uses masked-language modeling (MLM) or next-token prediction (NTP), learning token-level conditionals p_{\theta}(\cdot\mid\cdot) . We denote the pretrained backbone by h_{\theta} and attach a task head g_{\theta} at fine-tuning time; the predictor is f_{\theta}=g_{\theta}\circ h_{\theta} . In MLM, a random index set M\subseteq\{1,\dots,L\} is replaced by [MASK] to form x_{\setminus M} , and we maximize the masked likelihood \mathcal{L}_{\mathrm{MLM}}(\theta)=\mathbb{E}\!\left[-\sum_{i\in M}\log p_{\theta}\!\left(x_{i}\,\middle|\,x_{\setminus M}\right)\right]. In NTP, the joint is factorized autoregressively and optimized by \mathcal{L}_{\mathrm{NTP}}(\theta)=\mathbb{E}\!\left[-\sum_{i=1}^{L}\log p_{\theta}\!\left(x_{i}\,\middle|\,x_{<i}\right)\right],\quad x_{<i}=(x_{1},\dots,x_{i-1}). All methods here are agnostic to whether h_{\theta} arose from MLM or NTP. 2.2 Reverse complement and task-aware alignment Let \mathcal{A}=\{A,C,G,T,N\} . Define base complement c:\mathcal{A}\!\to\!\mathcal{A}\quad by c(A)=T , c(T)=A , c(C)=G , c(G)=C , c(N)=N , and let R reverse indices so (R(x))_{i}=x_{L+1-i} . We write the reverse complement as \tau(x)\;\coloneqq\;\mathrm{RC}(x)\;=\;R\!\big(c(x)\big),\qquad\mathrm{so}\quad\mathrm{RC}(x)_{i}=c\!\left(x_{L+1-i}\right), and note that \tau is an involution: \tau(\tau(x))=x . To compare predictions across orientations, we apply a task-aware alignment operator \Pi to outputs computed on \tau(x) . For sequence-level heads, \Pi=\mathrm{id} . For position/bin-wise heads with B=\lceil L/r\rceil bins and K channels, \Pi reverses the positional axis and, when present, permutes strand channels via a fixed self-inverse permutation \pi : \big(\Pi S(x)\big)_{b,k}\;=\;S\!\big(\tau(x)\big)_{B+1-b,\,\pi(k)}. In RC-symmetric settings the intended agreement is \phi\!\big(f_{\theta}(x)\big)\;=\;\phi\!\big(\Pi f_{\theta}(\tau(x))\big), where \phi is the task-specific link (identity for real-valued outputs; softmax for logits). We use \tau(\cdot) and \mathrm{RC}(\cdot) interchangeably in what follows. 2.3 Downstream tasks at arbitrary resolution We consider sequence-level and position/bin-wise prediction at arbitrary spatial resolution r\in\mathbb{N} (base-pair when r=1 , otherwise r bp per bin). Sequence-level classification. Logits