Instruction-Tuned, but Not More Verifiable Instruction-Following: A Cross-Task Diagnosis for LoRA Adapters

Junyi Zou · 2026

Read Instruction-Tuned, but Not More Verifiable Instruction-Following: A Cross-Task Diagnosis for LoRA Adapters by Junyi Zou (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-22379.txt.

Introduction

Zou tests whether a LoRA adapter's label tells the truth about what it improves. Zou runs one adapter across many tasks and compares name against fact. The test target is strict, machine-checkable instruction following on IFEval. Across seeds, base models, and settings, the label often fails to predict gains there. In one case, an instruction-tuned adapter sharply lifts a numeric benchmark. Its verifiable instruction following does not rise at all. Zou calls the pattern capability drift. The advice: test adapters across tasks before shipping them.

Catalog record

CREATORJunyi Zou
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE30102
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.22379

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Instruction-Tuned, but Not More Verifiable Instruction-Following: A Cross-Task Diagnosis for LoRA Adapters Junyi Zou Abstract Adapters are often selected and deployed based on nominal labels (e.g., instruction-tuned ), which implicitly suggest what capability improves after adaptation. We test whether nominal training objectives reliably align with realized cross-task capability gains by evaluating the same LoRA adapter across tasks. Our strongest evidence is tied to strict, automatically verifiable instruction following as measured by IFEval: across multiple seeds, base models, and LoRA settings, nominal labels recurrently—but not universally—fail to predict improvements on this verifiable target, with clear configuration sensitivity including a near-zero or negative case. As an illustrative strongest-case example in a controlled instruction-versus-numeric setting, an instruction-tuned adapter substantially improves off-target NM-based numeric benchmark performance (0.133 → \rightarrow 0.632) while not improving verifiable instruction following on IFEval (ILA: 0.313 → \rightarrow 0.271; PLA: 0.250 → \rightarrow 0.143; values rounded to three decimals). We refer to this nominal–realized mismatch pattern as capability drift as a descriptive label. The mismatch is visible in the raw cross-task performance matrix; we use a drift score only as a compact summary in the same units as the underlying metrics, not as a new formal metric contribution. Evidence from broader instruction-following benchmarks is benchmark-dependent and mixed, reflecting heterogeneity in how instruction following is operationalized; we therefore do not treat cross-benchmark agreement as a premise. Overall, the practical takeaway is to perform routine cross-task evaluation before deployment and to avoid treating nominal labels as reliable capability proxies. 1 Introduction LoRA adapters ( 5 ) are often trained with a nominal task label— instruction-tuned , numeric-reasoning-tuned , etc.—and then treated as if that label reliably describes what improves after deployment. In this paper, our strongest evidence is explicitly bounded to strict, automatically verifiable instruction following (IFEval) rather than to all notions of instruction following. We ask a deployment-facing question: do nominal training objectives reliably predict realized cross-task capability gains, especially on verifiable compliance that can be audited automatically? We frame the work as an empirical cross-task diagnosis: evaluate the same adapter across tasks and compare its nominal-objective gain to its realized gains elsewhere. Across multiple seeds, base models, and LoRA settings in our runs, we find the nominal–realized mismatch recurrent but not universal and clearly configuration-sensitive, including a near-zero or slightly negative case. Table 1 provides an illustrative strongest-case example: an instruction-tuned adapter substantially improves off-target NM-based numeric benchmark performance while not improving strict verifiable instruction following on IFEval ( 20 ) . The primary empirical support, however, comes from the robustness pattern across seeds/models/settings summarized in Table 2 and Appendix Table 6 . Instruction tuning is widely used to improve controllability ( 18 ) , and instruction following can sometimes be elicited via prompting even without instruction tuning ( 1 ) . But improved helpfulness or instruction-following behavior in broad settings does not imply improved verifiable compliance under strict constraints ( 20 ; 18 ) . We therefore treat benchmark heterogeneity as central rather than incidental: different benchmarks operationalize “instruction following” differently, so agreement across them should not be assumed a priori. We focus on empirical cross-task diagnosis rather than adapter composition, mechanistic attribution, or metric innovation. We use capability drift only as a descriptive label for the observed nominal–realized mismatch pattern. The practical motivation is that nominal labels are used in model selection and deployment; cross-task evaluation provides a low-cost check for unintended or off-target shifts. Contributions. We make three restrained contributions aligned with our experimental evidence: 1. We formulate a deployment-relevant cross-task diagnostic framing: test whether an adapter’s nominal label/training objective aligns with realized cross-task capability gains under evaluation. 2. We provide robustness evidence across seeds, base models, and LoRA settings, showing the mismatch recurrent but configuration-sensitive in magnitude, including a near-zero or slightly negative case. 3. We give a controlled counterexample: an instruction-tuned adapter can substantially improve off-target NM-based numeric benchmark performance while not improving strict, automatically verifiable instruction following on IFEval. We additionally report supplementary probing and breakdown analyses, but we treat them as descriptive/exploratory rather than as mechanistic evidence. 2 Related Work Adapters, composition, and merge side effects. Parameter-efficient adaptation via inserted modules (adapters) ( 5 ; 4 ) supports modular reuse and composition. Prior work explores sharing and composing task-specific components ( 10 ; 11 ) , including dynamic composition of LoRA modules ( 6 ) , as well as weight-space mixing and merging of fine-tuned models ( 16 ; 7 ; 17 ) . These lines emphasize interoperability and sometimes analyze when combinations succeed or fail; in contrast, our setting does not involve merging multiple trained adapters or composing multiple tasks at inference time, and we instead diagnose the mismatch between a single adapter’s nominal objective and its realized cross-task capability gains. Negative transfer and cross-task trade-offs. Multi-task and transfer learning can exhibit cooperation as well as competition, where optimizing for one objective harms performance on another ( 2 ; 13 ; 14 ) . Our empirical phenomenon is related but more specific: we diagnose a divergence between the nominal training objective (e.g., instruction tuning) and the most improved measured capability under cross-task evaluation, even without explicit multi-task training. We use capability drift as shorthand for this mismatch; the contribution is the cross-task diagnostic framing and controlled robustness evidence rather than introducing a new formal object. Instruction tuning and evaluation mismatch. Instruction tuning is broad and evaluated with heterogeneous benchmarks and protocols ( 9 ; 18 ; 15 ) . Strict, automatically verifiable instruction-following benchmarks such as IFEval ( 20 ) capture a narrower notion of compliance than preference-based or open-ended evaluations ( 19 ) . Recent work further highlights that verifiable instruction-following performance can be benchmark-dependent and may not generalize to unseen constraint families ( 12 ) . Our main quantitative evidence is therefore anchored on verifiable instruction following, and we treat mixed evidence across additional benchmarks as contextual rather than as generalizing to all notions of instruction following. Overlap, interference, and localized versus distributed explanations. Comparing learned changes across fine-tuning runs is often framed in terms of representation similarity and shared directions ( 8 ) . Related to this, weight-space “task vectors” ( 7 ) provide a lens for describing learned updates and their interactions. More broadly, interpretability work on superposition and polysemantic features suggests why changes may be distributed rather than localized to a single unit or module ( 3 ) . Our probing follows this spirit but remains deliberately conservative: we test whether a simple localized account emerges, and interpret negative results as evidence against a single-module explanation rather than as a definitive mechanistic conclusion. 3 Setup 3.1 Base model and LoRA adapters We consider a fixed base model and two primary LoRA adapters trained with different nominal objectives: reason (numeric-reasoning-tuned) and instr (instruction-tuned). We also train/evaluate a domain adapter as a secondary setting, but it is not a core pillar of the main claim and is deferred to the appendix. We do not retrain large models; all analyses reuse existing training and evaluation artifacts. 3.2 Tasks and metrics We evaluate three task families: (i) numeric reasoning benchmark , (ii) verifiable instruction following , and (iii) domain QA (secondary). Numeric reasoning benchmark. We report numeric match (NM), which measures

Read the complete text