Output Composability of QLoRA PEFT Modules for Plug-and-Play Attribute-Controlled Text Generation
Michela Lorandi, Anya Belz · 2026
Read Output Composability of QLoRA PEFT Modules for Plug-and-Play Attribute-Controlled Text Generation by Michela Lorandi, Anya Belz (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-12345.txt.
Introduction
Lorandi and Belz explore how to steer text generation by combining small fine-tuned modules instead of retraining for every new mix of traits. They compare training on combined data sets, adding module weights at inference, and adding module outputs at inference, using QLoRA modules on three language models. Summing outputs proves strongest, matching or beating the alternatives on sentiment, topic and multi-attribute control. Even against single-task specialists on their own tasks, three-module output composition gains about two points on average.
Catalog record
| CREATOR | Michela Lorandi, Anya Belz |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 92947 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.12345
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Output Composability of QLoRA PEFT Modules for Plug-and-Play Attribute-Controlled Text Generation Michela Lorandi, Anya Belz Abstract Parameter-efficient fine-tuning (PEFT) techniques offer task-specific fine-tuning at a fraction of the cost of full fine-tuning, but require separate fine-tuning for every new task (combination). In this paper, we explore three ways of generalising beyond single-task training/inference: (i) training on combinations of multiple, related datasets; (ii) at inference, composing the weight matrices of separately trained PEFT modules; and (iii) at inference, composing the outputs of separately trained PEFT modules. We test these approaches on three different LLMs, QLoRA as the PEFT technique, and three sets of controlled text generation datasets for sentiment control, topic control, and multi-attribute control. We find that summing PEFT module outputs is a particularly strong composition method, which consistently either outperforms or matches the performance of alternative approaches. This is the case even when comparing against single-task specialised modules on the single-task test set, where three-module output composition achieves an average 2% point performance increase across all models for sentiment control. 1 Introduction Given the substantial costs of training state-of-the-art language models, parameter-efficient finetuning (PEFT) techniques such as Adapters 12 , Prefix Tuning 20 , and LoRA 13 are have become an important part of the toolboxfor task-specific adaptation of pretrained models. PEFT techniques produce parameter matrices that are modular in the sense that they can be attached, detached and replaced in the same host model as needed, and this holds out the appealing (but currently far from realisable) vision of full plug-and-playability where individual task-specifically trained modules can be combined at will to achieve specific model behaviours, potentially even in conjunction with different host models 25 . One aim in combining modules in this way is to achieve the combined functionality, e.g. controlling multiple attributes of generated text for which the individual modules have been (separately) trained. However, it is also desirable to preserve performance at the separate tasks. We examine the extent to which (i) composite-task performance can be achieved, and (ii) single-task performance preserved, when QLoRA PEFT modules are combined in host models. We test the composability of PEFT modules in this sense with different composition techniques on the composite task of multiple-attribute control, and the single tasks of sentiment and topic control. We compare output and parameter composition, as well as a baseline of training a single module on multiple datasets. We focus on attribute-controlled text generation tasks, because these well established tasks enable systematic evaluation across multiple datasets while providing a controlled setting to isolate the core mechanics of PEFT module composition. In presentation order, our main contributions are: 1. A new plug-and-play implementation of QLoRA supporting PEFT module composition for any number of modules, with three different composition techniques (Section 3 ); 2. New output composition techniques for PEFT modules (Section 3.1 ); 3. A new method for combining disparate datasets into a single, representative dataset (Section A ); and 4. New test results for all composition techniques on 3 models, 14 datasets, and 3 composition techniques (Section 7 ). 2 Related Research Parameter-efficient fine-tuning (PEFT) has been shown ( 21 ; 13 ; 24 ; 30 ) to effectively inject task-specific knowledge into pretrained models, including in the case of raw models ( 32 ) where (most of) the task-level knowledge must be acquired during finetuning. PEFT modules have been tested for cross-task transferability, with prompt tuning 27 ; 29 , and other PEFT techniques, where e.g. 8 showed that PEFT-tuning, e.g. with LoRA, maintains performance on other tasks only when they are closely related. Multiple LoRA modules have been used in mixture-of-expert set-ups, where the single most suitable module is selected on the fly for a given task ( 10 ; 9 ) . Recent work has also explored rank-wise clustering approaches to LoRA merging ( 33 ) . Most similar to our work, two papers have tested combining LoRA modules trained on general language tasks by computing the weighted sums of their parameters (with weights hardwired or optimised) 2 ; or by (gradient-free) combinatorial optimisation for automatic selection of modules 14 . The downsides are (i) additional learning steps after composition, and (ii) inefficient weight learning across many modules despite limited supervised data. Our work differs from 2 in three key ways: (1) we introduce output composition methods which to our knowledge have not been previously explored for PEFT modules; (2) we focus on true plug-and-play composition without any post-composition training or optimization; and (3) we demonstrate that output composition not only generalises across related tasks but can even improve individual module performance on their original tasks. The scenario we address is more ambitious: we want to take multiple task-trained LoRA modules off the shelf, combine them without further learning steps in a host model, to achieve performant results on each of the tasks for which we have loaded a module, as well as on combined tasks requiring their combined functionality. To the best of our knowledge, composability in this sense has not so far been explored; demonstrating it for QLoRA modules for the first time is our aim in this paper. Our output composition techniques can be understood through the lens of representation engineering and activation steering ( 28 ; 34 ) , where model behaviour is controlled by manipulating internal representations. When we sum PEFT module outputs, we in effect combine learned steering directions in the model’s representation space. Unlike typical activation steering approaches which require manual direction finding through techniques like contrastive activation addition or additional optimization steps ( 19 ) , our approach leverages task-specific PEFT modules as pre-learned steering vectors that can be composed without further training. This connection helps explain why output summing proves particularly effective: each PEFT module captures a task-specific representational direction, and their linear combination naturally integrates these learned behaviours. 3 Plug-and-play QLoRA We reimplemented 1 1 1 Available from https://github.com/anonymised QLoRA with additional functionality to support the attachment of multiple PEFT modules, and their composition with different techniques. In vanilla single-module QLoRA, a PEFT block consisting of a down-projection, and up-projection is attached (as shown in Figure 2 in Appendix) in parallel to every key, query, value and feed-forward layer in every transformer block. Adopting the notation from 2 , QLoRA learns a far smaller set of task-specific parameters \Delta\Theta in conjunction with the frozen pre-trained model \Theta_{0} . \Delta\Theta (the QLoRA module) consists of trainable low-rank decomposition matrices added as adapter modules to the query, key, value and feed-forward weight matrices in the frozen model \Theta_{0} . In other words, QLoRA adds task-specific parameters \mathbf{\Delta W} to pre-trained frozen weight matrices \mathbf{W}_{0}\in\mathbb{R}^{d\times c} in \Theta_{0} : \mathbf{\hat{W}}=\mathbf{W_{0}}+\frac{\alpha}{r}\mathbf{\Delta W} where \mathbf{\Delta W}=\mathbf{AB^{T}} is the QLoRA block, with \mathbf{A}\in\mathbb{R}^{d\times r} the down projection, \mathbf{B}^{T}\in\mathbb{R}^{r\times d} the up projection, \alpha a scaling factor, and r the rank. \mathbf{A} is initialised with the Kaiming uniform distribution, and \mathbf{B} with all-zero initialisation. \Delta\Theta is optimised with a standard conditional language modelling objective with CE loss computed over \Theta_{0}+\Delta\Theta (i.e. the frozen model with QLoRA blocks plugged in). For a given input x , the output h resulting from adapting the frozen parameters \mathbf{W_{0}} with a QLoRA block is: h=\mathbf{W_{0}}\>x+\frac{\alpha}{r}\mathbf{\Delta W}x In our implementation of plug-and-play