Latent Space Disentanglement via Activation Steering for Interpretable Attribute Control in Symbolic Music Generation
Ioannis Prokopiou, Pantelis Vikatos, Maximos Kaliakatsos-Papakostas, Theodoros Giannakopoulos, Themos Stafylakis · 2026
Read Latent Space Disentanglement via Activation Steering for Interpretable Attribute Control in Symbolic Music Generation by Ioannis Prokopiou, Pantelis Vikatos, Maximos Kaliakatsos-Papakostas, Theodoros Giannakopoulos, Themos Stafylakis (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-31295.txt.
Introduction
Prokopiou and colleagues steer musical traits in a symbolic music model with no new training. Their method finds latent paths for pitch and duration in the Multitrack Music Transformer. It uses a mean-difference method. Steering strength tracks trait shift closely, the tests show. Steering two traits at once tangles them, so a dual method with Gram-Schmidt pulls them apart in geometric terms. The result is separate, exact control of traits even against strong autoregressive pull, with less clash and signal loss.
Catalog record
| CREATOR | Ioannis Prokopiou, Pantelis Vikatos, Maximos Kaliakatsos-Papakostas, Theodoros Giannakopoulos, Themos Stafylakis |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 22514 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.31295
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Latent Space Disentanglement via Activation Steering for Interpretable Attribute Control in Symbolic Music Generation Ioannis Prokopiou, Pantelis Vikatos, Maximos Kaliakatsos-Papakostas, Theodoros Giannakopoulos, Themos Stafylakis Abstract Transformer-based architectures have significantly advanced the generation of complex symbolic sequences, yet a significant gap remains in achieving fine-grained, interpretable control over discrete signal attributes. This paper investigates the mechanistic interpretability of the Multitrack Music Transformer (MMT) and proposes a framework for deterministic attribute modulation without retraining to bridge this gap via inference-time activation steering. Utilizing the Difference-in-Means (DiffMean) methodology, we isolate latent directions for signal attributes, specifically Pitch and Duration, within the residual stream. We validate the Linear Representation Hypothesis in this domain, achieving high correlation between steering magnitude and attribute shift. To address the inherent feature entanglement in multi-attribute steering, we introduce a Dual Steering framework utilizing Gram-Schmidt Orthogonalization. Experimental results demonstrate that this geometric decoupling reduces conceptual interference and signal degradation compared to naive vector addition, enabling independent deterministic control even against strong autoregressive conditioning. I Introduction The integration of deep learning into the creative arts has produced state-of-the-art autoregressive models capable of respecting long-term structural dependencies [ 1 ] . However, these models often function as “black boxes” [ 23 ] where the relationship between high-dimensional internal activations and discrete output attributes is obscured. For applications requiring precise intervention such as algorithmic composition or signal synthesis, global conditioning often lacks the granularity needed for specific manipulation of internal states. There is a pressing need for precise, inference-time intervention methods that do not require computationally expensive retraining without disrupting the entire generative process [ 14 ] . The field of mechanistic interpretability [ 4 ] seeks to reverse-engineer neural networks to understand the algorithms they implement. A central tenet is the Linear Representation Hypothesis, which posits that high-level concepts are represented as vectors within the activation space [ 9 , 2 ] . While validated in Large Language Models (LLMs), its application to symbolic signal domains remains nascent. Identifying these vectors enables Activation Steering, the process of intervening on the model’s internal activations during the forward pass to bias the generation toward a desired attribute [ 18 ] . While recent advancements have led to unified frameworks capable of high-quality generation via multi-modal conditioning, such as Seed-Music [ 3 ] , XMusic [ 22 ] , and the flow-matching based JASCO [ 20 ] , these systems often require extensive retraining or complex architectural changes. This work addresses the problem of inference-time steerability. Models such as the Multitrack Music Transformer (MMT) [ 8 ] utilize specialized data representations to encode discrete musical events into token sequences, learning the high-dimensional probability distributions of musical corpora like the Symbolic Orchestral Database (SOD) [ 7 ] . Rather than focusing on computationally expensive fine-tuning or prompt engineering, we apply mechanistic interpretability to reverse-engineer the latent representations of a pre-trained MMT, focusing on Average Note Pitch and Average Note Duration. Our contribution provides a layer-wise sensitivity analysis to map attribute encoding topology and investigate linear steerability within the MMT and evaluates steering dynamics across the network depth. We implement orthogonalization methods to disentangle correlated features, ensuring independent modulation and precise dual-concept control. Audio examples and code on https://giannisprokopiouorfium.github.io/music-transformer-sae/ . II Related Work Symbolic music generation has transitioned from rule-based systems to sophisticated deep learning architectures, with the Transformer [ 25 ] architecture revolutionising the modeling of long-term structural dependencies. The MMT introduced a compact 6-tuple event representation, incorporating type, beat, position, pitch, duration, and instrument, to significantly reduce sequence lengths compared to prior formats like REMI+ [ 26 ] . This efficiency makes MMT an ideal backbone for real-time steering research. Activation engineering [ 24 ] facilitates this shift by optimizing internal model states rather than input prompts grounded in the Linear Representation Hypothesis [ 17 ] . Research on LLMs has demonstrated that injecting these vectors during the forward pass can deterministically bias outputs without weight optimization [ 12 ] . The Difference-in-Means (DiffMean) [ 15 , 18 ] methodology serves as a robust baseline for extracting these vectors by capturing the centroid difference between contrasting concept clusters. Recent benchmarks like AxBench [ 28 ] confirm that DiffMean often outperforms complex alternatives like Sparse Autoencoders (SAEs) for steering tasks. In the musical domain, mechanistic interpretability [ 4 ] is an emerging frontier. Early attempts at controllable synthesis utilized latent regularization in architectures like Music FaderNets [ 21 ] , or compositional augmentation of Transformer architectures [ 30 ] . Panda et al. [ 16 ] demonstrated that steering residual streams and attention heads can enable fine-grained style transfer and genre fusion. Modern techniques such as SMITIN [ 13 ] employ self-monitored interventions with classifier probes to dynamically adjust steering strength, while MusicRFM [ 31 ] analyzes internal gradients to identify concept directions for autoregressive models. Most relevant to our study is the work of Facchiano et al. [ 10 ] , which applies activation patching to interpretable latent directions, confirming that localized interventions can effectively modulate high-level musical attributes. In their work they applied activation patching to MusicGen [ 6 ] to control tempo and timbre, identifying mid-range layers as the most effective locus for intervention. Our research distinguishes itself by applying these concepts to the symbolic domain, where the discrete nature of MIDI-based attributes allows for mathematically precise evaluation and manipulation. A persistent challenge in multi-attribute steering is the conceptual interference between correlated features [ 29 ] , such as the natural entanglement of pitch and duration. While techniques involving geometric operations, such as angular steering [ 27 ] , have been proposed to define stable steering planes in latent space, and the Composer Vector method [ 11 ] that utilizes latent space arithmetic to fuse styles, we build upon these mathematical foundations by applying Gram-Schmidt Orthogonalization [ 5 ] . By explicitly decoupling correlated features, this approach prevents interference during generation and addresses the entanglements typically learned by the model’s residual stream, ensuring independent control over discrete musical attributes. III Methodology III-A Signal Representation and Model Infrastructure We utilize the pre-trained MMT, a decoder-only Transformer architecture trained on the SOD dataset. The model processes musical events as discrete 6-dimensional tuples, which are embedded into a 512-dimensional continuous vector space. Each event tuple consists of: • Type (4 tokens): Categorizes the event (note, time-shift, end-of-song, instrument-change). • Beat (257 tokens): Integer beat position (0 to 256). • Position (384 tokens): Sub-beat resolution at intervals. • Pitch (128 tokens): MIDI values ranging from 0–127. • Duration (768 tokens): Note length in ticks, from staccato (1 tick) to sustained (768). • Instrument (128 tokens): MIDI program numbers for orchestral instrumentation (0 to 127). Our steering interventions target Pitch (dimension 4) and Duration (dimension 5) for interpretability by modifying the residual stream activations that influence the probability distribution over these discrete vocabularies at generation time. III-B Latent Vector Extraction and Concept Definition To compute steering vectors via DiffMean, we defined “High” and “Low” concept thresholds by analyzing the statistical distribution of the SOD corpus. Thresholds (Table I ) were empirically set at the 20th and 80th quantiles, yielding robust, representative, non-overlapping clusters and stable latent directions across alternative extreme quantile choices. TABLE I: Data Curation Quintile Thresholds