Attractive and Repulsive Pattern Control in Sequence Generation

François Pachet · 2026

Read Attractive and Repulsive Pattern Control in Sequence Generation by François Pachet (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-24911.txt.

Introduction

François Pachet gives sequence generators a dial that can attract or repel chosen patterns. Variable order Markov models generate symbolic sequences, like melodies, while preserving local style. Over long runs they fall into tunnels of repeated suffixes and near periodic passages. Earlier work compiled regular constraints into automata and sampled exactly with belief propagation. Pachet shows the same machinery works softly, in both directions. A weighted automaton scores how active a family of target patterns is in a candidate continuation.

Belief propagation then samples from a distribution tilted by that score. With negative coupling, target patterns grow costly and get avoided. With positive coupling, the same patterns become controlled attractors. Targets can be mined from the generated history, read off a score, drawn from a style vocabulary, or designed as probes. The main experiments target whichever patterns become overactive during sampling. Monophonic symbolic music is the test material throughout.

Catalog record

CREATORFrançois Pachet
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.24911

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Attractive and Repulsive Pattern Control in Sequence Generation François Pachet Abstract Variable-order Markov models are useful for symbolic sequence generation because they preserve local syntax or style while adapting context length. Over long continuations, they may enter “tunnels”: attractor-like corridors of recurring high-order contexts, repeated suffixes, or locally periodic passages. This tunneling is already present in unconstrained Markov continuation, before any belief propagation (BP) or regular constraints are introduced. Previous BP-Regular work showed that regular constraints can be compiled into automata and sampled exactly with BP; in particular, a max-order automaton can forbid overlong copied patterns, including patterns longer than the formal Markov order. This paper shows that the same automaton/BP interface can be used more softly, adaptively, and symmetrically. A weighted recurrence automaton computes a non-negative recurrence activation R R , for any specified family of target patterns, and BP samples exactly from P β ​ ( x ) ∝ P 0 ​ ( x ) ​ exp ⁡ ( β ​ R ​ ( x ) ) P_{\beta}(x)\propto P_{0}(x)\exp(\beta R(x)) . Negative coupling ( β < 0 ) (\beta<0) makes the target patterns progressively costly during sampling; positive coupling ( β > 0 ) (\beta>0) rewards the same patterns and turns them into controlled attractors. The target patterns need not be the most common patterns in the generated history: they may be mined from the history, supplied by a score, derived from a style vocabulary, or designed as an experimental probe. The core experiments instantiate the general mechanism homeostatically, by choosing patterns that become overactive in the sampling history. Monophonic symbolic music is used as a demanding testbed because recurrence is audible, measurable, and structurally meaningful. Controlled long-horizon comparisons on six duration-bearing monophonic sources, including Bach and Telemann material, show that this negative branch reduces generated 8-gram self-reuse, increases the effective number of generated 8-grams, and increases coverage of training-supported 4-gram contexts, while preserving substantial lower-order support. A pitch-sequence replication on five Weimar Jazz Database jazz solos gives the same anti-reuse signature outside Baroque material. The mechanism provides signed control of the generator’s recurrence landscape: negative weights provide homeostatic anti-collapse control, while positive weights provide an exact way to probe attractor basins, phase transitions, and hysteresis in the underlying variable-order model. 1 Introduction Symbolic sequence generation often requires more than locally plausible next-symbol decisions. A continuation may fit the learned short-range statistics while entering a recurring corridor: an overused suffix, a repeated motif, a locally periodic passage, or a copied fragment longer than the model’s formal order. This problem is especially visible in high-order and variable-order Markov models, where longer contexts improve local specificity but can also make long-horizon self-reuse more likely. Reducing memory or adding noise may break these repetitions, but it also weakens the structure that made the generator useful. Regular constraints provide a precise interface for controlling such behavior. In BP-Regular Markov generation, finite automata encode admissible continuations and belief propagation samples exactly from the Markov distribution conditioned on automaton acceptance [ 2 ] . This framework has already been used for hard endpoint and avoidance constraints, including max-order automata that forbid copied chunks exceeding a chosen length [ 4 ] . A recognizer can also define a graded preference: a pattern can be discouraged, rewarded, or swept continuously from neutral to repulsive to attractive. This paper introduces signed pattern control. A regular recognizer computes an activation R(x) for a candidate continuation x , and the sampler draws from a reweighted distribution P_{\beta}(x)\propto P_{0}(x)\exp(\beta R(x)). Negative coupling (\beta<0) penalizes the recognized pattern family, positive coupling (\beta>0) rewards it, and \beta=0 recovers the unmodified generator. The target family may be a motif, cadence, rhythm, contour, syntactic fragment, hand-authored taboo, shuffled control, or a family mined online from the generator itself. This separation between pattern selection and signed sampling is central. The experiments instantiate one policy homeostatically, by choosing patterns that become overactive during generation, and show that negative weights reduce long-horizon tunneling without changing the source model. A complementary fixed-motif probe shows that the same recognizer can be held neutral, made repulsive, or made attractive. Positive weights then become an experimental instrument for probing attractor basins. The idea that absence can itself be musically informative has a precedent in Conklin’s work on antipatterns: patterns that are rare, under-represented, or absent in a piece, corpus, or genre, and can serve as descriptive or discriminative musical features [ 5 ] . Conklin and Weisser further apply antipattern discovery to Ethiopian bagana songs, where rare motifs can reveal structural properties of a musical style [ 6 ] . The present use is complementary: a specified regular recognizer becomes a signed generative energy and is sampled exactly within the BP interface. Monophonic symbolic music is used here as a demanding testbed because recurrence is audible, measurable, and structurally meaningful. The construction applies beyond music: any discrete sequence domain with regular pattern families and a tractable constrained sampler can use the same signed control principle. In creative domains, signed pattern control steers and probes a generator’s recurrence landscape; proposal and selection remain separate modelling components. This connects to computational-creativity evaluation criteria, where novelty is balanced against typicality or value [ 7 ] , and to learning-progress accounts that distinguish arbitrary surprise from learnable regularity [ 9 ] . 2 BP-Regular Variable-Order Generation Belief propagation (BP) is the sum-product message-passing algorithm used to compute marginal weights in a factor graph. When the factor graph is a chain or a tree, BP is exact: messages propagated forward and backward give the exact normalizing mass and the exact conditional distributions needed for sequential sampling. Papadopoulos, Pachet, Roy, and Sakellariou used this observation to sample Markov sequences under Regular constraints by building a linear factor graph whose binary factors combine a Markov transition with the transition of a finite automaton [ 2 ] . BP-Regular sampling draws exactly from the Markov distribution conditioned on acceptance by one or more automata. The present generator keeps this BP-Regular interface but uses a variable-order Markov model. The exact variable-order extension is the sparse context-state construction of Pachet [ 3 ] : BP runs on the observed context states retained by the variable-order/backoff model and then crosses them with the regular constraint automaton. Given a current history h , the variable-order model supplies a distribution over the next symbol by backing off through an order stack: p(x_{t}\mid h_{t})=p_{o_{t}}(x_{t}\mid x_{t-o_{t}:t-1}), where o_{t} is the highest admissible context order retained by the model and its backoff policy. In ordinary sampling, this distribution is used directly. In BP-regular sampling, one instead samples a short future x_{1:T} subject to a regular constraint. A deterministic finite automaton A=(Q,q_{0},\delta,F) tracks whether the sampled future remains admissible: q_{t}=\delta(q_{t-1},x_{t}),\qquad q_{T}\in F. The hard constrained distribution is: P(x_{1:T}\mid h,A)=\frac{1}{Z(h,A)}\left(\prod_{t=1}^{T}p(x_{t}\mid h,x_{1:t-1})\right)\mathbf{1}[q_{T}\in F], where Z(h,A) is the normalizing success mass. BP computes and samples from this constrained product on the reachable context-state/automaton product. The automaton can change while the underlying Markov model stays fixed: hard ending constraints, avoidance constraints, and recurrence fields all enter through the same BP-regular interface. The recurrence automaton used here is also directly descended from the max-order constraint introduced by Papadopoulos, Roy, and Pachet [ 4 ] . That paper observed that generation can copy chunks

Read the complete text