GenCtrl – A Formal Controllability Toolkit for Generative Models
Emily Cheng, Carmen Amo Alonso, Federico Danieli, Arno Blaas, Luca Zappella, Pau Rodríguez, Xavier Suau · 2026
Read GenCtrl – A Formal Controllability Toolkit for Generative Models by Emily Cheng, Carmen Amo Alonso, Federico Danieli, Arno Blaas, Luca Zappella, Pau Rodríguez, Xavier Suau (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-05637.txt.
Introduction
Emily Cheng and 6 co-authors ask, in a 2026 paper, whether generative models are truly controllable before trying to control them better. Prompting and fine-tuning methods for steering generation are everywhere. Yet fine-grained control is pointless if the underlying model cannot reach the desired states at all.
Their GenCtrl toolkit frames human-model interaction as a control process and estimates a model's controllable set in dialogue. The estimates carry formal error guarantees that depend only on sample counts and bounded outputs, with no other assumptions. Tests on language and image models show controllability is surprisingly fragile and highly setting-dependent. The authors argue rigorous controllability checks should come before building more steering methods.
Catalog record
| CREATOR | Emily Cheng, Carmen Amo Alonso, Federico Danieli, Arno Blaas, Luca Zappella, Pau Rodríguez, Xavier Suau |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 106 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-05637
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
GenCtrl – A Formal Controllability Toolkit for Generative Models Emily Cheng, Carmen Amo Alonso, Federico Danieli, Arno Blaas, Luca Zappella, Pau Rodríguez, Xavier Suau Abstract As generative models become ubiquitous, there is a critical need for fine-grained control over the generation process. Yet, while controlled generation methods from prompting to fine-tuning proliferate, a fundamental question remains unanswered: are these models truly controllable in the first place? In this work, we provide a theoretical framework to formally answer this question. Framing human-model interaction as a control process, we propose a novel algorithm to estimate the controllable sets of models in a dialogue setting. Notably, we provide formal guarantees on the estimation error as a function of sample complexity: we derive probably-approximately correct bounds for controllable set estimates that are distribution-free, employ no assumptions except for output boundedness, and work for any nonlinear control system ( i.e., any generative model). We empirically demonstrate the theoretical framework on different tasks in controlling dialogue processes, for both language models and text-to-image generation. Our results show that model controllability is surprisingly fragile and highly dependent on the experimental setting. This highlights the need for rigorous controllability analysis, shifting the focus from simply attempting control to first understanding its fundamental limits. \metadata [Code] https://github.com/apple/ml-genctrl \metadata [Correspondence] [email protected] 1 Introduction The widespread deployment of generative models has driven significant research effort into controlling their outputs ( Zhang et al. 2023 ) . A diverse array of controlled generation methods has been developed, from prompting ( Marvin et al. 2023 ) and finetuning ( Wei et al. 2022a ; Ouyang et al. 2022b ) to steering towards specific styles or concepts ( Li et al. 2024 ; Rodriguez et al. 2025b ; Rimsky et al. 2024 ; Wu et al. 2024 ) . However, beneath the surface of this empirical progress lies a set of foundational, yet largely unexamined, assumptions. Specifically, current methods implicitly assume the model is fundamentally controllable. This rests on three key premises. First, they assume reachability : that a desired set of outputs is achievable using a given control mechanism and initial prompt. Second, they assume universal controllability , meaning the desired output is reachable from any initial state. Finally, they assume calibration , that is, the output is a direct function of the control variable. The absence of tools to formally verify these assumptions casts uncertainty over the reliability and fundamental limits of current controlled generation techniques. In this work, we bring these assumptions to the forefront. We provide tools to verify them by turning to control theory ( Sontag 1998 ) , adapting its methods to formalize the problem for modern AI systems. Casting controlled generation as a control process , we propose a framework to quantify its reachability and controllability with probabilistic guarantees. Our framework treats a generative model as an opaque nonlinear control system; it is not only agnostic to model architecture but is also designed to handle discrete or continuous-valued input and output spaces ( e.g., textual prompts and generations). Finally, we demonstrate our framework on a user-model dialogue setting, a common use-case of current generative models ( Google 2024 ; Yang et al. 2025 ; OpenAI 2023 ) . Our primary contributions are as follows: • A formal framework for controllability. We introduce a control-theoretic framework to rigorously define and quantify the reachable and controllable sets for any opaque system, including generative models. To the best of our knowledge, this provides the first formal language to characterize the operational boundaries of generative model control. • PAC algorithms for controllable set estimation. We present Monte Carlo algorithms for reachable and controllable set estimation from opaque system interactions, along with probably-approximately correct (PAC) confidence bounds ( sections 4.2 and 5 ). The algorithms are particularly well-suited for generative models as they are distribution-free, only assume boundedness of the target attribute, and explicitly account for the discrete bottlenecks inherent in many generative systems. • Empirical findings. We conduct a broad empirical analysis of controllability across models and tasks for text generation with LLMs, and image generation with text-to-image models (T2IMs). Leveraging our theoretical results ( sections 4.2 and 5 ), our experiments reveal that model controllability is not guaranteed. The significant heterogeneity of results across modalities, models, and tasks signals the need for case-specific controllability analyses. • An open-source toolkit. We release our framework and algorithms in https://github.com/apple/ml-genctrl , a PyTorch library at to facilitate analysis of controllability by the broader research community. This work argues for a paradigm shift where the controllability of generative models is not an implicit assumption but an explicit subject of analysis. By providing formal tools to do so, we lay a more principled foundation for future work in controllable AI. 2 Related Work Controlling generative models Research on controlling generative model outputs follows three main paradigms: (i) Prompt engineering, e.g., in-context learning or chain-of-thought prompting ( Brown et al. 2020 ; Wei et al. 2022b ) ; (ii) Finetuning, e.g., via RLHF or DPO ( Ouyang et al. 2022a ; Rafailov et al. 2023 ) , to align the model behavior; (iii) Representation engineering, which directly manipulates model activations ( Turner et al. 2023 ; Li et al. 2024 ; Suau et al. 2024 ; Rodriguez et al. 2025a ; Wu et al. 2024 ; Cheng et al. 2024 ) . Our framework can assess any control mechanism including all examples above. In this work we mainly focus on prompting, given its widespread use. Control theory and reachability analysis for generative models Controllability and reachability are core concepts in control and dynamical systems theory ( Sontag 1998 ; Borrelli et al. 2017 ) . Despite their widespread use in other areas of engineering, such as robotics ( Nakamura et al. 2025 ) , the application of control theoretic concepts to machine learning has mainly been limited to reinforcement learning ( Recht 2019 ) , or to analyze training dynamics ( Han et al. 2019 ) . Despite recent efforts to develop theory for generative models ( Soatto et al. 2023 ; Marchi et al. 2024 ) , these often rely on impractical assumptions. Extending control-theoretic concepts to large generative models poses several challenges, mostly due to their nonlinear and high-dimensional nature ( Bansal et al. 2017 ) . In the control literature, past works have addressed model-free reachable set estimation by proposing data-driven methods. However, existing methods do not apply to generative model outputs for three reasons: either they (i) apply to state and not output reachability ( Devonport et al. 2021 ; Devonport and Arcak 2020 ; Dietrich et al. 2024 ) ; (ii) impose assumptions, e.g., Lipschitzness, on system dynamics that are not verifiable for opaque LLMs and T2IMs ( Choi et al. 2025 ; Park et al. 2024 ; Xue et al. 2020 ) ; (iii) return continuous reachable set estimates while LLMs and T2IMs’ reachable sets are countable, making the estimates vacuously large ( Devonport and Arcak 2019 ; Sivaramakrishnan et al. 2025 ; Devonport and Arcak 2020 ; Dietrich et al. 2024 ) . We bridge these gaps by developing a sampling algorithm that estimates non-vacuous reachable and controllable sets for generative models, with probabilistic guarantees. This paves the way