Closed-Loop LLM Discovery of Non-Standard Channel Priors in Vision Models
Tolgay Atinc Uzun, Dmitry Ignatov, and Radu Timofte · 2026
Read Closed-Loop LLM Discovery of Non-Standard Channel Priors in Vision Models by Tolgay Atinc Uzun, Dmitry Ignatov, and Radu Timofte (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-08517.txt.
Introduction
Tolgay Atinc Uzun, Dmitry Ignatov, and Radu Timofte use language models to discover channel configurations in vision models. Their 2026 arXiv paper treats channel search as a hard combinatorial problem. Layer widths must respect tensor shape compatibility and computational budgets at once. The authors argue language models can reason about architectural code structure in ways heuristics cannot. A closed-loop search lets the model propose, test, and revise non-standard channel priors. Smarter architecture search could yield leaner vision networks for phones and cameras.
Catalog record
| CREATOR | Tolgay Atinc Uzun, Dmitry Ignatov, and Radu Timofte |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-08517
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Closed-Loop LLM Discovery of Non-Standard Channel Priors in Vision Models Tolgay Atinc Uzun, Dmitry Ignatov, and Radu Timofte Abstract Channel configuration search—the optimization of layer specifications such as layer widths in deep neural networks—presents a complex combinatorial challenge constrained by tensor shape compatibility and computational budgets. We posit that Large Language Models (LLMs) offer a transformative approach to Neural Architecture Search (NAS), capable of reasoning about architectural code structure in ways that traditional heuristics cannot. In this paper, we investigate the application of an LLM-driven NAS framework to the problem of channel configuration. We formulate the search as a sequence of conditional code generation tasks, where an LLM refines architectural specifications based on performance telemetry. Crucially, we address the data scarcity problem by generating a vast corpus of valid, shape-consistent architectures via Abstract Syntax Tree (AST) mutations. While these mutated networks are not necessarily high-performing, they provide the critical volume of structural data required for the LLM to learn the latent relationship between channel configurations and model performance. This allows the LLM to internalize complex design patterns and apply them to optimize feature extraction strategies. Experimental results on CIFAR-100 validate the efficacy of this approach, demonstrating that the model yields statistically significant improvements in accuracy. Our analysis confirms that the LLM successfully acquires domain-specific architectural priors, distinguishing this method from random search and highlighting the immense potential of language-driven design in deep learning. 1 Introduction Neural architecture design is typically formulated as a discrete optimization problem and solved using reinforcement learning [ 1 ] , evolutionary algorithms, or differentiable relaxations [ 2 ] . These methods generally treat the model definition as a fixed graph and do not utilize the semantic structure of the source code. LLMs provide an alternative method for NAS. Inspired by recent advancements in LLM applications across diverse domains [ 3 , 4 , 5 , 6 ] , we leverage pre-trained knowledge of code syntax and logical flow to approach architectural refinement as an iterative programmatic optimization task rather than a set of discrete graph operations. A primary challenge in applying LLMs to NAS is the lack of domain-specific training data. Standard repositories do not contain sufficient variations of executable architectures to train a model on structural design. This study addresses this limitation through a programmatic bootstrapping mechanism using Abstract Syntax Tree (AST) manipulation. This technique generates a large corpus of syntactically valid and tensor-consistent network variants to resolve the initialization problem common in learning-based search methods. This research focuses on the optimization of channel configurations. Modifying layer widths requires maintaining consistency across coupled network components such as residual connections. Training on the programmatically generated dataset enables the model to learn the syntax required for execution and the relationship between channel dimensions and performance. The model therefore functions as an optimizer that iteratively refines architectures to improve accuracy. 2 Related Work 2.1 Neural Architecture Search Conventional strategies for NAS including reinforcement learning [ 1 ] and differentiable architecture search [ 2 ] have automated the engineering of model structures. These methods typically require the prior definition of restrictive supernets or search spaces. Recent research has investigated the use of generative models for architecture definition. However, these approaches face challenges regarding the discrete and constrained nature of valid neural network code. Reinforcement learning methods formulate network generation as a sequential decision process operating on a predefined vocabulary of layers and connections. This formulation restricts the search to a known subspace. The proposed method treats architecture definition as open-ended code modification at the token level. This approach enables flexible exploration of the design space and facilitates the discovery of architectural patterns not explicitly encoded in the search space definition. 2.2 Automated Channel Pruning and Model Compression The specific problem of optimizing channel configurations has a rich history in model compression. He et al. [ 28 ] propose using a deep deterministic policy gradient reinforcement learning agent to sequentially sample the compression ratio for each layer of a pre-trained network. The agent learns to balance accuracy against hardware constraints (FLOPs, latency) by receiving the fine-tuned accuracy as a reward signal. While the prior approach treats channel counts as continuous numerical actions within a reinforcement learning environment, our solution treats them as tokens in a code sequence, allowing the LLM to leverage syntactic patterns and programming logic rather than just numerical rewards. Network slimming [ 29 ] introduces a differentiable approach to pruning by enforcing L1 regularization on the scaling factors of batch normalization layers. During training, the model automatically identifies insignificant channels, those with near-zero scaling factors, which can then be safely pruned without significant accuracy loss. Unlike network slimming, which relies on differentiable scaling factors optimized during training, our method optimizes the architecture definition itself before training begins, using the LLM’s code-generation capabilities to propose efficient configurations directly in source code. MetaPruning [ 30 ] trains a separate hypernetwork to predict the weights for various pruned networks, enabling the evaluation of many different channel configurations without retraining each one from scratch. While MetaPruning uses a meta-network to predict weights for numerical channel values, our work uses an LLM to predict the code that defines these channels, framing the problem as language modeling rather than weight prediction. 2.3 LLM-Driven Optimization and Discovery The intersection of LLMs and evolutionary search is a rapidly emerging field. EvoPrompting [ 15 ] utilizes LLMs as mutation operators within an evolutionary algorithm. Instead of relying on random bit-flipping or subtree exchange, EvoPrompting prompts the LLM to combine the logic of parent codes or optimize specific functions, effectively treating the LLM as an intelligent genetic operator. Similarly, FunSearch [ 17 ] demonstrates that LLMs can discover novel mathematical algorithms by iteratively evolving code within a sandbox evaluator. FunSearch employs evolutionary model where multiple independent populations evolve in parallel, feeding back only the highest-scoring, executable programs into the prompt. This creates a self-reinforcing loop that has led to discoveries surpassing human baselines in domains like bin packing. FunBO [ 26 ] extends this concept to Bayesian optimization, using LLMs to discover novel acquisition functions that generalize better across tasks than standard heuristics. Other approaches like LLM-NAS [ 10 ] and GPT-NAS [ 19 ] explore using LLMs to directly propose layers or connections. GPT-NAS, for instance, encodes neural architectures into a specialized token sequence and fine-tunes a transformer to predict high-performing strings. LeMo-NADe [ 27 ] focuses on multi-objective discovery for edge devices, using an expert system to validate constraints like latency and power consumption. A common limitation in prompt-only approaches is the reliance on the LLM’s pre-trained knowledge to produce valid code. When applied to complex constraints like channel consistency in residual networks, these methods frequently generate invalid code, wasting evaluation budget. Our approach differs by explicitly fine-tuning the LLM on a programmatically generated, task guided corpus of valid codes, directing the LLM during generation phase. 2.4 Code Generation and Validation Ensuring the validity of generated code is a central challenge in LLM-based search. LLMatic [ 18 ] addresses this by combining LLMs with quality-diversity optimization, specifically the MAP-Elites algorithm. Instead of converging to a single best model, LLMatic maintains an archive of high-performing models spread across a feature map (e.g., model size vs.