NeuroGen : Neural Network Parameter Generation via Large Language Models

Jiaqi Wang Yusen Zhang Xi Li · 2025

Read NeuroGen : Neural Network Parameter Generation via Large Language Models by Jiaqi Wang Yusen Zhang Xi Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-12470.txt.

Introduction

Jiaqi Wang, Yusen Zhang, and Xi Li propose NeuroGen, which asks a language model to write a neural network's weights in this 2025 paper. Networks normally gain their parameters by slow, step-by-step fitting to data. The authors test a different route: generate the weights directly, guided by a description of the data, task, and network shape.

Stage one pretrains the language model on saved network checkpoints, so it learns how parameter space is organized. Stage two tunes it with rich, task-aware prompts for the target job. Experiments show the method produces usable network parameters. The authors present the result as a new paradigm in which language models and small networks work side by side.

Catalog record

CREATORJiaqi Wang Yusen Zhang Xi Li
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-12470

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

NeuroGen : Neural Network Parameter Generation via Large Language Models Jiaqi Wang Yusen Zhang Xi Li Abstract Acquiring the parameters of neural networks (NNs) has been one of the most important problems in machine learning since the inception of NNs. Traditional approaches, such as backpropagation and forward-only optimization, acquire parameters via iterative data fitting to gradually optimize them. This paper aims to explore the feasibility of a new direction: acquiring NN parameters via large language model generation. We propose NeuroGen , a generalized and easy-to-implement two-stage approach for NN parameter generation conditioned on descriptions of the data, task, and network architecture. Stage one is Parameter Reference Knowledge Injection, where LLMs are pretrained on NN checkpoints to build foundational understanding of parameter space, whereas stage two is Context-Enhanced Instruction Tuning, enabling LLMs to adapt to specific tasks through enriched, task-aware prompts. Experimental results demonstrate that NeuroGen effectively generates usable NN parameters. Our findings highlight the feasibility of LLM-based NN parameter generation and suggest a promising new paradigm where LLMs and lightweight NNs can coexist synergistically 1 1 1 The codes will be public after being accepted . 1 Introduction The acquisition of parameters in neural networks is primarily driven by gradient-based optimization. A typical approach to obtain a functional neural network involves forward and backward propagation over training data using gradient descent [ 1 ] . This classical framework has demonstrated remarkable effectiveness across various neural network architectures and domains, including but not limited to computer vision [ 2 , 3 ] , time-series analysis [ 4 ] , and natural language processing [ 5 , 6 ] . Recently, several studies utilize diffusion models to obtain neural network parameters [ 7 , 8 , 9 ] . Limited by the diffusion technique itself, most of the approachaes suffer from slow sampling speeds [ 10 ] and limited controllability [ 11 ] . Also, many of them generate parameter distributions rather than directly usable model parameters. In this paper, we explore the possibility and feasibility of a novel neural network parameter acquisition approach through LLM-based generation. . Many research work [ 12 , 13 , 14 ] have shown that the paramaters may carry the information of the data and there could be a hidden mapping between the parameters and data. As we know, LLMs have demonstrated remarkable capabilities in content understanding and generation across diverse tasks [ 15 , 16 ] , such as question answering, summarization, and image generation. Their ability to handle multimodal inputs and generate outputs via prompts makes them a promising candidate for parameter synthesis. Furthermore, advancements in parameter-efficient fine-tuning and instruction tuning have enabled LLMs to adapt to downstream tasks more effectively. Inspired by the work above, we are tring to create this new direction to utilize LLMs to generate nerual network parameters given data and instructions. This direction can also encourage potential research works to solve the real-world challenges, e.g., instrution-guided customized neural network acquisition and the scenario with limited training data. To make preliminary exploration, we propose NeuroGen —a novel and easy-to-implement framework for neural network parameter generation using LLMs. However, enabling LLMs to generate neural network parameters introduces several new challenges: (1) There are no existing LLMs pretrained with an understanding of neural network parameters, nor existing methodologies for using such parameters as inputs for fine-tuning; (2) It is non-trivial to prompt LLMs to generate nerual network parameters conditioned on data, task context and network architecture; To tackle these issues, our approach adopts a two-stage training strategy: stage 1: Parameter Referecne Knowledge injection and stage 2: Context-enhance Instruction Tuning. In Stage 1, we introduce neural network checkpoints and corresponding general instructions into pretrained LLMs to inject foundational knowledge of parameter structures. In Stage 2, we perform instruction tuning with task-specific data and enriched prompts that include information about the task, data, and neural networks. This context-aware training allows the model to learn flexible and adaptive generation strategies. Our main contributions are summarized as follows: • To the best of our knowledge, this is the first work to explore the feasibility of directly generating neural network parameters using LLMs, without gradient-based optimization. • We propose NeuroGen , a novel, easy-to-implement, and extensible framework. Using a two-stage strategy, i.e., Parameter Reference Knowledge Injection and Context-enhanced Instruction Tuning, we demonstrate that LLMs can generate functional neural network parameters. • Our study introduces a new perspective: neural network parameters may exhibit latent structures tied to training context, which can be learned and reproduced by LLMs. This opens a new direction for understanding and interpreting neural network parameters through the lens of language models. 2 Related Work 2.1 Neural Network Parameter Acquisition Typical model training paradigm based on backpropagation is commonly used approach and has demonstrated its success across different domains. Besides it, there are several other approaches about the nerual network acquisition. Ha et al. [ 17 ] propose using a hypernetwork—a smaller network that takes layer embedding vectors as input—to generate the parameters of a larger target network, referred to as the main network. Finn et al. [ 18 ] focus on meta-learning, where the goal is to train a model’s initial parameters so that it can quickly adapt to new tasks with just a few gradient updates using a small amount of data. More recently, several studies have explored using diffusion models to synthesize neural network parameters. Peebles et al. [ 9 ] construct a dataset of neural network checkpoints from multiple training runs, where each checkpoint includes model parameters along with metadata such as test loss and error for supervised tasks. Given an initial parameter vector and a target metric (e.g., loss or accuracy), their model learns to predict a distribution over updated parameter vectors for a fixed network architecture that achieves the target. Similarly, Wang et al. train diffusion models to learn the distribution of high-performing parameters [ 7 ] . They first use an autoencoder to map model parameters to a latent space, then train a diffusion model to generate these latent representations from random noise. Jin et al. [ 8 ] improve upon this by introducing a conditional latent diffusion model that synthesizes high-performing parameters based on specific task conditions. 2.2 Large Lauguage Model Large language models (LLMs) have demonstrated remarkable capabilities across a wide range of domains, including natural language processing [ 19 ] , computer vision [ 20 ] , bioinformatics [ 21 ] , and robotics [ 22 ] . They have been successfully applied to various tasks such as question answering [ 23 ] , reasoning [ 24 ] , and data generation [ 25 ] . In particular, substantial research has been devoted to LLM-driven image generation [ 26 ] , text generation [ 27 ] , and code generation [ 28 ] . Multimodal LLMs combine the advanced capabilities of LLMs with additional modalities such as images, audio, and video, enabling them to process information across multiple perceptual channels and mimic human multisensory cognition [ 29 ] . They typically incorporate modality-specific encoders (e.g., Vision Transformers for images) to fuse diverse inputs. This multi-modality enriches input representation and enhances both understanding and generation [ 30 , 31 , 32 ] , supporting applications such

Read the complete text