TeNet: Text-to-Network for Compact Policy Synthesis

Ariyan Bighashdel, Kevin Sebastian Luck · 2026

Read TeNet: Text-to-Network for Compact Policy Synthesis by Ariyan Bighashdel, Kevin Sebastian Luck (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-15912.txt.

Introduction

Ariyan Bighashdel and Kevin Sebastian Luck turn plain instructions into small robot controllers in a 2026 arXiv paper. Robots that follow language usually split into two camps. One plans at a high level through hand-designed interfaces. The other uses large end-to-end models that are hard to run in real time. Their framework, TeNet, takes a middle path. It builds compact, task-specific policies directly from text descriptions.

Text-to-Network is the name's expansion. A network here is a small control policy, not a giant model. Compact policies can run fast on robot hardware. Generating them per task avoids hand-coding each new job. For deployment, small and specific often beats large and general. TeNet aims to give language-driven robots that practical edge.

Catalog record

CREATORAriyan Bighashdel, Kevin Sebastian Luck
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-15912

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

TeNet: Text-to-Network for Compact Policy Synthesis Ariyan Bighashdel, Kevin Sebastian Luck Abstract Robots that follow natural-language instructions often either plan at a high level using hand-designed interfaces or rely on large end-to-end models that are difficult to deploy for real-time control. We propose TeNet (Text-to-Network), a framework for instantiating compact, task-specific robot policies directly from natural language descriptions. TeNet conditions a hypernetwork on text embeddings produced by a pretrained large language model (LLM) to generate a fully executable policy, which then operates solely on low-dimensional state inputs at high control frequencies. By using the language only once at the policy instantiation time, TeNet inherits the general knowledge and paraphrasing robustness of pretrained LLMs while remaining lightweight and efficient at execution time. To improve generalization, we optionally ground language in behavior during training by aligning text embeddings with demonstrated actions, while requiring no demonstrations at inference time. Experiments on MuJoCo and Meta-World benchmarks show that TeNet produces policies that are orders of magnitude smaller than sequence-based baselines, while achieving strong performance in both multi-task and meta-learning settings and supporting high-frequency control. These results show that text-conditioned hypernetworks offer a practical way to build compact, language-driven controllers for ressource-constrained robot control tasks with real-time requirements. 1 Introduction Recent advances in large language models (LLMs), such as GPT 3 and LLaMA 26 , have demonstrated that natural language can act as a powerful and flexible interface across a wide range of domains. In robotics, this has led to growing interest in language-conditioned control, where robots are guided by natural-language instructions, often alongside perceptual inputs. Prominent examples include vision–language–action (VLA) systems such as PaLM-E 8 , SayCan 2 , RT-2 33 , OpenVLA 14 , and OCTO 24 . These systems highlight how expressive language can be for specifying complex robotic behaviors. However, this expressiveness often comes at the cost of scale. Many recent approaches to language-conditioned control rely on large end-to-end architectures, which can be computationally expensive and difficult to deploy in high-frequency control loops or on robots with limited onboard compute. At the other end of the spectrum are compact sequence models such as Decision Transformers (DT) 5 and Prompt-DT 30 , which prioritize efficiency and ease of deployment in offline reinforcement learning. However, these models do not naturally incorporate language: instead, they rely on trajectory prompts or task-specific demonstrations to distinguish tasks, often requiring demonstrations even at test time and degrading as task diversity increases. This leaves a clear gap between expressive language-conditioned systems and compact policies that are efficient but are not language-enabled. Several works attempt to bridge this gap by using language indirectly. Code-as-Policies 15 translates instructions into robot API calls, while Code-as-Rewards 27 maps task descriptions into reward functions for reinforcement learning. Although effective in specific settings, these methods depend on predefined interfaces or accurate simulators, limiting their applicability to real-world robotics. In this work, we ask a simpler question: can language itself be used as a direct conditioning signal for policy instantiation? Rather than executing a large language model inside the control loop, we use it once—at policy instantiation—through a hypernetwork 10 . We introduce TeNet (Text-to-Network) , a framework that conditions a hypernetwork on LLM-derived text embeddings to generate compact, task-specific policies. The resulting controller operates solely on low-dimensional state inputs, requires no demonstrations at inference time, and can run at high control frequencies on resource-constrained robots. While direct text-to-policy instantiation is effective, we find that performance improves when language is grounded in behavior. By aligning language representations with expert trajectories during training, task descriptions capture not only linguistic intent but also behavioral semantics, leading to stronger generalization in multi-task and meta-learning settings. Importantly, grounding is used only during training: at inference, policies are instantiated from text alone. This paper investigates a first approach into the possibility to utilze large robotic foundation models for resource-constrained robots via language-enabled hypernetworks for compact policy synthesis . While this could be applied to VLAs in the future, our first steps focus on low-dimensional, trajectory-based domains (Mujoco and Meta-World), isolating the role of language in policy instantiation without addressing perception. In summary, our contributions are: • Text-to-Network Policy Generation. We introduce TeNet, a framework that conditions a hypernetwork on LLM text embeddings to synthesize compact, task-specific robot policies suitable for real-time deployment. • Grounding Language in Behavior. We show that aligning language with expert trajectories during training enriches linguistic representations with behavioral semantics and improves generalization in multi-task and meta-learning settings. • Empirical Insights into a New Paradigm. We provide an extensive study on Mujoco and Meta-World benchmarks, highlighting both the promise and limitations of language-enabled hypernetworks and outlining paths toward future vision-grounded extensions. 2 Related Work Language-Conditioned Control in Robotics. Large language models (LLMs) have recently been integrated into robotic systems to enable natural language instruction following and high-level planning. Early approaches such as SayCan 2 and PaLM-E 8 leverage pretrained LLMs to map language into symbolic plans or action primitives executed by low-level controllers. These methods exploit LLMs’ world knowledge but typically operate at a goal or planning level rather than synthesizing executable control policies. Other works connect language and control indirectly. Code-as-Policies 15 translates instructions into robot API calls, while Code-as-Rewards 27 maps task descriptions into reward functions for reinforcement learning. SayTap 23 similarly maps language into structured locomotion patterns. While effective in constrained settings, these approaches rely on predefined interfaces or accurate simulators, limiting their generality. More recent vision-language-action systems, such as RT-2 33 , OpenVLA 14 , and OCTO 24 , integrate language and perception in large end-to-end models, but their computational demands hinder deployment in resource-constrained or high-frequency control settings. Several works also explore grounding language in behavior through representation learning. For example, CLASP 20 learns joint language–state–action embeddings via contrastive objectives, focusing on representation pretraining rather than policy synthesis. In contrast, our use of alignment is auxiliary: language grounding serves to stabilize text-conditioned policy generation rather than constituting the primary modeling objective. Compact Sequence Models for Policy Learning. A separate line of research explores compact sequence models as policies for reinforcement learning. The Decision Transformer (DT) 5 formulates offline RL as conditional sequence modeling, generating actions autoregressively given states and return-to-go. While effective in single-task settings, DT lacks an explicit mechanism for task identification and therefore struggles in multi-task or meta-learning regimes. Extensions such as Prompt-DT 30 and Meta-DT 28 introduce task-conditioning via trajectory prompts, improving generalization at the cost of requiring demonstrations at inference time. Diffusion-based approaches, including MTDiff 11 and MetaDiffuser 19 , similarly condition on trajectories or task contexts to generalize across tasks. Although these methods demonstrate strong performance, their reliance on prompt trajectories limits scalability when demonstrations are unavailable or expensive. In parallel, visuomotor diffusion policies such as Diffusion Policy 6 generate actions directly from images and have shown impressive real-world results. These approaches differ fundamentally from the low-dimensional, state-based settings we consider. We therefore focus on DT-based baselines to maintain architectural comparability and isolate the role of language-conditioned policy instantiation. Overall, compact sequence models demonstrate that lightweight architectures can scale to multi-task RL, but their dependence on trajectory prompts and lack of direct language grounding constrain their applicability as instruction-following agents. Hypernetworks and Policy Generation. Hypernetworks 10 generate

Read the complete text