Decoupled Behavioral Cloning for Scalable Inductive Generalization in RL from Specifications

Vignesh Subramanian, Subhajit Roy, Suguman Bansal, [0.5em] School of Computer Science, Georgia Institute of Technology, USA, [0.5em] {vignesh,suguman}@gatech.edu, [email protected] · 2026

Read Decoupled Behavioral Cloning for Scalable Inductive Generalization in RL from Specifications by Vignesh Subramanian, Subhajit Roy, Suguman Bansal, [0.5em] School of Computer Science, Georgia Institute of Technology, USA, [0.5em] {vignesh,suguman}@gatech.edu, [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-00838.txt.

Introduction

Subramanian, Roy, and Bansal split a hard learning job in two in this 2026 paper. Their field is RL from specifications, where a family of tasks shares an inductive shape. In principle, related tasks should have related policies. Past work learns a policy-evolution rule by RL itself. As tasks grow, mixed reward signals clash and training turns unstable.

Their method, DIBS, decouples the two jobs. First, standard RL trains a teacher policy for each task on its own. Then behavioral cloning fits the evolution rule on state-action pairs the teachers label. Dense, calm supervision replaces noisy pooled rewards. The authors report large gains in training stability from this swap. Zero-shot generalization to new tasks in the family also improves against RL and meta-RL baselines. The lesson is plain: clone first-class teachers, and let the shared rule learn from their acts, not from raw reward noise.

Catalog record

CREATORVignesh Subramanian, Subhajit Roy, Suguman Bansal, [0.5em] School of Computer Science, Georgia Institute of Technology, USA, [0.5em] {vignesh,suguman}@gatech.edu, [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.00838

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Decoupled Behavioral Cloning for Scalable Inductive Generalization in RL from Specifications Vignesh Subramanian, Subhajit Roy, Suguman Bansal, [0.5em] School of Computer Science, Georgia Institute of Technology, USA, [0.5em] {vignesh,suguman}@gatech.edu, [email protected] Abstract Inductive generalization is a framework for reinforcement learning (RL) generalization in which inductively related task instances admit inductively related policies. Prior work captures this structure via a higher-order policy-evolution function learned directly with RL, but suffers from poor training scalability : as training tasks grow, aggregated reward feedback becomes noisy and conflicting, destabilizing training and weakening generalization. We propose DIBS , a decoupled behavioral cloning approach that separates learning task-specific policies from learning the evolution function. We first learn individual teacher policies per task via standard RL, then fit the evolution function via behavioral cloning on teacher-labeled state-action pairs. This replaces noisy reward aggregation with dense, stable supervision. DIBS achieves significant improvements in both training stability and zero-shot generalization against existing RL and meta-RL algorithms. 1 Introduction The generalization problem in reinforcement learning (RL) [ 33 ] asks how agents can learn policies that transfer to new, unseen situations [ 20 , 8 , 41 , 7 ] . Despite remarkable successes, generalization remains a fundamental challenge. RL agents that perform well on individual tasks often fail on even slightly modified scenarios. Inductive generalization ( GenRL ) [ 32 ] is a notion of generalization that leverages inductive similarities between tasks. The core intuition is that inductively related tasks must have inductively related policies. Inductive relationships arise naturally whenever tasks exhibit recursion or iteration: once we understand how to transform the policy for the i-th task to the i+1 task by training on a few tasks, we can generalize to arbitrarily many tasks. This structure is evident in robotics and control, where physical constraints impose regular, repeating patterns. Prior work formalizes this intuition by encoding inductive families of tasks using temporal logic specifications , where tasks share identical logical structure and differ only in the instantiation of low-level predicates. (a) Reacher benchmark environment (b) Inductive task family in the Reacher benchmark (c) GenRL flow diagram (d) DIBS flow diagram Figure 1 : Left column: Benchmark illustrations. (a) Reacher benchmark environment. (b) Inductive task family in the Reacher benchmark. Right column: Contrasting flow diagrams. (c) GenRL flow diagram. (d) DIBS flow diagram. Consider the robotic arm in Fig. 1 . It has to displace a tower of blocks from a source to a target. This complex task decomposes into a series of inductively related subtasks. In the i -th subtask, the robot relocates the topmost block from the source at height i to the top of the target at height (h-i) , where h is the initial height of the source. The 0 -th task is the base task, and successive tasks are obtained by updates to the locations of the topmost blocks in the source and target. The inductive generalization algorithm GenRL captures this structure by learning a higher-order policy-evolution function \Omega:\Pi\to\Pi such that \pi_{i+1}=\Omega(\pi_{i}) , approximated as an m -degree polynomial over policy parameters. Learning reduces to inferring a small set of \kappa -coefficients via an RL-style training loop over training tasks (Fig. 1.(c) ). Despite stronger generalization than several state-of-the-art methods, GenRL suffers from a fundamental training scalability bottleneck: policy learning for training tasks is tightly coupled with \kappa -coefficient training. As the number of training tasks grows, aggregated reward feedback becomes noisy and conflicting, making the training loop increasingly difficult to stabilize, ultimately constraining its generalization capability. We propose DIBS , a decoupled behavioral cloning approach that resolves this bottleneck by separating two phases that GenRL conflates (Fig. 1.(d) ). In Stage I , we independently learn teacher policies for each training task using standard RL. In Stage II , we fit the \kappa -template via behavioral cloning on teacher-labeled state-action datasets [ 36 , 21 , 29 ] , replacing noisy multi-task reward aggregation with stable, dense supervision. Since teacher policies are fixed during \kappa -training, the destabilizing feedback loop in GenRL is eliminated. A further advantage is that teacher policies can be learned using any RL algorithm, unlike GenRL , which is tightly coupled to ARS [ 26 ] , fundamentally limiting the environments it can handle. This decoupling raises non-trivial challenges. One of them is that independently trained teachers can drift arbitrarily in parameter space across task indices, making it difficult for a single \kappa to fit all of them simultaneously. We address this through cross-index regularization during teacher training. In summary, our contributions are: • Algorithm. We propose DIBS , a decoupled behavioral cloning approach to inductive generalization that replaces GenRL ’s unstable multi-task RL loop with stable supervised learning (Section 4 ). • Techniques. We introduce cross-index regularization and confidence-based dataset filtering to address the challenges that decoupling introduces. • Empirics. We conduct our evaluation on challenging continuous environments spanning increasing dynamical complexity, from simple planar motion ( Car2D ) to higher-dimensional 3D navigation and attitude control ( SimpleDrone , DroneAttitude ), and a manipulation-style benchmark ( Reacher ). DIBS achieves 3.82\times better training scalability and 6.19\times better zero-shot generalization over GenRL . We also compare DIBS against four other RL and meta-RL baselines; DIBS achieves 3.02 \times and 3.04 \times better training scalability and zero-shot generalization (resp.) against the closest of these baselines (Section 5 ). Related Work. Prior work tackles generalization in reinforcement learning through several complementary ideas. Meta-learning [ 43 , 12 , 16 ] prepares an agent for rapid adaptation by learning an initialization or latent task representation that can be updated with a small amount of new experience; unlike GenRL and DIBS , meta-learning approaches do not perform zero-shot generalization. Goal-conditioned RL [ 31 , 27 , 23 ] trains a single policy conditioned on a desired outcome, enabling zero-shot generalization by changing the goal input. Multi-task RL [ 30 , 28 , 35 , 11 ] learns shared representations or a shared policy across a fixed training distribution, encouraging skill reuse. Programmatic and logic-based policy representations [ 4 , 37 , 42 , 6 ] impose symbolic structure on the policy class, improving systematic reuse and interpretability relative to purely neural policies. While all these lines of work enable generalization by adapting to a new task, conditioning on a goal, or sharing representations across a fixed training set, none directly addresses the setting we study: an indexed inductive family where task instances vary systematically with the index, and the objective is to learn a single rule that predicts how the policy should change as the index changes. GenRL [ 32 ] is, to our knowledge, the only prior method that explicitly targets this form of inductive generalization. The use of temporal logics in RL has enabled long-horizon learning across a range of settings [ 10 , 1 , 5 , 9 , 15 , 14 , 40 , 38 , 22 , 19 , 2 , 39 , 24 , 34 , 13 , 25 , 3 ] ; temporal logics are useful for inductive generalization as they provide structured syntax to capture task similarities. 2 Preliminaries Markov Decision Process (MDP). The environment in RL is modeled as a Markov Decision Process (MDP) \mathcal{M}=(S,A,P,\eta) ,

Read the complete text