LANTERN: LLM-Augmented Neurosymbolic Transfer with Experience-Gated Reasoning Networks
Mahyar Alinejad, Yue Wang, Amrit Singh Bedi, George Atia · 2026
Read LANTERN: LLM-Augmented Neurosymbolic Transfer with Experience-Gated Reasoning Networks by Mahyar Alinejad, Yue Wang, Amrit Singh Bedi, George Atia (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-05478.txt.
Introduction
Learning a new task goes faster when an agent can borrow skills from old ones. Mahyar Alinejad, Yue Wang, Amrit Singh Bedi, and George Atia present LANTERN, a transfer framework for reinforcement learning with a symbolic side. Earlier methods needed hand-written task machines, one source task, and fixed rules for mixing knowledge.
LANTERN generates task automata from plain-language descriptions using a language model. It blends several source policies by semantic similarity, and it gates teaching by error and uncertainty signals. Across resource, navigation, and control tasks, sample efficiency rose forty to sixty percent over baselines, and poorly matched sources did little harm.
Catalog record
| CREATOR | Mahyar Alinejad, Yue Wang, Amrit Singh Bedi, George Atia |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.05478
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LANTERN: LLM-Augmented Neurosymbolic Transfer with Experience-Gated Reasoning Networks Mahyar Alinejad, Yue Wang, Amrit Singh Bedi, George Atia Abstract Transfer learning in reinforcement learning (RL) seeks to accelerate learning in new tasks by leveraging knowledge from related sources. Existing neurosymbolic transfer methods, however, typically rely on manually specified task automata, assume a single source task, and use fixed knowledge-integration mechanisms that cannot adapt to varying source relevance. We propose LANTERN, a unified framework for multi-source neurosymbolic transfer that addresses these limitations through three components: (i) deterministic finite automata generated from natural language task descriptions using large language models, (ii) semantic embedding-based aggregation of multiple source policies weighted by cross-task similarity, and (iii) adaptive teacher-student gating based on temporal-difference error and semantic uncertainty. Across domains spanning resource management, navigation, and control, LANTERN achieves 40–60% improvements in sample efficiency over existing baselines while remaining robust to poorly aligned sources. These results demonstrate that multi-source, adaptively weighted neurosymbolic transfer can improve scalability and robustness in symbolic RL settings. 1 1,2 2 1,2 keywords Reinforcement Learning, Transfer Learning, Neurosymbolic AI, Large Language Models, Automata Learning 1 Introduction Reinforcement learning (RL) has achieved strong empirical performance in game playing ( 25 ; 31 ) , robotics ( 22 ; 23 ) , and autonomous systems ( 21 ) . However, effective policy learning often requires extensive interaction, limiting applicability when data collection is costly or unsafe ( 13 ) . Transfer learning mitigates this by leveraging knowledge from related tasks ( 35 ; 40 ) , yet existing approaches remain challenged by structured, long-horizon objectives that are naturally non-Markovian. Neurosymbolic RL integrates symbolic task representations (such as deterministic finite automata (DFAs) or reward machines ( 18 ; 19 ) ) into learning. By encoding temporal structure through product MDP constructions ( 7 ) , these methods improve sample efficiency for complex tasks. Despite these advances, several limitations remain. 1) Manual specification: Most methods assume expert-provided DFAs or temporal logic formulas ( 24 ; 10 ; 16 ) . Grammatical inference approaches can recover automata from demonstrations ( 6 ; 27 ; 2 ; 4 ) , but they require structured trajectory data and are difficult to apply in sparse or exploratory RL settings. 2) Single-source transfer: Automaton distillation transfers symbolic guidance via DFA transitions ( 32 ; 1 ) , while policy distillation provides action-level knowledge ( 30 ) . CADENT combines both using experience-based gating ( 3 ) . However, these approaches rely on a single source task, which can limit effectiveness when source-target alignment varies. 3) Fixed integration mechanisms: Existing methods typically employ predetermined weighting schemes (e.g., exponential decay ( 32 ) ) or static hyperparameters, limiting adaptability when source relevance changes across states or over time. Key insight and technical novelty. We consider neurosymbolic transfer in a setting where multiple source tasks may have partially related but distinct goals from the target. In this regime, transfer cannot rely on direct reuse of a single source policy or automaton; instead, it requires semantic alignment and aggregation of structured knowledge across heterogeneous tasks. To address this setting, we introduce LANTERN ( L LM- A ugmented N eurosymbolic T ransfer with E xperience-gated R easoning N etworks). LANTERN integrates three components. First, DFAs are generated from natural language task descriptions using large language models (LLMs), eliminating manual specification. Second, we construct a shared embedding space over automaton state descriptions, enabling aggregation of partial knowledge from multiple source tasks with heterogeneous goals. Third, we introduce a dual-volatility gating mechanism that combines semantic alignment (measured via embedding similarity) with experience-based reliability (measured via TD error), allowing adaptive weighting of teacher influence during learning. Contributions. Our contributions are threefold: 1) We formulate multi-source neurosymbolic transfer in a setting where source tasks may have heterogeneous goals, requiring semantic aggregation rather than direct reuse of a single source policy or automaton. 2) We develop LANTERN, which integrates LLM-based automaton generation, semantic multi-source aggregation, and adaptive trust gating within a single neurosymbolic transfer architecture. 3) Across diverse domains, we demonstrate 40–60% improvements in sample efficiency over single-source and static-integration baselines, while maintaining robustness to poorly aligned sources. 1.1 Related Work Transfer learning in RL. Classical transfer methods include value function reuse ( 34 ) , policy distillation ( 30 ; 12 ) , and successor features ( 8 ; 9 ) . Meta-learning ( 14 ; 29 ) and multi-task learning ( 28 ; 36 ) share representations across related tasks. These approaches typically assume Markovian reward structures and do not explicitly model temporal logic or automaton-based task decomposition. Neurosymbolic RL. To address non-Markovian objectives, reward machines ( 18 ; 19 ) and temporal logic specifications ( 24 ; 10 ; 16 ) encode structured task progression via product MDP constructions. Extensions integrate automata with deep RL ( 17 ; 15 ) or infer specifications from demonstrations ( 37 ) . However, these works focus primarily on single-task learning rather than transfer across heterogeneous tasks. Automaton-based transfer. Recent work leverages automaton structure for transfer. Automaton distillation ( 32 ) transfers high-level task decomposition through DFA-guided Q-value aggregation. Bidirectional transfer frameworks ( 1 ) enable mutual knowledge exchange, while CADENT ( 3 ) combines strategic automaton guidance with tactical policy distillation using experience-based gating. ARM-FM ( 11 ) employs LLM-generated reward machines for transfer. These approaches, however, rely on single-source settings and fixed or experience-only integration mechanisms. LLMs in RL. LLMs have been used to provide planning guidance ( 20 ) , programmatic policy representations ( 38 ; 5 ) , and zero-shot generalization signals ( 26 ) . In contrast to approaches that use language primarily for prompting or reward shaping, LANTERN generates formal DFAs compatible with product MDP constructions and integrates them into a multi-source neurosymbolic transfer framework. The remainder of this paper is structured as follows: Section 2 provides necessary background on product MDPs and transfer learning. Section 3 details the LANTERN framework. Section 4 reports experimental results across four domains. Section 5 concludes with future directions. 2 Background 2.1 Markov Decision Processes and Q-Learning A Markov Decision Process (MDP) ( 33 ) is a tuple \mathcal{M}=\langle\mathcal{S},\mathcal{A},\mathcal{T},\mathcal{R},\gamma\rangle , where \mathcal{S} is the state space, \mathcal{A} is the action space, \mathcal{T}:\mathcal{S}\times\mathcal{A}\times\mathcal{S}\to[0,1] is the transition function, \mathcal{R}:\mathcal{S}\times\mathcal{A}\to\mathbb{R} is the reward function, and \gamma\in[0,1) is the discount factor. A policy \pi:\mathcal{S}\to\Delta(\mathcal{A}) maps states to action distributions, where \Delta(\mathcal{A}) is the probability simplex on \mathcal{A} . The goal is to find \pi^{*}=\arg\max_{\pi}\mathbb{E}_{\pi}[\sum_{t=0}^{\infty}\gamma^{t}\mathcal{R}(s_{t},a_{t})\mid s_{0}] . The optimal action-value function Q^{*}(s,a)=\max_{\pi}Q^{\pi}(s,a) satisfies: Q^{*}(s,a)=\mathbb{E}_{s^{\prime}}\left[\mathcal{R}(s,a)+\gamma\max_{a^{\prime}}Q^{*}(s^{\prime},a^{\prime})\right]. (1) Q-learning ( 39 ) iteratively estimates Q^{*} via: Q(s_{t},a_{t})\leftarrow Q(s_{t},a_{t})+\alpha\left[\mathcal{R}(s_{t},a_{t})+\gamma\max_{a^{\prime}}Q(s_{t+1},a^{\prime})-Q(s_{t},a_{t})\right], (2) where \alpha\in(0,1] is the learning rate. 2.2 Product MDPs for Non-Markovian Objectives Many tasks involve non-Markovian objectives depending on state history ( 7 ) . A DFA \mathcal{D}=\langle\Omega,\Sigma,\delta,\omega_{0},F\rangle specifies such tasks, where \Omega is the set of automaton states, \Sigma is the set of labels, \delta:\Omega\times\Sigma\to\Omega is the transition function, \omega_{0} is the initial state, and F\subseteq\Omega are accepting states. A labeling function L:\mathcal{S}\to\Sigma maps MDP states to labels. The product MDP \mathcal{M}\times\mathcal{D}=\langle\mathcal{S}\times\Omega,\mathcal{A},\mathcal{T}^{\prime},\mathcal{R}^{\prime},\gamma\rangle has state space \mathcal{S}\times\Omega with (s,\omega) representing the agent at MDP state s with the automaton in state \omega . The transition function is \mathcal{T}^{\prime}((s,\omega),a,(s^{\prime},\omega^{\prime}))=\mathcal{T}(s,a,s^{\prime}) if \omega^{\prime}=\delta(\omega,L(s^{\prime})) , and 0 otherwise. The reward function \mathcal{R}^{\prime}((s,\omega),a) is designed based