GIFT: Generalizing Intent for Flexible Test-Time Rewards

Fin Amin, Nathaniel Dennler, Andreea Bobu · 2026

Read GIFT: Generalizing Intent for Flexible Test-Time Rewards by Fin Amin, Nathaniel Dennler, Andreea Bobu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-22574.txt.

Introduction

Amin and co-authors present GIFT, a framework that helps robot reward functions travel to new settings. Rewards learned from demonstrations often latch onto surface quirks and fail elsewhere. GIFT instead asks a language model to infer the person's real intent, by contrasting liked and disliked behavior. At test time it matches new states to training states that mean the same thing under that intent. On tabletop tasks with unseen objects, GIFT beats similarity-based rivals. It also transfers to a real robot arm.

Catalog record

CREATORFin Amin, Nathaniel Dennler, Andreea Bobu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60037
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.22574

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

GIFT: Generalizing Intent for Flexible Test-Time Rewards Fin Amin, Nathaniel Dennler, Andreea Bobu Abstract Robots learn reward functions from user demonstrations, but these rewards often fail to generalize to new environments. This failure occurs because learned rewards latch onto spurious correlations in training data rather than the underlying human intent that demonstrations represent. Existing methods leverage visual or semantic similarity to improve robustness, yet these surface-level cues often diverge from what humans actually care about. We present Generalizing Intent for Flexible Test-Time rewards (GIFT), a framework that grounds reward generalization in human intent rather than surface cues. GIFT leverages language models to infer high-level intent from user demonstrations by contrasting preferred with non-preferred behaviors. At deployment, GIFT maps novel test states to behaviorally equivalent training states via intent-conditioned similarity , enabling learned rewards to generalize across distribution shifts without retraining. We evaluate GIFT on tabletop manipulation tasks with new objects and layouts. Across four simulated tasks with over 50 unseen objects, GIFT consistently outperforms visual and semantic similarity baselines in test-time pairwise win rate and state-alignment F1 score. Real-world experiments on a 7-DoF Franka Panda robot demonstrate that GIFT reliably transfers to physical settings. Further discussion can be found at https://mit-clear-lab.github.io/GIFT/ I Introduction Imagine a human teaching a robot how to pack their bag for an art class (Fig. 1 ). The human might demonstrate placing a paintbrush and sketchbook in the bag, expecting the robot to understand their goal of gathering art supplies. Later, when preparing for a different art session, the human would naturally trust the robot to assist them autonomously with packing the right supplies—whether it’s the same paintbrush and sketchbook, or instead molding clay, an easel, or any other materials needed for the task. Unfortunately, while robots can learn reward functions from user demonstrations [ 19 , 22 ] , these reward functions rarely generalize to new scenarios. Instead, learned rewards often latch onto spurious correlations in the training data [ 2 ] (e.g., reward functions learn to “pack paintbrush”) rather than capturing the underlying high-level intent (e.g., a reward function to “pack art supplies”). As a result, when robots encounter new scenarios at test time, the learned reward may fail catastrophically, causing the robot to either ignore important task aspects or fixate on irrelevant ones. Prior work has attempted to improve robustness through data augmentation based on visual [ 11 ] or language [ 12 ] similarity, but these measures often diverge from what humans actually care about when defining task success [ 6 ] . Fig. 1: Top. During training, the robot gets task demonstrations (loading a paintbrush) and uses them to infer the user’s high-level intent (“load art supplies”). Bottom. At test time, the robot encounters three unseen objects. GIFT uses the inferred intent to correctly identify that the molding clay is relevant. In contrast, visual-similarity baselines incorrectly prefer the dish scrubber due to its appearance, and language-similarity baselines make an analogous mistake (“ tooth brush ” and “ paint brush. ”) Our key insight is that state similarity is driven by people’s high-level task intent, not by surface-level visual or language cues . In our example in Fig. 1 , a vision-based model might judge a scrubber to be most similar to a paintbrush because they look alike, while a language model might instead group a paintbrush with a toothbrush due to lexical similarity. Neither captures what actually matters for the task: under the intent “pack art supplies,” a paintbrush is closer to molding clay. In addition, different intents induce different notions of similarity: if the intent were instead “set aside painting supplies,” then paintbrush, watercolor, and easel would be similar to the intention, while molding clay and sketchbook would be dissimilar. Without conditioning on intent, the robot cannot know which notion of similarity is appropriate, and thus risks generalizing along the wrong notion of similarity. Fig. 2: G eneralizing I ntent for F lexible T est-Time rewards. Left. GIFT infers the human’s intent given pairs of human-preferred demonstrations and reference trajectories. Right. During inference, GIFT deduces which objects in the unseen states should behave as objects in the training states. Afterwards, the unseen state components are aligned to training states so that the reward function learned before deployment can be used for planning. To address this issue, we introduce G eneralizing I ntent for F lexible T est-Time rewards (GIFT), a framework that grounds reward generalization in high-level intent. We leverage language models’ (LMs) commonsense reasoning abilities in two stages. First, we infer the human’s high-level intent by prompting LMs with contrasting pairs of user-preferred trajectories and non-preferred trajectories. Second, at test time, the inferred intent conditions another LM call that aligns novel test states to semantically similar training states. This lets the robot reuse its learned reward function without retraining. In experiments with a 7-DoF Franka Panda robot, both in simulation and the real world, we show that GIFT substantially outperforms low-level visual and language baselines on test-time reward generalization. II Related Work Learning Rewards from Humans. Robots commonly learn human goals from demonstrations [ 1 , 19 , 22 ] , preferences [ 13 , 30 ] , or corrections [ 29 , 5 , 4 ] . Language has recently emerged as a powerful medium for reward learning, enabling free-form text or natural language critiques to refine reward inference [ 38 , 26 ] . Other methods leverage LMs as priors to propose interpretable reward features or to ground abstract task objectives directly into reward functions [ 23 , 28 , 27 , 33 ] . While these approaches make it easier for non-experts to convey their intent without hand-engineering reward functions, the resulting models often overfit to surface correlations in limited data, leading to reward misspecification —where the learned reward captures proxies of success rather than the user’s true objective [ 3 , 7 ] . GIFT aims to avoid reward misspecification by interpreting demonstrations as expressions of high-level intent that can be inferred and transferred across domains to enable flexible test-time reward generalization. Test-Time Adaptation in Robots. Reward learning approaches often require users to teach the robot again when it encounters a new environment. Collecting data for every new environment can be an onerous process for users [ 8 , 9 , 17 , 16 ] , so other techniques aim to transfer the robot’s understanding of user preferences to unseen contexts at test time [ 34 , 21 , 32 , 39 , 11 , 31 ] . Test-time adaptation is often achieved by using large pretrained models to directly map from test-time observations to data seen during training by using textual correspondence between objects [ 39 , 25 ] , visual correspondence between videos [ 11 ] , or by projecting test images to the training manifold of images [ 31 ] . In contrast to previous approaches that perform mappings that directly reflect the data distribution, GIFT aims to leverage higher-order representations of user intention that are robust to distribution shifts. III Problem Formulation We consider the problem of adapting a robot’s behavior to align with human objectives under distribution shift. Our goal is to enable a learned reward function to generalize across

Read the complete text