TAB-Fields: A Maximum Entropy Framework for Mission-Aware Adversarial Planning
Gokul Puthumanaillam *, Jae Hyuk Song *, Nurzhan Yesmagambet, Shinkyu Park, Melkior Ornik · 2024
Read TAB-Fields: A Maximum Entropy Framework for Mission-Aware Adversarial Planning by Gokul Puthumanaillam *, Jae Hyuk Song *, Nurzhan Yesmagambet, Shinkyu Park, Melkior Ornik (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-02570.txt.
Introduction
Gokul Puthumanaillam and co-authors build behavior fields that forecast an adversary without guessing its exact plan in a 2024 paper. A rival robots mission, such as reaching a gate by a deadline, may be known while its steering policy is not. Classic planning treats the foe as hidden state and stalls on the unknown policy.
Their Task-Aware Behavior Fields instead spread probability over where the foe could be, using the least-biased spread that fits the mission and the terrain. A maximum entropy solve builds the field over time. Planners can then route around likely danger without inventing enemy habits. Tests show the fields track real foe motion.
Catalog record
| CREATOR | Gokul Puthumanaillam *, Jae Hyuk Song *, Nurzhan Yesmagambet, Shinkyu Park, Melkior Ornik |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-02570
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
TAB-Fields: A Maximum Entropy Framework for Mission-Aware Adversarial Planning Gokul Puthumanaillam *, Jae Hyuk Song *, Nurzhan Yesmagambet, Shinkyu Park, Melkior Ornik Abstract Autonomous agents operating in adversarial scenarios face a fundamental challenge: while they may know their adversaries’ high-level objectives, such as reaching specific destinations within time constraints, the exact policies these adversaries will employ remain unknown. Traditional approaches address this challenge by treating the adversary’s state as a partially observable element, leading to a formulation as a Partially Observable Markov Decision Process (POMDP). However, the induced belief-space dynamics in a POMDP require knowledge of the system’s transition dynamics, which, in this case, depend on the adversary’s unknown policy. Our key observation is that while an adversary’s exact policy is unknown, their behavior is necessarily constrained by their mission objectives and the physical environment, allowing us to characterize the space of possible behaviors without assuming specific policies. In this paper, we develop Task-Aware Behavior Fields (TAB-Fields), a representation that captures adversary state distributions over time by computing the most unbiased probability distribution consistent with known constraints. We construct TAB-Fields by solving a constrained optimization problem that minimizes additional assumptions about adversary behavior beyond mission and environmental requirements. We integrate TAB-Fields with standard planning algorithms by introducing TAB-conditioned POMCP, an adaptation of Partially Observable Monte Carlo Planning. Through experiments in simulation with underwater robots and hardware implementations with ground robots, we demonstrate that our approach achieves superior performance compared to baselines that either assume specific adversary policies or neglect mission constraints altogether. Evaluation videos and code are available at https://tab-fields.github.io . \coltauthor University of Illinois Urbana-Champaign, USA and King Abdullah University of Science and Technology, Saudi Arabia and University of Illinois Urbana-Champaign, USA keywords Adversarial planning, Mission-constrained planning, Planning under uncertainty 1 Introduction Effective autonomy in adversarial settings remains a fundamental problem in autonomous systems. ( Gu et al. 2014 ; Agmon 2017 ; Huang et al. 2019 ) . A core challenge in such settings lies in reasoning about the adversary’s state and its future trajectories, especially when critical aspects of their behavior—such as decision-making policies—are unknown ( Paruchuri 2007 ) . This lack of knowledge is further complicated by environmental factors like obstacles, terrain constraints, and dynamic operational constraints. Figure 1 : Overview of the proposed approach applied to an interception task. The adversary’s task is defined by mission objectives and environmental constraints (left). TAB-Fields are generated over time (top) to represent adversary state distributions and integrated into the planning process via TAB-conditioned POMCP (right). The resulting trajectories show the adversary’s path (red line), the agent’s response (green line), and the interception area ( ). One way to address this challenge is to treat the adversary’s state as a partially observable element within a broader system ( Gronauer and Diepold 2022 ; Zhang et al. 2020 ) . In this extended state space, the problem can be described as a Partially Observable Markov Decision Process (POMDP) ( Kaelbling et al. 1998 ) . POMDPs enable reasoning about uncertainty through belief dynamics—probability distributions over possible states—thereby facilitating structured decision-making. However, a fundamental obstacle arises in this context: the transition dynamics of the system depend on the adversary’s unknown policy, making them inherently indeterminate. Traditional POMDP planning methods rely on a priori knowledge of transition dynamics ( Castellini et al. 2021 ) , which is unavailable here. Our key observation is that while an adversary’s exact policy is unknown, their behavior is necessarily constrained by their mission objectives and the physical environment. Building on this observation, we propose an alternative approach: instead of assuming a specific adversary policy, we characterize the entire space of possible adversary behaviors that satisfy known mission objectives and environmental constraints. The key idea behind our approach is grounded in the principle of maximum entropy ( Jaynes 1982 ) — among all probability distributions consistent with the given constraints, the one with the highest entropy offers the most unbiased and comprehensive representation of the current state of knowledge. Leveraging this principle, we construct a distribution over adversary states that encapsulates the uncertainty in their decision-making while remaining consistent with all available information. This perspective shifts the focus from predicting specific adversarial behavior to reasoning about them in a mission-aware manner. Through this lens, we introduce Task-Aware Behavior Fields (TAB-Fields), a novel representation that encodes adversary state distributions over time using constrained entropy maximization. As shown in Figure 1 , TAB-Fields capture the evolution of belief states, demonstrating their ability to focus the distribution on regions consistent with mission constraints. TAB-Fields enable us to directly integrate adversary behavior into the belief update and planning process without relying on explicit policy assumptions or extensive training data. Statement of Contributions. The primary contribution of this work is TAB-Fields, a novel representation that captures the distribution of possible adversary states through principled entropy maximization subject to mission and environmental constraints. We show how this representation can be effectively integrated with existing planners through TAB-conditioned POMCP, an adaptation that maintains computational tractability while leveraging our structured representation. Through comprehensive evaluation across diverse scenarios in both simulation and hardware experiments, we demonstrate that our approach significantly improves mission-constrained adversarial planning compared to existing methods. 2 Related Work The presented work intersects several research areas in adversarial planning, behavior prediction, and planning under uncertainty. We discuss and highlight how our method differs from prior work. Planning Under Uncertainty. Planning in environments with uncertainty has been extensively studied within the framework of Partially Observable Markov Decision Processes (POMDPs) ( Kaelbling et al. 1998 ) . Traditional POMDP solvers ( Silver and Veness 2010 ; Somani et al. 2013 ) rely on known transition and observation models ( Shani et al. 2013 ; Lauri et al. 2022 ) to perform belief updates and compute optimal policies. However, when the environment includes other agents with unknown policies—such as adversaries—the transition dynamics become partially unknown, complicating standard POMDP approaches ( Ng et al. 2010 ; Egorov et al. 2016 ) . Several works have extended POMDP frameworks to handle interactions with other agents. Interactive POMDPs ( Han and Gmytrasiewicz 2018 ; Han and Gmytrasiewicz 2019 ) model other agents by maintaining beliefs over their beliefs and policies, but this quickly becomes intractable due to the curse of dimensionality. Decentralized POMDPs ( Czechowski and Oliehoek 2021 ) consider multiple cooperative agents, but are less suited for adversarial settings. Modeling Adversary Behavior. In surveillance and security domains, predicting adversary behavior is critical. Traditional methods ( Zhou and Tokekar 2021 ; Santos Jr and Zhao 2006 ; Santos Jr et al. 2008 ) often assume specific models of adversary policies, such as rational decision-makers optimizing a known utility function ( Zuckerman et al. 2012 ) . However, these assumptions may not hold in practice, leading to ineffective strategies. To mitigate this, some approaches use learning-based methods to model adversary behavior from observed data ( Abouelyazid 2023 ; Huang et al. 2019 ) . While effective when ample data is available, these methods struggle when observations are sparse. Robust planning methods consider worst-case scenarios without relying on specific adversary models ( Nilim and El Ghaoui 2005 ; Iyengar 2005 ) . However, these can be overly conservative. Maximum Entropy Methods for