Latent Perspective‑Taking via a Schrödinger Bridge in Influence‑Augmented Local Models Kevin Alcedo, Pedro U. Lima, Rachid Alami Abstract Operating in environments alongside humans requires robots to make decisions under uncertainty. In addition to exogenous dynamics, they must reason over others’ hidden mental-models and mental-states . While Interactive POMDPs and Bayesian Theory of Mind formulations are principled, exact nested-belief inference is intractable, and hand-specified models are brittle in open-world settings. We address both by learning structured mental-models and an estimator of others’ mental-states. Building on the Influence-Based Abstraction, we instantiate an Influence-Augmented Local Model to decompose socially-aware robot tasks into local dynamics, social influences, and exogenous factors. We propose (a) a neuro-symbolic world model instantiating a factored, discrete Dynamic Bayesian Network, and (b) a perspective-shift operator modeled as an amortized Schrödinger Bridge over the learned local dynamics that transports factored egocentric beliefs into other-centric beliefs. We show that this architecture enables agents to synthesize socially-aware policies in model-based reinforcement learning, via decision‑time mental‑state planning (a Schrödinger‑Bridge in belief space), with preliminary results in a MiniGrid social navigation task. 1 Introduction A socially-aware robot acts alongside humans and robots, pursuing its own goals while enabling others to achieve theirs and accommodating known or inferred needs. Hence, it must make decisions under partial observability: both of the environment and of others’ mental-models and mental-states . We adopt the lens of classical Interactive POMDPs (I-POMDPs) ( Gmytrasiewicz & Doshi 2004 ) and Bayesian Theory of Mind (BToM) ( Baker et al. 2011 ) : mental-models are (known or inferred) internal transition, observation, reward and action models, and mental-states are probability distributions over latent states (beliefs). The knowledge-representation community has formalized perspective-taking ( Flavell 1992 ) with ontologies/semantic memory ( Milliez et al. 2014 ; Sarthou et al. 2019 ) , epistemic human-aware task planners ( Shekhar et al. 2025 ) , and dynamic epistemic logic ( Bolander & Andersen 2011 ; Bolander et al. 2021 ) , yielding interpretable and verifiable systems. Yet these symbolic models are brittle in open-world settings with stochastic dynamics, and exact nested-belief reasoning quickly becomes intractable; PSPACE at level-0, and EXPSPACE in the worst case with factored dynamics and exponential horizon ( Mundhenk et al. 2000 ) . In contrast, probabilistic sequential decision-making naturally handles uncertainty but typically assumes prescribed models and still suffers from the nested-belief explosion. As a step toward unifying symbolic knowledge representation and probabilistic sequential decision-making, we propose a data-driven architecture that learns (i) factored, discrete mental-models and (ii) a tractable belief-to-belief map for perspective-taking in factored latent space. The key idea is to cast perspective-taking as belief transport via a Schrödinger Bridge (SB) ( Fortet 1940 ) defined over learned local reference dynamics, producing other-centric beliefs compatible with local dynamics. Building on the Influence-Based Abstraction (IBA) framework ( Oliehoek et al. 2021 ) , we instantiate an Influence-Augmented Local Model (IALM) to decompose socially-aware robot tasks into local dynamics, social influence, and exogenous factors. We operationalize this with a neuro-symbolic world model and a perspective-shift operator as an amortized SB that runs online at decision-time. Inspired by Simulation Theory ( Goldman 1992 ) , the agent seeds a mental-state and reuses its cognitive machinery to simulate others’ mental-state dynamics, yielding a “what-if” other-centric belief for decision-making akin to model predictive control in mental-state space. Preliminary results in a grid-world social navigation task show our SB-based perspective-taking learns faster and achieves higher returns than using the true context-free current belief. Our contributions are: (1) a practical influence-augmented decomposition for social tasks; (2) a factored, discrete neuro-symbolic world model; and (3) perspective-taking via an amortized Schrödinger Bridge that enables model-based belief-space planning at decision-time. (a) (b) Figure 1: (a) Influence-Augmented Local Model; (b) Neuro-Symbolic World Model Architecture. 2 Background 2.1 Influence-Augmented Local Model The IBA framework ( Oliehoek et al. 2021 ) addresses the challenge of partitioning large state spaces in structured multi-agent settings by factorizing the global model into smaller, tractable local models and by identifying influence points that compactly capture boundary effects. When modeled exactly, the influence is a sufficient statistic that enables an IALM to compute an exact best response (for fixed external policies) while avoiding reasoning over the full global space. In practice, enumerating all influence sources and updating their estimates is intractable, and approximations are used ( Suau et al. 2022 ) . The factored representation makes the probabilistic dependency structure explicit and yields a compact two-stage Dynamic Bayesian Network (2DBN) ( Boutilier et al. 1999 ; Poupart 2005 ) . An IALM extends a local factored POMDP (FPOMDP) with a factored global state \mathcal{S}=\{S_{1},\dots,S_{k}\} partitioned into the following variables: local state \mathcal{X} , influence sources \mathcal{U} , and non-local states \mathcal{Y} . Let s_{t}\in S=\times_{i=1}^{k}S_{i} , x_{t}\in X=\times_{i\in\mathcal{X}}S_{i} , u_{t}\in U=\times_{i\in\mathcal{U}}S_{i} , and action a_{t}\in\mathcal{A} at time t . The key insight from Oliehoek et al. 2021 is that a minimal d -separating set ( Koller & Friedman 2009 ) D_{t} , a subset of the action-local-state history h_{t}=\langle x_{1},a_{1},\dots,a_{t-1},x_{t}\rangle , is sufficient to predict the influence distribution I(u_{t}\mid D_{t}) . D_{t+1} is constructed deterministically with function d(\cdot) : D_{t+1}=d(h_{t}) . We arrive at the tuple \mathcal{M}^{\mathrm{IALM}}=\langle\bar{\mathcal{S}},\mathcal{A},\bar{T},R,\mathcal{O},\bar{O},b_{0}\rangle , where \bar{\mathcal{S}} is the augmented local state space \bar{s}_{t}=\langle x_{t},D_{t+1}\rangle , \bar{T}(x_{t+1}\mid\bar{s}_{t},a_{t}) is the augmented local transition, \bar{O}(o_{t+1}\mid a_{t},x_{t+1}) is the local observation with o_{t+1}\in\mathcal{O} , R(x_{t},a_{t},x_{t+1}) is the local reward, and b_{0} is an initial local belief. The influence adjusts the local transition by marginalization: \bar{T}(x_{t+1}\mid\bar{s}_{t},a_{t})=\sum_{u_{t+1}\in U}\Pr(x_{t+1}\mid x_{t},a_{t},u_{t+1})\;I(u_{t+1}\mid D_{t+1}) (1) 2.2 Schrödinger Bridge Originally conceived to explain the most likely path distribution (the bridge) between the start and end points of moving particles under an assumed dynamics model, a Schrödinger Bridge (SB) can be formulated as the probability law on paths linking two endpoint marginals ( Schrödinger 1931 ; Chetrite et al. 2021 ) ; the goal is to discover a new law describing the dynamics of the trajectory. SB can be viewed as a special case of Optimal Transport: dynamic entropic optimal transport with reference dynamics ( Léonard 2014 ) ; we minimize a pathwise KL divergence to the trajectory distribution induced by the reference transition model. Given a reference local dynamics \dot{T}(x_{t+1}\mid x_{t},a_{t}) , start/end marginals (beliefs) b_{0} and b_{n} , and conditioning on a fixed action sequence a_{0:n-1} , we seek a bridge \bar{T}=\{\bar{T}_{t}\}^{n-1}_{t=0} (a family of time-varying transitions) whose induced trajectory distribution is closest to that of the reference \dot{T} . Concretely, we compute \bar{T} such that its induced trajectory distribution \mathbb{P}_{\bar{T}} minimizes \mathrm{KL}\big(\mathbb{P}_{\bar{T}}\,\|\,\mathbb{P}_{\dot{T}}\big) , subject to the endpoint marginals ( b_{0},b_{n} ). This yields a first-order, time-inhomogeneous Markov chain. Here we focus on the space-time Sinkhorn method ( Fortet 1940 ) , where forward and backward potentials \phi_{t},\psi_{t}\in\mathbb{R}_{>0}^{|\mathcal{X}|} are propagated through the reference \dot{T} and produce the bridge. We parametrize the potentials with context c_{t}:=D_{t+1}=d(h_{t}) with h_{t}=\langle x_{1},a_{1},\dots,a_{t-1},x_{t}\rangle . The Schrödinger system becomes: \displaystyle\psi_{t}(x_{t};c_{t})=\sum_{x_{t+1}}\dot{T}(x_{t+1}\mid x_{t},a_{t})\psi_{t+1}(x_{t+1};c_{t+1}), (2) \displaystyle\phi_{t+1}(x_{t+1};c_{t+1})=\sum_{x_{t}}\phi_{t}(x_{t};c_{t})\dot{T}(x_{t+1}\mid x_{t},a_{t}), \displaystyle p_{t}(x_{t})=\phi_{t}(x_{t};c_{t})\psi_{t}(x_{t};c_{t}),\quad p_{0}=b_{0},\;p_{n}=b_{n} The steered transitions take the Doob h -transform form ( Doob 1957 ) : \bar{T}_{t}(x_{t+1}\mid\bar{s}_{t},a_{t})=\dot{T}(x_{t+1}\mid x_{t},a_{t})\frac{\psi_{t+1}(x_{t+1};c_{t+1})}{\psi_{t}(x_{t};c_{t})},\quad\mathrm{where}\;\bar{s}_{t}:=\langle x_{t},D_{t+1}\rangle (3) The induced trajectory distribution, with initial state distribution \mu_{0} , is: \mathbb{P}_{\bar{T}}(x_{0:n})\propto\phi_{0}(x_{0};c_{0})\mu_{0}(x_{0})\bigg[\prod_{t=0}^{n-1}\dot{T}(x_{t+1}\mid x_{t},a_{t})\bigg]\psi_{n}(x_{n};c_{n}) (4) One way to view this process is as control-as-inference ( Todorov 2006 ) , targeting a minimum-effort steering of a noisy passive dynamic process. In Section 3 we instantiate an IALM as a neuro-symbolic world model that learns (\bar{\mathcal{S}},\dot{T},\mathcal{O},\bar{O},D) . We then realize \bar{T}=\{\bar{T}_{t}\} as an approximation of the ideal IALM transition in Eq. 1 via an amortized Schrödinger Bridge tilt of \dot{T} for perspective-taking. The potentials \phi and \psi are trained offline with epistemic counterfactuals. 3 Method 3.1 Neuro-Symbolic World Model We instantiate the IALM as a neuro-symbolic world model, composed of a hierarchy of discrete variational-autoencoders (dVAEs) with Gumbel-Softmax ( Jang et al. 2017 ; Maddison et al. 2017 ) trained end-to-end on transitions (o_{t},a_{t},o_{t+1}) , explicitly modeling the structural components of a 2DBN (See Fig. 1 ). Raw observations are encoded into a latent of shape (C,F) , where C is the number of discrete observation factors and each has F categories. These are then mapped to factored state latents (N,K) , with N state factors each with K categories. A local, influence-naïve transition model \dot{T} maps the current factored state and a one-hot action to the next factored latent state. The transition model uses \alpha -entmax ( Peters et al. 2019 ) cross-attention over the input state factors to induce exact-zero, data-driven sparsity, exploiting latent parent–child structure. 3.2 Perspective-Shift Operator Extending Alcedo et al. 2025 , we estimate others’ beliefs via an SB-based perspective-shift operator: \hat{b}_{i}=\Phi_{\mathrm{SB}}(b_{0},\hat{x}_{i},a_{0:n-1},\dot{T}) , where b_{0} is the egocentric belief, \hat{x}_{i} is the locally observed relative state of another agent i , \dot{T} is the learned local influence-naïve transition model (the reference dynamics), a_{0:n-1} is a proposed action sequence hypothesized for agent i generated via an upstream local planner, and \hat{b}_{i} is the estimated other-centric belief. We implement \Phi_{\mathrm{SB}} by learning the potentials ( \hat{\phi_{t}},\hat{\psi_{t}} ) using a bi-directional sequence model (BiGRU) with \alpha -entmax attention on the action-local-state history h_{t}=\langle x_{1},a_{1},\dots,a_{t-1},x_{t}\rangle generated from a roll-out using the reference model \dot{T} . The sparsity induced by \alpha -entmax on h_{t} serves as the deterministic update to compute the approximate minimal d-set from the action-local-state history D_{t+1}=d(h_{t}) . At training time, we sample “epistemic counterfactuals” (other-centric beliefs) b_{n} in a single-agent task from the environment to amortize the potentials that realize \bar{T} via the Doob h -transform of \dot{T} , minimizing \mathrm{KL}\big(\mathbb{P}_{\bar{T}}\,\|\,\mathbb{P}_{\dot{T}}\big) subject to the endpoints. At inference, we tilt \dot{T} with the amortized potentials ( \hat{\phi_{t}},\hat{\psi_{t}} ) parametrized by c_{t}:=D_{t+1}=d(h_{t}) to reach an estimate for \hat{b}_{i} . The learning objective is done at the per-factor level, enabling d(h_{t}) to contextually manage the tilt by masking factors. The objective becomes: \mathcal{L}=\mathcal{L}_{\bar{O}}+\mathcal{L}_{\dot{T}}+\mathcal{L}_{\Phi} . We employ training stabilization techniques that will be discussed and ablated in the full manuscript; these include: Gumbel-softmax \tau annealing, knowledge distillation confidence-gating, and socially-weighted epistemic counterfactuals objectives at two levels of reasoning. (a) (b) Figure 2: (a) Perspective-shift representations. Left: global state. Right: ground-truth observations (top); factored, discrete latent states (second); actions (third); next-step latent states (fourth); reconstructed observation (last). (b) Median episodic returns over 5 seeds, shaded band denotes 95% bootstrap CI of the median; dashed line: Perfect-Information baseline final median. 4 Preliminary Results, Limitations & Next Steps We evaluate our approach for influence-aware policy learning in a multi-agent model-based reinforcement learning setting. Environment: partially-observable Minigrid ( Chevalier-Boisvert et al. 2023 ) , with discrete observations and actions. Person-following Task: follow a fixed‑policy agent to an unknown target while satisfying social norms ( Alcedo et al. 2025 ) . The environment rewards proximity and visibility, gives a large terminal reward for reaching the goal, penalizes collisions, and truncates episodes when the follower stays too far away for too long. Influence-aware Policy: a_{t}=\pi_{\mathrm{DQN}}(b^{0}_{t},\hat{b}^{i}_{t}) . Baselines: (i) Perfect-information, true current belief of the other agent with no context/history: \hat{b}^{i}_{t}=b^{i}_{t} , (ii) No-information: \hat{b}^{i}_{t}\sim\mathrm{Uniform}(\Delta_{X}) , and (iii) Reference (local dynamics) multi-step rollout: \hat{b}^{i}_{t}=\prod^{n-1}\dot{T}(\cdot) . Results: SB learns faster and achieves higher returns than all baselines (See Fig. 2 ), highlighting the benefit of decision‑time, context‑aware mental‑state planning over context‑free estimates. Currently, our approach assumes homogeneity in the perception and action models of other agents, we aim to address this in future work by adapting the belief-to-belief transform objective via heterogeneous epistemic counterfactuals. 5 Related Work Latent perspective‑taking has been approached via meta‑learned agent embeddings ( Rabinowitz et al. 2018 ) , latent state‑space world models that generate other‑centric observations ( Chen et al. 2023 ) , and latent opponent/teammate modeling under partial observability ( Labash et al. 2020 ; Papoudakis et al. 2021 ) . In contrast, we cast perspective‑taking as belief‑to‑belief transport, implementing an amortized SB within a learned IALM to produce other‑centric beliefs for decision‑time mental-state planning; we extend the local-model-only approach of Alcedo et al. 2025 . Suau et al. 2022 instantiate IALMs at scale in multi-agent reinforcement learning settings, with prescribed influence sources and learned predictors, and Suau et al. 2025 learn influence‑aware memory and a d-set selector in pixel space (ours operates in latent space). Finally, Wang et al. 2023 proposed the generalized belief transport method for teacher-student learning settings modeled via a static unbalanced entropic optimal transport, whereas we use a Schrödinger Bridge.