Online Relational Inference for Evolving Multi-agent Interacting Systems
Beomseok Kang, Priyabrata Saha, Sudarshan Sharma, Biswadeep Chakraborty · 2024
Read Online Relational Inference for Evolving Multi-agent Interacting Systems by Beomseok Kang, Priyabrata Saha, Sudarshan Sharma, Biswadeep Chakraborty (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-01442.txt.
Description
Abstract We introduce a novel framework, Online Relational Inference (ORI), designed to efficiently identify hidden interaction graphs in evolving multi-agent interacting systems using streaming data. Unlike traditional offline methods that rely on a fixed training set, ORI employs online backpropagation, updating the model with each new data point, thereby allowing it to adapt to changing environments in real-time.
Catalog record
| CREATOR | Beomseok Kang, Priyabrata Saha, Sudarshan Sharma, Biswadeep Chakraborty |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2411-01442
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Online Relational Inference for Evolving Multi-agent Interacting Systems Beomseok Kang, Priyabrata Saha, Sudarshan Sharma, Biswadeep Chakraborty Abstract We introduce a novel framework, Online Relational Inference (ORI), designed to efficiently identify hidden interaction graphs in evolving multi-agent interacting systems using streaming data. Unlike traditional offline methods that rely on a fixed training set, ORI employs online backpropagation, updating the model with each new data point, thereby allowing it to adapt to changing environments in real-time. A key innovation is the use of an adjacency matrix as a trainable parameter, optimized through a new adaptive learning rate technique called AdaRelation, which adjusts based on the historical sensitivity of the decoder to changes in the interaction graph. Additionally, a data augmentation method named Trajectory Mirror (TM) is introduced to improve generalization by exposing the model to varied trajectory patterns. Experimental results on both synthetic datasets and real-world data (CMU MoCap for human motion) demonstrate that ORI significantly improves the accuracy and adaptability of relational inference in dynamic settings compared to existing methods. This approach is model-agnostic, enabling seamless integration with various neural relational inference (NRI) architectures, and offers a robust solution for real-time applications in complex, evolving systems. Code is available at https://github.com/beomseokg/ORI . 1 Introduction Multi-agent interacting systems have been studied in various fields, including particle-based physical simulations [ 1 , 2 , 3 ] , traffic systems [ 4 , 5 ] , and social networks [ 6 , 7 , 8 ] . Interaction among agents are crucial information to accurately model such systems and providing the interpretability in agent behaviors as well [ 9 ] . However, external observers can only access the trajectory of agents without knowing interaction graphs. Accordingly, identifying unknown interaction graphs from observable trajectories of agents has been emerged as a specific problem referred to as relational inference [ 10 ] . In recent years, neural relational inference (NRI) and its variants have shown promising performance in synthetic and real-world environments [ 11 , 12 , 13 , 14 , 15 , 16 ] . Prior studies mostly aimed to present better network for NRI based on variational autoencoder (VAE) built with graph neural networks (GNN) [ 14 , 15 , 12 , 17 ] . These methods involve an encoder to infer an interaction graph as an adjacency matrix from observed trajectories and a decoder to predict the future trajectories employing the inferred interaction graph. They generally perform training offline assuming the well-aligned distribution in training and testing data. Unfortunately, such assumption is frequently violated in practice due to the shifts in test condition, including sudden changes in the interaction, evolving system parameters and even dynamics itself. Building a relational inference model generalizable to all the different scenarios is challenging [ 18 , 19 , 20 ] . In this case, online learning is an attractive approach to continuously adapt the model to the newly observed environments [ 5 ] . However, online learning for relational inference has been rarely explored. Online backpropagation using gradient descent is a widely used online learning method as it is compatibility to various neural network designs [ 21 ] . However, online backpropagation on existing NRI models significantly degrades the accuracy on relational inference since their decoder quickly learns the trajectory prediction even before the encoder generates reasonable interaction graphs. It is important to note that while the models are trained in self-supervised manner ( i.e ., trajectory prediction), true labels of interaction graphs are never provided to the models, indicating that identifying interaction graphs is essentially unsupervised. That is, optimizing the unsupervised encoder is more challenging than the self-supervised decoder in nature. The key problem is how to match the learning speed between the interaction identification and the trajectory prediction so that both the tasks can be collaboratively optimized. In this paper, we propose a novel framework named O nline R elational I nference ( ORI ) to efficiently identify hidden interaction graphs in evolving multi-agent interacting systems. Our method strategically allocates an adjacency matrix representing the interaction as a trainable parameter in the model and directly optimizes it through online backpropagation on the predicted trajectories. It effectively accelerates the update of the adjacency matrix than the encoder-based approach and ensures the following decoder to be learned with reasonable adjacency matrices from the early stage of training. ORI can seamlessly integrate with prior NRI models, offering architectural flexibility ( i.e. , model-agnostic). Moreover, we developed an adaptive learning rate technique named AdaRelation particularly designed for relational inference in the evolving systems. It employs the historical adjacency matrix to indirectly estimate the decoder’s sensitivity over the adjacency matrix and determine whether the learning needs to be accelerated. In addition, we introduce a data augmentation technique named Trajectory Mirror (TM) to expose various trajectories by flipping the axis in the systems. We experimentally demonstrate the effectiveness of ORI on various NRI models in both synthetic and the real-world (CMU MoCap for human motion [ 22 ] ) dataset. Our key contributions are as follows: • To the best of our knowledge, ORI is the first model-agnostic online relational inference framework for evolving multi-agent interacting systems. ORI employs online backpropagation to optimize an adjacency matrix from the trajectory information without any assumptions on the model architecture. • We experimentally demonstrate that ORI identifies unknown interaction graphs in various evolving multi-agent interacting systems, such as sudden changes in interaction (Figure 2 ), parameters in the dynamics (Figure 3 ), and even dynamics itself (Figure 3 ), outperforming existing NRI models (Table 2 ). • We propose AdaRelation, a novel adaptive learning rate technique particularly designed for relational inference in evolving multi-agent systems. It automatically tunes the learning rate for the adjacency matrix when interaction or/and dynamics in the system suddenly change (Figure 3 ). • We propose Trajectory Mirror, a data augmentation technique to ensure the reasonable relational inference regardless of the trajectory axis. It significantly improves the convergence speed and overall interaction prediction accuracy in the several evolving scenarios (Supplementary). Table 1: Comparison of key features between prior works and this work. Method Description Model Consider Evolution in Criteria Agnostic Interaction Parameter Dynamics Acc. MSE Prior offline works [ 11 , 7 , 12 , 23 , 16 ] \cdot offline backpropagation \cdot novel encoder and decoder - ✓ × × ✓ ✓ Prior online work [ 5 ] \cdot online convex optimization \cdot constant learning rate × ✓ × × × ✓ This work \cdot online backpropagation \cdot AdaRelation; Traj. Mirror ✓ ✓ ✓ ✓ ✓ ✓ • ✓ indicates the presence of a feature, and × indicates the absence of a feature. 2 Related Works Neural relational inference. NRI [ 11 ] is the first work that formulated the problem of an unsupervised relational inference from observed agent trajectories. They implemented a VAE-based graph neural network that encodes trajectories into an adjacency matrix, representing relation types, and then decodes the trajectory using the predicted adjacency matrix. This approach has been established as a standard in several follow-up works. For example, dNRI [ 12 ] and EvolveGraph [ 15 ] introduced graph recurrent network-based architectures to encode and generate dynamic priors for predicting evolving interactions. Additionally, a memory-augmented architecture