Security-Aware Sensor Fusion with MATE: the Multi-Agent Trust Estimator

R. Spencer Hallyburton, Miroslav Pajic · 2025

Read Security-Aware Sensor Fusion with MATE: the Multi-Agent Trust Estimator by R. Spencer Hallyburton, Miroslav Pajic (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-04954.txt.

Introduction

Hallyburton and Pajic build MATE in twenty twenty-five, a trust aware way to merge sensor data from many agents. Fleets of robots or smart city sensors blend their readings to track what is going on. A hacked or faulty agent can poison that blend. MATE scores how much each agent should be trusted, then weighs its data by that score.

Trust is tracked as a hidden state that updates step by step as new data arrive. Raw readings are turned into trust clues by a new field of view tool. Tests in a game engine simulator, under attack, kept the merged picture sound. Scores on new and classic safety measures backed the design.

Catalog record

CREATORR. Spencer Hallyburton, Miroslav Pajic
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE76 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-04954

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Security-Aware Sensor Fusion with MATE: the Multi-Agent Trust Estimator R. Spencer Hallyburton, Miroslav Pajic Abstract Lacking security awareness, sensor fusion in systems with multi-agent networks such as smart cities is vulnerable to attacks. To guard against recent threats, we design security-aware sensor fusion that is based on the estimates of distributions over trust . Trust estimation can be cast as a hidden Markov model, and we solve it by mapping sensor data to trust pseudomeasurements (PSMs) that recursively update trust posteriors in a Bayesian context. Trust then feeds sensor fusion to facilitate trust-weighted updates to situational awareness. Essential to security-awareness are a novel field of view estimator, logic to map sensor data into PSMs, and the derivation of efficient Bayesian updates. We evaluate security-aware fusion under attacks on agents using case studies and Monte Carlo simulation in the physics-based Unreal Engine simulator, CARLA. A mix of novel and classical security-relevant metrics show that our security-aware fusion enables building trustworthy situational awareness even in hostile conditions. 1 Introduction Securing cyber-physical systems (CPS) against attacks is a perennial problem. The proliferation of low-cost, networked agents has upended traditional monolithic platforms and elucidated security challenges. Of particular concern is collaborative sensor fusion . While multi-agent fusion is effective at maintaining accurate situational awareness (SA), it is vulnerable to adversaries as it assumes agents behave according to benign models of sensing. Even small numbers of compromised nodes can take down collaborative data fusion in large-scale heterogeneous networks [ 29 , 20 ] . In this work, we consider an urban environment composed of networked mobile and static agents (“ smart city ”) that wish to maintain accurate SA by estimating both the number of objects that exist and the states of those objects (e.g., position, velocity) with multiple target tracking (MTT). Unfortunately, agents are limited by inter-object occlusions and may be adversarially compromised. To overcome such limitations, agents are connected to the smart city network and perform collaborative sensor fusion . Agents send local information to the smart-city data aggregator that builds unified SA on the urban environment with MTT. Collaboration is useful for distributing both global SA and specific warnings including intersection collision warnings (ICW), mobile accessible pedestrian signaling (MAPS), and blind merge warnings (BMW) during safety-critical interactions [ 4 ] . CPS security necessitates measures both proactive at the network-level (e.g., asymmetric cryptography) and reactive at the algorithm-level (e.g., misbehavior detectors, MBDs) [ 25 , 46 ] . We consider an insider attacker possessing valid cryptographic keys. In each frame, the adversary may perturb sensing/detection data of the compromised agent by injecting false positives (FPs), creating false negatives (FNs), or translating existing objects. In the worst case, manipulations will propagate over time in attacker-defined trajectories stealthily consistent with plausible dynamics to engender safety-critical outcomes such as an impending collision. Since the set of compromised agents is unknown, security awareness at data fusion is important. Unfortunately, many secure computing algorithms were designed with Byzantine fault models for distributed ad-hoc networks, e.g., [ 39 , 44 ] , are MBDs that only detect malicious data and cannot recover trusted SA [ 44 , 17 ] , or require that the ego agent is trusted [ 44 , 39 ] . These have limited applicability for autonomous agents with error-prone perception data derived from real sensors in applications such as smart cities or remote reconnaissance. Consequently, in this work, we design a two-part security-aware sensor fusion algorithm consisting of (1) estimating probability density functions (PDFs) over the trust of agents and tracked objects, and (2) trust-informed collaborative data fusion. The trust of agents and their tracked objects are estimated in a Bayesian context allowing incorporating prior information and measurement uncertainty. Trust-informed data fusion then influences the multi-platform data aggregator. The security-aware sensor fusion both detects misbehaving agents and recovers accurate SA under adversarial manipulation. Trust estimation is a two-step hidden Markov model (HMM). The first step is to propagate the estimate forward in time. The second step is to update the estimate with measurements. Since there is no sensor providing direct measurements of trust (unlike e.g., GPS providing position), we design a novel method of mapping real perception-oriented sensor data to trust pseudomeasurements (PSMs) composed of a value and confidence on domain [0,1] . We decompose the trust update between agents/objects using conditional probability inspired by Gibbs sampling and employ conjugate prior-likelihood pairs to obtain feasible and efficient updates to trust PDFs. PSMs are made comparing an agent’s locally-estimated SA with a prediction of what it should have observed. The prediction leverages dynamic estimates of the agent’s field of view (FOV) at each timestep. Prior works fixed constant FOV shapes (e.g., circles [ 17 , 37 , 42 , 1 ] ). This restriction fails in realistic cases because occlusions by objects/infrastructure constantly alter each sensor’s FOV. Instead, to build robust FOV models that are dynamic on a frame-to-frame basis and capture occlusions, we implement a ray tracing algorithm on each agent’s LiDAR point cloud to estimate FOVs online. Without security-informed fusion, any compromised agent can corrupt collaborative SA, even in large networks [ 29 , 20 ] . Instead, we propose trust-informed multi-agent fusion. Agent trust PDFs are used as weights so distrusted agents have less influence on collaborative SA. Object trust PDFs inform track management, as distrusted tracks are flagged for investigation or removed. Employing trust-informed data fusion enables mitigation and recovery from adversarial influence on SA. Evaluations of attacks and defenses are performed on longitudinal multi-agent scenes constructed from the pipelined smart-city dataset generator [ 19 ] built on the Unreal Engine simulator, CARLA [ 15 ] . We design baseline (unattacked) scenes and derive parameterized adversarial versions by applying threat models from state of the art attacks on sensing (e.g., [ 10 , 18 , 21 ] ). We Monte Carlo (MC) randomize the adversary parameters to generate a large volume of attacks for analysis. We convey model performance with detailed case studies highlighting the efficacy of security-aware fusion. Two traditional classes of metrics capture performance of the object existence (number of objects) and state estimation capability of unsecured and trust-informed fusion: (1) precision, recall, and F1-score, and (2) optimal subpattern assignment (OSPA) [ 35 ] . To evaluate MATE, we derive novel trust-oriented metrics that compare trust PDFs to agent and track states. Metrics illustrate that security-aware fusion detects and identifies distrusted agents/tracks and recovers accurate SA despite adversarial data manipulation. Compared to fusion lacking security awareness, our approach leads to 94\% reduction in adversary-driven OSPA error versus the unattacked baseline. Trust-based metrics report MATE adeptly detects compromised agents with near 90\% accuracy. Security-aware fusion maintains high levels of mission performance in both benign and adversarial scenarios and is sufficiently robust to be deployed in dynamic, unknown environments. Contributions. In summary, our contributions are: • First full-stack framework and implementation for security-aware sensor fusion. Multi-agent trust estimation (MATE) and trust-informed data fusion significantly mitigates adversaries in multi-agent networks. • Real-time LiDAR-based FOV estimation to predict the visible area and observable objects free from occlusions. • Novel trust pseudomeasurement algorithm to map real sensing data to the trust domain. • Efficient updates to agent/track trust PDFs using Gibbs decomposition and weighted

Read the complete text