Decentralized Vision-Based Autonomous Aerial Wildlife Monitoring

Makram Chahine *,†,1, William Yang *,1 Alaa Maalouf 1, Justin Siriska 2, Ninad Jadhav 2, Daniel Vogt 2, Stephanie Gil 2, Robert Wood 2, Daniela Rus 1 · 2025

Read Decentralized Vision-Based Autonomous Aerial Wildlife Monitoring by Makram Chahine *,†,1, William Yang *,1 Alaa Maalouf 1, Justin Siriska 2, Ninad Jadhav 2, Daniel Vogt 2, Stephanie Gil 2, Robert Wood 2, Daniela Rus 1 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-15038.txt.

Introduction

Chahine and co-authors present a team of camera drones that monitor wildlife without a central controller in this 2025 paper. Field teams need to find and follow specific animals for behavior studies and health checks. Earlier robot solutions steer whole herds or need human pilots.

Their system flies several quadrotors, each with a single ordinary color camera. The drones coordinate through vision-based algorithms built for messy outdoor settings. They need no central communication hub and use little bandwidth. Real field experiments showed reliable deployment across varied conditions. The design targets large species in their natural habitat. The authors aim for scale: many drones working in parallel at once.

Catalog record

CREATORMakram Chahine *,†,1, William Yang *,1 Alaa Maalouf 1, Justin Siriska 2, Ninad Jadhav 2, Daniel Vogt 2, Stephanie Gil 2, Robert Wood 2, Daniela Rus 1
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE23 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-15038

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Decentralized Vision-Based Autonomous Aerial Wildlife Monitoring Makram Chahine *,†,1, William Yang *,1 Alaa Maalouf 1, Justin Siriska 2, Ninad Jadhav 2, Daniel Vogt 2, Stephanie Gil 2, Robert Wood 2, Daniela Rus 1 Abstract Wildlife field operations demand efficient parallel deployment methods to identify and interact with specific individuals, enabling simultaneous collective behavioral analysis, and health and safety interventions. Previous robotics solutions approach the problem from the herd perspective, or are manually operated and limited in scale. We propose a decentralized vision-based multi-quadrotor system for wildlife monitoring that is scalable, low-bandwidth, and sensor-minimal (single onboard RGB camera). Our approach enables robust identification and tracking of large species in their natural habitat. We develop novel vision-based coordination and tracking algorithms designed for dynamic, unstructured environments without reliance on centralized communication or control. We validate our system through real-world experiments, demonstrating reliable deployment in diverse field conditions. † † institute: ∗ Equal contribution, 1 CSAIL MIT, Cambridge MA 02139, USA, 2 SEAS Harvard University, Boston MA 02134, USA, † Corresponding author: Figure 1: Coordinated multi-quadrotor system for wildlife monitoring. 1 Introduction Monitoring wildlife at scale requires systems capable of parallel, individual-level observation across large environments. In particular, understanding species behavior and communication demands the ability to identify and track multiple animals simultaneously—critical for behavioral analysis, health monitoring, and intervention. Traditional monitoring strategies are often manual, centralized, or constrained in scale and adaptability. We tackle the problem of detecting and tracking individual animals in the wild using autonomous aerial agents. Our solution targets scenarios such as marine mammal monitoring, where long-range detection and real-time coordination are vital. Using sperm whales as a running example, we introduce a fully autonomous, decentralized aerial system that relies solely on onboard RGB vision to detect species, register identities across agents, assign goals, and execute monitoring behaviors—entirely without centralized control or GPS. This system introduces new modules for decentralized visual registration (Box-ICP), task allocation using graph neural networks, and end-to-end real-world deployment under communication and sensing constraints. Through real and simulated experiments, we validate its ability to perform robust group-level monitoring of large animals in unstructured, dynamic environments. 2 Related Work Vision-Based Detection and Navigation. Object detection has seen dramatic progress over the past decade, beginning with two-stage frameworks like Faster R-CNN ren2016fasterrcnnrealtimeobject , which separate region proposal and classification, followed by efficient single-stage models such as SSD Liu_2016 and YOLO redmon2016lookonceunifiedrealtime . More recent detectors incorporate transformer architectures, as in DETR carion2020endtoendobjectdetectiontransformers , or enhance inference speed and robustness through improved convolutional designs, such as in YOLOv8 and YOLOv12 reis2024realtimeflyingobjectdetection ; tian2025yolov12 . These advances have enabled high-performance onboard detection suitable for real-time deployment on aerial robots. In parallel, vision-based navigation has progressed from classical imitation learning systems pomerleau1988alvinn ; bojarski2016end to modern end-to-end deep learning approaches chib2023recent , which achieve robust navigation in structured environments. However, these models often struggle under domain shift and lack interpretability. To address these issues, recent work has explored safety-aware learning xiao2023barriernet and robust generalization techniques, such as liquid neural networks chahine2023robust , Gaussian Splatting quach2024gaussiansplattingrealworld , and model-driven adaptation wang2023learning ; Yin2023 ; swift2023 . Moreover, emerging paradigms now integrate foundation models to enable flexible, instruction-conditioned policies chahine2024flexendtoendtextinstructedvisual , and scalable internet-scale perception backbones for real-time deployment maalouf2024follow . Goal Assignment. The Linear Sum Assignment (LSA) problem kuhnLSA is a core formulation for multi-agent matching, with centralized solutions like the Hungarian algorithm widely used in robotics. Scalable extensions have been explored for team coordination aziz2022taskallocationusingteam , while decentralized alternatives address limited communication using Lyapunov-based methods decgol , distributed Hungarian approximations dechun , and iterative consensus protocols dishun . More recently, learning-based approaches have been developed to approximate LSA solutions. Deep neural networks have shown early promise DNNLSA , and graph neural networks (GNNs) offer a structured and scalable alternative for decentralized assignment. GLAN liu2022glangraphbasedlinearassignment frames the problem as bipartite edge selection, while subsequent works tackling ; gnndeclsa demonstrate near-optimal decentralized goal allocation across varied team sizes and conditions, making them well suited to real-world swarm deployment. Visual Registration. Registration techniques are fundamental for aligning observations across distributed agents. Classical approaches include the Iterative Closest Point (ICP) algorithm chen1992object , which aligns point clouds via nearest-neighbor matching and rigid transformations. Probabilistic extensions such as Gaussian Mixture Models (GMMs) joshi1995problem and Coherent Point Drift (CPD) myronenko2010point incorporate uncertainty and soft matching. Recent methods leverage learned representations wang2019deep or structural constraints such as graph or geometric features agarwal2006bipartite . These methods inform the design of our Box-ICP module, which incorporates structure-aware matching across bounding box edge sets in a decentralized visual registration setting. Consensus and Scalable Coordination. Decentralized coordination frameworks have addressed robust consensus protocols consensus , decentralized formation flying ffgps1 , and scalable local game-theoretic planning localchahine ; intentionchahine . These approaches enable agents to plan safe trajectories with minimal communication, informing our design of low-bandwidth, robust multi-agent control. 3 Technical Approach The decentralized multi-agent wildlife monitoring pipeline proposed consists of six algorithmic components depicted in Figure 1 and presented below: 1. Scouting: Identify areas of potential ninadrdv and search with spiral trajectories. 2. Detection: Scan with vision model reis2024realtimeflyingobjectdetection and share group’s GPS coordinates. 3. Rendezvous: Guide swarm to flock via decentralized formation control ffgps1 . 4. Registration: Reach visual ID consensus on detection bounding boxes reis2024realtimeflyingobjectdetection . 5. Goal Assignment: Infer goals decentralized GNN-LSA gnndeclsa . 6. Execution: Perform visual monitoring via segmentation and tracking maalouf2024follow . Our main contribution is the design of a novel, unified, cohesive vision-based system that encapsulates everything from scouting to individual task execution. A cornerstone of the pipeline is the visual detection module, trained on a curated dataset of real whale imagery obtained from field footage in the ocean as well as overhead shots of printed posters over land, augmenting the dataset for robust detection across different conditions. Detection is relied upon by the scout when looking for whales in a scene, but more importantly by the swarm of agents who use the produced bounding boxes as input to the core individual registration algorithm which enables fine grained goal assignment. Scouting, Detection, Rendezvous: Scouting is achieved by deploying the scout agent to follow a predefined search pattern, e.g. growing spiral trajectory, around an area of high potential (either guessed by field teams or informed by sensing ninadrdv ). The scout runs the detection model and, upon obtaining a robust whale presence signal (80% of frames with detections in a buffer of recent images), is directed to visually track the filtered barycenter of all bounding boxes. Its current position is used as a target for the decentralized formation flying swarm. Critical to our design are the registration and goal assignment modules, both of which operate under strict communication constraints using only visual input. To ensure scalability and reliability, we restrict inter-agent communication to a ring topology, where each drone communicates only with its two immediate neighbors. In fact, we focus our technical contributions to vision-based decentralized registration and goal assignment. Algorithm 1 Box-ICP 0: Boxes \mathcal{B}_{1}=\{B_{1}^{i}\}_{i=1}^{N_{1}} , \mathcal{B}_{2}=\{B_{2}^{j}\}_{j=1}^{N_{2}} 0: Transformation T\in\mathbb{R}^{4\times 4} , Matching \sigma:\{1,\ldots,N_{1}\}\to\{1,\ldots,N_{2}\} 1: Initialize T\leftarrow I_{4} 2: while not converged do 3: C\leftarrow 0\in\mathbb{R}^{N_{1}\times N_{2}} 4: for i=1 to N_{1} do 5: for j=1 to N_{2} do 6: Compute D\in\mathbb{R}^{4\times 4} :

Read the complete text