A Multi-Agent Feedback System for Detecting and Describing News Events in Satellite Imagery
Madeline Anderson, Mikhail Klassen, Ash Hoover, Kerri Cahoy · 2026
Read A Multi-Agent Feedback System for Detecting and Describing News Events in Satellite Imagery by Madeline Anderson, Mikhail Klassen, Ash Hoover, Kerri Cahoy (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-12772.txt.
Introduction
Madeline Anderson and three colleagues find news events in satellite imagery with agentic feedback. Multi-temporal data are lacking in remote sensing. Finding events and labeling sequences take heavy time and labor. SkyScraper tags news articles and writes captions for image sequences. SkyScraper finds five times more events than standard methods. A new set holds 5,000 items. Linking imagery to news events supports journalism.
Catalog record
| CREATOR | Madeline Anderson, Mikhail Klassen, Ash Hoover, Kerri Cahoy |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 8348 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.12772
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Multi-Agent Feedback System for Detecting and Describing News Events in Satellite Imagery
Madeline Anderson, Mikhail Klassen, Ash Hoover, Kerri Cahoy
Abstract
Changes in satellite imagery often occur over multiple time steps. Despite the emergence of bi-temporal change captioning datasets, there is a lack of multi-temporal event captioning datasets (at least two images per sequence) in remote sensing. This gap exists because (1) searching for visible events in satellite imagery and (2) labeling multi-temporal sequences require significant time and labor. To address these challenges, we present SkyScraper, an iterative multi-agent workflow that geocodes news articles and synthesizes captions for corresponding satellite image sequences. Our experiments show that SkyScraper successfully finds 5x more events than traditional geocoding methods, demonstrating that agentic feedback is an effective strategy for surfacing new multi-temporal events in satellite imagery. We apply our framework to a large database of global news articles, curating a new multi-temporal captioning dataset with 5,000 sequences. By automatically identifying imagery related to news events, our work also supports journalism and reporting efforts.
Introduction
Traditional remote sensing change captioning methods typically rely on manual annotation or rules-based approaches applied to existing datasets with fine-grained change labels (e.g., segmentation maps). Because labels are scarce, these methods usually operate on bi-temporal image pairs focusing on land-use land-cover (LULC) changes, such as building and road modifications
(
8
;
4
;
6
;
12
)
. Recent work incorporates Large Language Models (LLMs) to improve scalability and caption diversity
(
2
;
9
;
5
;
3
)
. However, these approaches still rely on pre-labeled temporal datasets and inherit similar limitations. Multi-temporal captioning datasets (with more than two images) remain largely restricted to UAV video
(
1
)
.
Expanding current datasets remains difficult because surfacing
new
change events in satellite imagery and annotating multi-temporal image sequences demand significant time and labor. Building on recent work that curates a multi-temporal dataset by geocoding FEMA-sourced disaster reports
(
10
)
, we generalize the approach to global news articles and introduce multi-agent feedback for event geocoding and captioning, producing a new multi-temporal captioning dataset spanning diverse global events.
Traditional Geocoding
For rules-based geocoding methods, we first extract all named geographic entities from the article, each with associated geocoded coordinates
c_{i}=(\phi_{i},\lambda_{i})
and weight
w_{i}
, which is the number of times that name appears in the article. We experiment with two traditional geocoding methods: (1) weighted centroid, and (2) Geo-referenced Information Processing SYstem (GIPSY)
(
11
)
. The weighted centroid approach converts the coordinates to cartesian Earth-Centered, Earth-Fixed (ECEF) format, then computes their weighted average. Alternatively, GIPSY is a weighted polygon-based approach that represents each candidate location as a 3D polyhedron whose base corresponds to its geographic bounding box, and whose height,
z_{i}
, corresponds with its weight
w_{i}
. It then overlays the polyhedra based on the following criteria:
1.
No overlap: place at
z=0
.
2.
Fully contained: stack on top of the containing polygon.
3.
Partial overlap: place non-overlapping part at
z=0
, stack overlapping parts on intersecting regions.
After adding all polygons, GIPSY estimates the article location using the centroid of the highest-elevation region.
Approach
Traditional geocoding methods are susceptible to noise, such as incorrect location names and geocoding inaccuracies. Moreover, the frequency that an article mentions a location (used as the weighting factor) does not always correlate with event occurrence
and
visibility at that location.
SkyScraper addresses these limitations using an agentic iterative approach (Figure
1
) with the following steps:
1.
Extract:
An LLM
article agent
extracts a named geographic entity and event timeline from the article text.
2.
Geocode:
A geocoding API converts named geographic entities to latitude-longitude coordinates.
3.
Fetch:
A data API retrieves satellite imagery at the geocoded coordinates over the timeline range.
4.
Verify:
A multimodal LLM
verifier agent
cross-references the article and fetched imagery to verify whether the event is visible while providing reasoning.
5.
Caption:
A multimodal LLM
captioning agent
writes a change caption, using the article as context.
Figure 1:
SkyScraper iterative feedback pipeline.
Rather than extracting all location names at once, the system requests one candidate location at a time and repeats steps 1-4 to refine searches if (1) geocoding fails or (2) the verifier agent does not detect the event. In each case, it requests a new candidate, incorporating the failed location and reasoning (Algorithm
1
). If it does not find the event before reaching the maximum attempts, it returns an empty result.
Algorithm 1
Iterative Search and Verification
0:
Article text
T
, maximum attempts
MAX_ATTEMPTS
1:
attempts\leftarrow 0
2:
failures\leftarrow
empty list
3:
\ell^{*}\leftarrow
None
4:
while
attempts<
MAX_ATTEMPTS
do
5:
attempts\leftarrow attempts+1
6:
\ell_{t}\leftarrow
ArticleAgent(
T
,
failures
)
7:
c_{t}\leftarrow
GeocodeAPI(
\ell_{t}
)
8:
if
c_{t}
is invalid
then
9:
Append(
failures
, (
\ell_{t}
,
geocode_error
))
10:
continue
11:
else
12:
D_{t}\leftarrow
DataAPI(
c_{t}
)
13:
(visible,reason)\leftarrow
VerifierAgent(
T
,
D_{t}
)
14:
if
visible=\texttt{false}
then
15:
Append(
failures
, (
\ell_{t}
,
reason
))
16:
continue
17:
else
18:
\ell^{*}\leftarrow\ell_{t}
19:
CaptioningAgent(
T
,
D_{t}
)
20:
break
21:
end
if
22:
end
if
23:
end
while
Experiments
We collected 1,000 news articles using the Google Search API, covering diverse global events such as natural disasters, construction, and civil unrest. We applied each geocoding approach (weighted centroid, GIPSY, and SkyScraper agentic feedback) to retrieve PlanetScope imagery (3m resolution). We used Mapbox as the geocoding API and Gemini-2.5-flash for the LLM agents.
We manually verified positive event detections for each approach. Table
1
reports the geocoding performance, where
yield
represents the percentage of correct detections (true positives) out of the initial set of articles. Because we only count true positives, articles without visible events reduce raw yield, making relative improvement over the centroid baseline more informative than absolute yield.
Method
Event
Detections
Yield
Increase over
Centroid Baseline
Weighted Centroid
17
1.7%
–
GIPSY
47
4.7%
2.8
\times
SkyScraper (ours)
84
8.4%
4.9
\times
Table 1:
Event detection yield and relative improvement over centroid baseline for news article geocoding.
Dataset Curation
We applied SkyScraper to news articles sampled from the Global Database of Events, Language, and Tone
1
1
1
https://www.gdeltproject.org/
(GDELT)
(
7
)
from 2022–2024 using PlanetScope imagery. Annotators verified captions and event dates to produce the final datasets (Table
2
). Figure
2
shows a detected tornado event. We generated a Sentinel-2 version of the dataset, available at
https://tinyurl.com/skyscraper-s2
.
Imagery
Source
Total
Sequences
Confirmed
Events
Average # Images
Per Sequence
PlanetScope
5,000
3,288
21
Sentinel-2
4,463
2,931
8
Table 2:
SkyScraper GDELT dataset statistics.
Figure 2:
PlanetScope imagery and caption for a geocoded tornado event identified by SkyScraper.
Conclusion
We introduce SkyScraper, a novel multi-agent feedback system that locates and captions events in multi-temporal satellite imagery using news articles. Compared to traditional geocoding methods, our approach increases event detections by nearly 5x. We apply SkyScraper to the GDELT database to produce new PlanetScope and Sentinel-2 multi-temporal captioning datasets with
\sim