A Multi-Agent Feedback System for Detecting and Describing News Events in Satellite Imagery

Madeline Anderson, Mikhail Klassen, Ash Hoover, Kerri Cahoy · 2026

Read A Multi-Agent Feedback System for Detecting and Describing News Events in Satellite Imagery by Madeline Anderson, Mikhail Klassen, Ash Hoover, Kerri Cahoy (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-12772.txt.

Introduction

Madeline Anderson and three colleagues find news events in satellite imagery with agentic feedback. Multi-temporal data are lacking in remote sensing. Finding events and labeling sequences take heavy time and labor. SkyScraper tags news articles and writes captions for image sequences. SkyScraper finds five times more events than standard methods. A new set holds 5,000 items. Linking imagery to news events supports journalism.

Catalog record

CREATORMadeline Anderson, Mikhail Klassen, Ash Hoover, Kerri Cahoy
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE8348
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.12772

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Multi-Agent Feedback System for Detecting and Describing News Events in Satellite Imagery
Madeline Anderson, Mikhail Klassen, Ash Hoover, Kerri Cahoy

Abstract
Changes in satellite imagery often occur over multiple time steps. Despite the emergence of bi-temporal change captioning datasets, there is a lack of multi-temporal event captioning datasets (at least two images per sequence) in remote sensing. This gap exists because (1) searching for visible events in satellite imagery and (2) labeling multi-temporal sequences require significant time and labor. To address these challenges, we present SkyScraper, an iterative multi-agent workflow that geocodes news articles and synthesizes captions for corresponding satellite image sequences. Our experiments show that SkyScraper successfully finds 5x more events than traditional geocoding methods, demonstrating that agentic feedback is an effective strategy for surfacing new multi-temporal events in satellite imagery. We apply our framework to a large database of global news articles, curating a new multi-temporal captioning dataset with 5,000 sequences. By automatically identifying imagery related to news events, our work also supports journalism and reporting efforts.



Introduction

Traditional remote sensing change captioning methods typically rely on manual annotation or rules-based approaches applied to existing datasets with fine-grained change labels (e.g., segmentation maps). Because labels are scarce, these methods usually operate on bi-temporal image pairs focusing on land-use land-cover (LULC) changes, such as building and road modifications
(
8
;
4
;
6
;
12
)
. Recent work incorporates Large Language Models (LLMs) to improve scalability and caption diversity
(
2
;
9
;
5
;
3
)
. However, these approaches still rely on pre-labeled temporal datasets and inherit similar limitations. Multi-temporal captioning datasets (with more than two images) remain largely restricted to UAV video
(
1
)
.

Expanding current datasets remains difficult because surfacing
new
change events in satellite imagery and annotating multi-temporal image sequences demand significant time and labor. Building on recent work that curates a multi-temporal dataset by geocoding FEMA-sourced disaster reports
(
10
)
, we generalize the approach to global news articles and introduce multi-agent feedback for event geocoding and captioning, producing a new multi-temporal captioning dataset spanning diverse global events.

Traditional Geocoding

For rules-based geocoding methods, we first extract all named geographic entities from the article, each with associated geocoded coordinates
c_{i}=(\phi_{i},\lambda_{i})
and weight
w_{i}
, which is the number of times that name appears in the article. We experiment with two traditional geocoding methods: (1) weighted centroid, and (2) Geo-referenced Information Processing SYstem (GIPSY)
(
11
)
. The weighted centroid approach converts the coordinates to cartesian Earth-Centered, Earth-Fixed (ECEF) format, then computes their weighted average. Alternatively, GIPSY is a weighted polygon-based approach that represents each candidate location as a 3D polyhedron whose base corresponds to its geographic bounding box, and whose height,
z_{i}
, corresponds with its weight
w_{i}
. It then overlays the polyhedra based on the following criteria:

1.

No overlap: place at
z=0
.

2.

Fully contained: stack on top of the containing polygon.

3.

Partial overlap: place non-overlapping part at
z=0
, stack overlapping parts on intersecting regions.

After adding all polygons, GIPSY estimates the article location using the centroid of the highest-elevation region.

Approach

Traditional geocoding methods are susceptible to noise, such as incorrect location names and geocoding inaccuracies. Moreover, the frequency that an article mentions a location (used as the weighting factor) does not always correlate with event occurrence
and
visibility at that location.
SkyScraper addresses these limitations using an agentic iterative approach (Figure
1
) with the following steps:

1.

Extract:
An LLM
article agent
extracts a named geographic entity and event timeline from the article text.

2.

Geocode:
A geocoding API converts named geographic entities to latitude-longitude coordinates.

3.

Fetch:
A data API retrieves satellite imagery at the geocoded coordinates over the timeline range.

4.

Verify:
A multimodal LLM
verifier agent
cross-references the article and fetched imagery to verify whether the event is visible while providing reasoning.

5.

Caption:
A multimodal LLM
captioning agent
writes a change caption, using the article as context.

Figure 1:
SkyScraper iterative feedback pipeline.

Rather than extracting all location names at once, the system requests one candidate location at a time and repeats steps 1-4 to refine searches if (1) geocoding fails or (2) the verifier agent does not detect the event. In each case, it requests a new candidate, incorporating the failed location and reasoning (Algorithm
1
). If it does not find the event before reaching the maximum attempts, it returns an empty result.

Algorithm 1

Iterative Search and Verification

0:

Article text
T
, maximum attempts
MAX_ATTEMPTS

1:

attempts\leftarrow 0

2:

failures\leftarrow
empty list

3:

\ell^{*}\leftarrow

None

4:

while

attempts<

MAX_ATTEMPTS

do

5:

attempts\leftarrow attempts+1

6:

\ell_{t}\leftarrow
ArticleAgent(
T
,
failures
)

7:

c_{t}\leftarrow
GeocodeAPI(
\ell_{t}
)

8:

if

c_{t}
is invalid
then

9:

Append(
failures
, (
\ell_{t}
,
geocode_error
))

10:

continue

11:

else

12:

D_{t}\leftarrow
DataAPI(
c_{t}
)

13:

(visible,reason)\leftarrow
VerifierAgent(
T
,
D_{t}
)

14:

if

visible=\texttt{false}

then

15:

Append(
failures
, (
\ell_{t}
,
reason
))

16:

continue

17:

else

18:

\ell^{*}\leftarrow\ell_{t}

19:

CaptioningAgent(
T
,
D_{t}
)

20:

break

21:

end

if

22:

end

if

23:

end

while

Experiments

We collected 1,000 news articles using the Google Search API, covering diverse global events such as natural disasters, construction, and civil unrest. We applied each geocoding approach (weighted centroid, GIPSY, and SkyScraper agentic feedback) to retrieve PlanetScope imagery (3m resolution). We used Mapbox as the geocoding API and Gemini-2.5-flash for the LLM agents.

We manually verified positive event detections for each approach. Table
1
reports the geocoding performance, where
yield
represents the percentage of correct detections (true positives) out of the initial set of articles. Because we only count true positives, articles without visible events reduce raw yield, making relative improvement over the centroid baseline more informative than absolute yield.

Method

Event

Detections

Yield

Increase over

Centroid Baseline

Weighted Centroid

17

1.7%

–

GIPSY

47

4.7%

2.8
\times

SkyScraper (ours)

84

8.4%

4.9
\times

Table 1:
Event detection yield and relative improvement over centroid baseline for news article geocoding.

Dataset Curation

We applied SkyScraper to news articles sampled from the Global Database of Events, Language, and Tone
1
1

1

https://www.gdeltproject.org/
(GDELT)
(
7
)
from 2022–2024 using PlanetScope imagery. Annotators verified captions and event dates to produce the final datasets (Table
2
). Figure
2
shows a detected tornado event. We generated a Sentinel-2 version of the dataset, available at
https://tinyurl.com/skyscraper-s2
.

Imagery

Source

Total

Sequences

Confirmed

Events

Average # Images

Per Sequence

PlanetScope

5,000

3,288

21

Sentinel-2

4,463

2,931

8

Table 2:
SkyScraper GDELT dataset statistics.

Figure 2:
PlanetScope imagery and caption for a geocoded tornado event identified by SkyScraper.

Conclusion

We introduce SkyScraper, a novel multi-agent feedback system that locates and captions events in multi-temporal satellite imagery using news articles. Compared to traditional geocoding methods, our approach increases event detections by nearly 5x. We apply SkyScraper to the GDELT database to produce new PlanetScope and Sentinel-2 multi-temporal captioning datasets with
\sim

Read the complete text