Platonic Representations for Poverty Mapping: Unified Vision–Language Codes or Agent–Induced Novelty?

Satiyabooshan Murugaboopathy, Fraunhofer Center, Adel Daoud, Chalmers & Linköping University · 2025

Read Platonic Representations for Poverty Mapping: Unified Vision–Language Codes or Agent–Induced Novelty? by Satiyabooshan Murugaboopathy, Fraunhofer Center, Adel Daoud, Chalmers & Linköping University (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-01109.txt.

Introduction

Satiyabooshan Murugaboopathy, Connor Jerzak, and Adel Daoud test whether wealth leaves traces in both photos of the Earth and in text. Their paper is from two thousand twenty-five. Aid groups need maps of poverty, but surveys are slow and costly. Satellite images show roofs and roads, while web text holds local history and description.

The team paired satellite views of African neighborhoods with text from the web and from language models. Five pipelines then guessed household wealth, using images alone, text alone, or both. Mixing images with text beat images alone by a clear margin. The model's built-in knowledge of places also guessed wealth surprisingly well. Image and text codes lined up only in part, the authors caution.

Catalog record

CREATORSatiyabooshan Murugaboopathy, Fraunhofer Center, Adel Daoud, Chalmers & Linköping University
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-01109

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Platonic Representations for Poverty Mapping: Unified Vision–Language Codes or Agent–Induced Novelty? Satiyabooshan Murugaboopathy, Fraunhofer Center, Adel Daoud, Chalmers & Linköping University Abstract We here investigate whether socio-economic indicators, such as household wealth, leave recoverable informational imprints in both satellite imagery (capturing physical features like buildings and roads) and Internet-sourced text (reflecting historical, cultural, and economic narratives of neighborhoods). Using Demographic and Health Survey (DHS) data from African neighborhoods (clusters), we pair high-resolution Landsat images with textual descriptions generated by large language models (LLMs) conditioned on location and year, as well as text retrieved by an LLM-driven AI Search Agent from web sources. We develop a multimodal framework that predicts household wealth (measured by the International Wealth Index (IWI)) through five pipelines: (i) a vision model on satellite images, (ii) an LLM using only location and year, (iii) an AI agent that searches and synthesizes web text, (iv) a joint image-text encoder, and (v) an ensemble of all signals. Our framework yields three contributions. First, evaluations show that fusing vision and agent/LLM-generated text outperforms vision-only baselines in wealth prediction (e.g., R 2 = 0.77 R^{2}=0.77 vs. 0.63 0.63 on out-of-sample splits), with LLM-internal knowledge (artificial neural memory) proving surprisingly more effective than agent-retrieved text, improving robustness to out-of-country and out-of-time generalization. Second, we find partial representational convergence: fused embeddings from vision and language modalities correlate moderately (median cosine similarity across modalities of about 0.60 after alignment), suggesting a shared latent code of material well-being while retaining complementary details, broadly consistent with the Platonic Representation Hypothesis. Although the superior performance of LLM-only text over agent-retrieved data challenges our Agent-Induced Novelty Hypothesis, modest gains from combining agent data in some splits offer weak support for the idea that agent-gathered information—emerging from dynamic interaction with the Internet—introduces a degree of unique representational structures not fully captured by static LLM knowledge. Third, we release a large-scale multimodal dataset comprising approximately 60,000 DHS clusters, each linked to satellite images, LLM-generated descriptions, and associated texts retrieved by AI agents. 1 Introduction In an era where the timely and accurate measurement of socio-economic disparities is crucial for effective policy-making, humanitarian aid, and sustainable development, traditional household surveys, such as the Demographic and Health Surveys (DHS), provide invaluable ground-truth data but face significant limitations in terms of scale, cost, and frequency ( DHS et al. 2013 ) . These surveys, conducted periodically across low- and middle-income countries, capture wealth metrics such as the International Wealth Index (IWI)—a composite measure of household assets and living conditions—but often leave significant gaps in geographic and temporal coverage, particularly in remote or rapidly changing regions ( Sakamoto et al. 2025 ) . To bridge these gaps, researchers have increasingly turned to remote sensing technologies, leveraging high-resolution Earth observation (EO) imagery from satellites, such as Landsat, to infer poverty patterns through visual cues, including infrastructure density, land use, and vegetation health ( Yeh et al. 2020 ; Pettersson et al. 2023 ; Kakooei et al. 2024a ; Burke et al. 2021 ) . While these vision-based approaches have shown promise, they are inherently limited by what can be “seen” from above, often missing nuanced socio-cultural, historical, or contextual factors that influence material well-being ( O’Brien 2023 ; Zhu et al. 2025 ) . The advent of large language models (LLMs) and multimodal AI systems offers a complementary pathway, enabling the extraction and synthesis of textual information from vast digital repositories, including web sources and encyclopedic knowledge ( Sarmadi et al. 2025 ) . This raises two interrelated questions: (1) To what extent can useful textual information about neighborhoods in low- and middle-income countries be recovered from LLMs’ artificial neural memory or found by AI Search Agents on the Internet? (2) If recoverable, how well can socio-economic status be distilled into a compact, latent representation across vision and language modalities, potentially converging toward a shared “Platonic” representation of material well-being ( Huh et al. 2024 ) ? Or, do modalities complement each other to enhance estimation? Figure 1: DAG representing the imprint of the poverty/wealth index Y_{i} on satellite imagery \mathbf{M}_{i} and textual data \mathbf{W}_{i} , processed by LLM’s neural weights and queried by AI Search Agent. \mathbf{W}_{i} is challenging to observe directly, hence colored in red. Instead, the goal is to reconstruct a faithful representation via the LLM’s neural memory or AI agent search capabilities. These texts are denoted \mathbf{W}_{i}^{\text{Desc}} for the LLM, and \mathbf{W}_{i}^{\text{Trace}} (raw search result) and \mathbf{W}_{i}^{\text{Summary}} (summary of search result) for the search agent. \mathbf{B}_{i} represents background factors. Figure 1 illustrates these questions via a directed acyclic graph (DAG), where the materialization of poverty/wealth index Y_{i} in African neighborhoods causally influences both satellite imagery \mathbf{M}_{i} and textual data \mathbf{W}_{i} . Socio-economic conditions leave observable traces in the physical environment (e.g., building density, road networks) visible in EO data, as well as in linguistic artifacts (e.g., historical accounts, news reports) ( Daoud et al. 2019 ; Daoud and Dubhashi 2023 ) . However, \mathbf{W}_{i} is challenging to observe directly, necessitating LLMs to reconstruct it from neural memory as \mathbf{W}_{i}^{\text{Desc}} , and AI Search Agents (ASA) to query the Internet for \mathbf{W}_{i}^{\text{Trace}} (raw traces) and thereafter generate \mathbf{W}_{i}^{\text{Summary}} (summaries). Despite growing interest in multimodality in poverty mapping, systematic investigations remain scarce, hampered by the lack of aligned, large-scale datasets ( Lamichhane et al. 2025 ; Kakooei et al. 2024b ) . To address this limitation, we here analyze whether household wealth on the African continent, as measured by DHS IWI scores, can be textually and visually reconstructed and encoded in a joint latent space. Drawing on a continent-scale corpus of more than 60,000 DHS clusters across Africa from 1990 to 2020, we pair high-resolution, cloud-free Landsat composites with LLM-generated spatiotemporal narratives and AI-agent-retrieved contextual information. At its heart, our approach taps five different signals to “see” and describe each neighborhood’s wealth. First, we use satellite images to capture physical clues—things like roads, buildings, and vegetation. Next, we ask a language model to imagine a narrative for the place based only on its location and year. Then, an AI agent goes online, gathers real-world text about the area, and boils it down into a concise summary. After that, we train a joint encoder to blend the visual and textual cues into a single shared representation. Finally, we let an ensemble weigh and combine all five perspectives before we analyze embedding spaces to investigate representational convergence and complementarity across modalities. Figure 2 provides a visual overview. The remainder of the paper is organized as follows: We first review the related work on remote sensing and AI in poverty mapping. Next, we detail our agent and data curation frameworks, followed by an experimental performance and representational analysis, and conclude with implications, limitations, and future directions. 2 Problem Setup & Related Work Early efforts in remote sensing for poverty estimation used nighttime light imagery as a proxy for economic activity ( Elvidge et al. 2009 ) . More recent advances have focused on daytime satellite imagery combined with deep learning techniques. For example, Jean et al. 2016 demonstrated the use of transfer learning from night lights to predict poverty from daytime images in African countries. Subsequent works have applied convolutional neural networks directly to satellite data

Read the complete text