NewsRECON: News article REtrieval for image CONtextualization
Jonathan Tonglet, Iryna Gurevych, Tinne Tuytelaars, Marie-Francine Moens · 2026
Read NewsRECON: News article REtrieval for image CONtextualization by Jonathan Tonglet, Iryna Gurevych, Tinne Tuytelaars, Marie-Francine Moens (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-14121.txt.
Introduction
Jonathan Tonglet and co-authors help journalists pin down where and when a news photo was taken in 2026. Identifying a news image's context is crucial for credible stories. It also helps debunk misinformation that recycles old photos. Many existing methods rely on reverse image search engines.
These tools often fail to return results, limiting practical use. The team addresses the hard scenario where reverse search evidence is unavailable. Their NewsRECON retrieves news articles to contextualize an image instead. Article matches can date and place a photo when search engines cannot.
Catalog record
| CREATOR | Jonathan Tonglet, Iryna Gurevych, Tinne Tuytelaars, Marie-Francine Moens |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-14121
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
NewsRECON: News article REtrieval for image CONtextualization Jonathan Tonglet, Iryna Gurevych, Tinne Tuytelaars, Marie-Francine Moens Abstract Identifying when and where a news image was taken is crucial for journalists and forensic experts to produce credible stories and debunk misinformation. While many existing methods rely on reverse image search (RIS) engines, these tools often fail to return results, thereby limiting their practical applicability. In this work, we address the challenging scenario where RIS evidence is unavailable. We introduce NewsRECON, a method that links images to relevant news articles to infer their date and location from article metadata. NewsRECON leverages a corpus of over 90,000 articles and integrates: (1) a bi-encoder for retrieving event-relevant articles; (2) two cross-encoders for reranking articles by location and event consistency. Experiments on the TARA and 5Pils-OOC show that NewsRECON outperforms prior work and can be combined with a multimodal large language model to achieve new SOTA results in the absence of RIS evidence. We make our code available. 1 1 1 github.com/jtonglet/arxiv2025-newsrecon 1 Introduction Images are frequently used to communicate news events, especially on social media platforms. However, they are often shared with missing or incorrect metadata, such as their date or location. These inaccuracies play a significant role in the spread of online misinformation ( Dufour et al. 2024 ) . As a result, journalists, fact-checkers, and forensic experts must frequently verify or recover the metadata of online images, i.e., image contextualization , a task that is crucial for several applications, including reliable news coverage, forensic investigation of war crimes ( Silverman 2013 ) , or early misinformation detection ( Puliafito 2025 ; Khan et al. 2024 ) . A common approach involves using reverse image search (RIS) tools to retrieve web pages containing the image, and inferring the date or location from the surrounding page content. However, RIS engines often fail to return results ( Tonglet et al. 2024 ; Tonglet et al. 2025 ) . In such cases, identifying the image’s date and location becomes substantially more difficult, requiring journalists to interpret visual cues and gather external evidence. Due to time and human resource constraints, these cases are also more interesting to automate. Figure 1: NewsRECON retrieves articles that are location or event-relevant, given an image as query. Several works have proposed methods to automate image contextualization ( Fu et al. 2022 ; Shi et al. 2024 ; Tonglet et al. 2024 ; Ayyubi et al. 2025 ; Phan et al. 2025 ; Siingh et al. 2025 ; Tonglet et al. 2025 ) . Some of them tackle the challenging scenario where RIS engines are unavailable, relying solely on the visual content of the image ( Fu et al. 2022 ; Siingh et al. 2025 ) or leveraging encyclopedic knowledge bases ( Shi et al. 2024 ) . However, none of these methods leverage large-scale news article corpora, which represent a rich source of contextual evidence ( Fu et al. 2022 ) . In this work, we introduce NewsRECON, the first image contextualization method leveraging news articles in scenarios where RIS engines are unavailable. As illustrated in Figure 1 , NewsRECON searches a large news corpus to identify relevant articles for a given news image, i.e., articles that relate to the same location or event, where an event is defined as the combination of a date and location. The metadata of these articles, including publication dates and geolocation keywords, can then be used as predictions of the image’s date and location. We construct a corpus of 91,376 news articles, spanning the years 2010 to 2023. NewsRECON comprises three main components: (1) a bi-encoder that retrieves the top- K event-relevant articles for a given image, (2) a cross-encoder that reranks these articles based on their location similarity to the image, (3) a second cross-encoder that reranks clusters of articles by their event consistency with the image. Experiments on the TARA benchmark show that NewsRECON outperforms retrieval-based baselines ( Fu et al. 2022 ) by 9.7 percentage points (pp) in GREAT score ( Siingh et al. 2025 ) , a joint metric for date and location prediction. Moreover, providing the top-3 articles as input to a multimodal large language model (MLLM) yields the new SOTA results. We further demonstrate that NewsRECON, after being trained on TARA, generalizes to the 5Pils-OOC benchmark ( Tonglet et al. 2025 ) , despite geographic and stylistic shifts. When combined with an MLLM, it becomes the strongest method for images lacking RIS web evidence. 2 Related work Journalists and forensic experts contextualize news images by identifying key context items such as the source, date, location, and motivation ( Urbani 2020 ; Tonglet et al. 2024 ; Phan et al. 2025 ) . Applications of this process include fact-checking images that have been taken out of context to spread misinformation ( Tonglet et al. 2025 ) , or identifying evidence for war crimes investigations ( Silverman 2013 ) . In this work, we focus on two core context items: the date and location, which are the most widely studied in the literature. Several datasets have been proposed for this task ( Fu et al. 2022 ; Tonglet et al. 2024 ; Geng et al. 2025 ) . Existing methods differ in both their model architectures and the types of external evidence they use. Some approaches formulate image contextualization as a classification task, fine-tuning CLIP ( Radford et al. 2021 ) to retrieve the best matching candidate date-location pairs present in the same dataset split ( Fu et al. 2022 ; Shi et al. 2024 ) . However, this setup lacks realism: the candidate set is limited to a small number of date-location pairs found in the split, whereas in real-world scenarios, there are thousands of unique dates and an even larger number of locations, resulting in a near-infinite search space. Other works treat it as a text generation task and rely on (M)LLMs to generate the answers ( Tonglet et al. 2024 ; Phan et al. 2025 ; Zhang et al. 2024 ; Ayyubi et al. 2025 ; Siingh et al. 2025 ; Tonglet et al. 2025 ) . Some of these approaches use RIS engines to retrieve external evidence ( Tonglet et al. 2024 ; Phan et al. 2025 ; Tonglet et al. 2025 ) . However, RIS engines may return no results ( Abdelnabi et al. 2022 ; Tonglet et al. 2024 ) , particularly for newly published or less widely circulated images. The absence of RIS evidence makes the task far more challenging but also more valuable to automate, as it requires interpreting subtle visual cues and relating them to world knowledge. Other methods explore alternative sources of external evidence, such as Wikipedia ( Shi et al. 2024 ; Tonglet et al. 2025 ) or celebrity recognition APIs ( Siingh et al. 2025 ) . However, none of the existing methods leverage news articles, despite their strong potential as a source of temporal and spatial context ( Fu et al. 2022 ) . 3 NewsRECON Figure 2: Overview of NewsRECON. Parts a to c illustrate the three training stages with images from TARA ( Fu et al. 2022 ) . Part d shows the retrieval pipeline at inference