LiveWeb-IE : A Benchmark For Online Web Information Extraction
Seungbin Yang, Jihwan Kim, Jaemin Choi, Dongjin Kim, Soyoung Yang, ChaeHun Park, Jaegul Choo · 2026
Read LiveWeb-IE : A Benchmark For Online Web Information Extraction by Seungbin Yang, Jihwan Kim, Jaemin Choi, Dongjin Kim, Soyoung Yang, ChaeHun Park, Jaegul Choo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-13773.txt.
Introduction
Seungbin Yang and co-authors introduce LiveWeb-IE, a benchmark for online web information extraction. Prior benchmarks use HTML snapshots from a single point in time, which misses how the web evolves. Performance on static benchmarks often fails to generalize to dynamic real-world settings, they note. LiveWeb-IE evaluates systems directly against live, permission-granted websites with natural language queries. Queries span text, images, and hyperlinks at four complexity levels. They also propose Visual Grounding Scraper, a multi-stage agentic framework that visually narrows page content to extract data.
Catalog record
| CREATOR | Seungbin Yang, Jihwan Kim, Jaemin Choi, Dongjin Kim, Soyoung Yang, ChaeHun Park, Jaegul Choo |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 88339 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.13773
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LiveWeb-IE : A Benchmark For Online Web Information Extraction Seungbin Yang, Jihwan Kim, Jaemin Choi, Dongjin Kim, Soyoung Yang, ChaeHun Park, Jaegul Choo Abstract Web information extraction (WIE) is the task of automatically extracting data from web pages, offering high utility for various applications. The evaluation of WIE systems has traditionally relied on benchmarks built from HTML snapshots captured at a single point in time. However, this offline evaluation paradigm fails to account for the temporally evolving nature of the web; consequently, performance on these static benchmarks often fails to generalize to dynamic real-world scenarios. To bridge this gap, we introduce LiveWeb-IE , a new benchmark designed for evaluating WIE systems directly against live websites. Based on trusted and permission-granted websites, we curate natural language queries that require information extraction of various data categories, such as text, images, and hyperlinks. We further design these queries to represent four levels of complexity, based on the number and cardinality of attributes to be extracted, enabling a granular assessment of WIE systems. In addition, we propose Visual Grounding Scraper (VGS), a novel multi-stage agentic framework that mimics human cognitive processes by visually narrowing down web page content to extract desired information. Extensive experiments across diverse backbone models demonstrate the effectiveness and robustness of VGS. We believe that this study lays the foundation for developing practical and robust WIE systems 1 1 1 Our dataset is available in https://github.com/sbY99/LiveWeb-IE . 1 Introduction With the growth of data on the web, automatic information extraction from web pages has become crucial for a wide range of applications, such as large-scale information analysis and decision-making ( Crescenzi et al. 2001 ; Thapelo et al. 2021 ; Li et al. 2023 ; Pichiyan et al. 2023 ) . The conventional approach to web information extraction (WIE) has been dominated by wrapper-based methods, which define a set of extraction rules based on the structural patterns of web pages ( Lerman et al. 2003 ; Reis et al. 2004 ; Omari et al. 2017 ) . With the advent of large language models (LLMs), language-agent-based methods have been proposed, leveraging LLMs to extract information directly from web pages ( UncleCode 2024 ; Lorenzo Padoan 2024 ) . However, both approaches face significant limitations; wrapper-based methods are often brittle and require substantial human effort to create and maintain Dalvi et al. 2009 , while using LLMs directly as extractors is impractical for large-scale scraping tasks due to the substantial costs incurred by processing each web page individually. To address this problem, hybrid approaches that leverage LLMs to generate reusable wrappers have emerged as a promising solution ( Huang et al. 2024 ; Kaur 2025 ) . Despite these advancements, it is unclear whether the success of WIE systems on existing benchmarks is a reliable indicator of their performance in real-world scenarios. While existing benchmarks rely on static HTML snapshots, real-world web pages frequently change their layouts and structures over time ( Hao et al. 2011 ; Bronzi et al. 2013 ; Lockard et al. 2019 ) . Given the strong dependence of WIE performance on the structural properties of web pages ( Zheng et al. 2007 ; Wang et al. 2022 ) , performance measured on these offline benchmarks, which fail to capture such temporal shifts, may not correlate with efficacy on the live websites ( Mialon et al. 2023 ; Pan et al. 2024 ) . Our empirical analysis in Appendix F confirms this discrepancy, showing that LLM-based wrapper generation methods suffer an average F1 degradation of over 15% when applied to the structurally evolved versions of the same websites compared to their static snapshots. Figure 1: Overview of the conventional WIE paradigm’s limitations and our solutions involving a new benchmark and an extraction method. (A) While existing offline benchmarks are constructed from static HTML snapshots, LiveWeb-IE evaluates WIE systems on live websites to reflect the evolving nature of the web. (B) On a complex live web page, methods that process full HTML often fail, whereas VGS leverages visual cues from the rendered page for accurate information extraction. To fill this gap, we introduce LiveWeb-IE , a benchmark for online web information extraction designed with four key features: (1) Live web evaluation. While conventional benchmarks evaluate WIE systems on static HTML snapshots, LiveWeb-IE mandates evaluation directly on live websites. By requiring WIE systems to access the target URL, LiveWeb-IE enables an assessment of their performance on the live web page at the time of evaluation, as shown in Figure 1 -(A). (2) Diverse and reliable websites. LiveWeb-IE is constructed from 15 permission-granted websites across 8 diverse domains. We select websites by considering high content stability, a crucial factor for ensuring the long-term validity of data annotation ( Mialon et al. 2023 ) . Specifically, we define this content stability as the temporal persistence of informational content, distinguishing it from layout stability. Our benchmark design ensures this content stability by curating queries for fact-based information that is unlikely to change. By targeting stable values within dynamic layouts, LiveWeb-IE enables a robust evaluation of a system’s ability to handle the web page structures at the moment of evaluation. (3) Query-driven WIE on diverse data categories. LiveWeb-IE follows an on-demand IE setup, allowing users to request information through free-form natural language queries ( Sekine 2006 ; Jiao et al. 2023 ; Qi et al. 2024 ) . In real-world scenarios, these queries are not limited to text; users often want to extract non-textual information, such as product images or event banners. However, existing benchmarks have focused on text-only extraction, failing to represent this full scope. LiveWeb-IE addresses this gap by comprising a diverse set of manually curated queries that require WIE systems to extract various data categories, including text, images, and hyperlinks, to closely align with these real-world scenarios. (4) Multi-dimensional task complexity. To support a comprehensive evaluation, LiveWeb-IE organizes extraction tasks based on two properties: the number of attributes and the cardinality of their values. This combination yields four complexity levels, from extracting a single attribute with a single value ( Type I ) to multiple attributes with lists of values ( Type IV ). Extensive experiments on LiveWeb-IE show that even methods with strong performance on existing benchmarks struggle significantly, especially on complex extraction tasks. We attribute this performance degradation to approaches that only rely on parsing HTML, as its inherently verbose nature makes it challenging to extract the desired information ( Bevendorff et al. 2023 ; He et al. 2024 ) . As the structure of web pages grows more complex over time ( Vogel & Springer 2022 ) , the evolving nature of the web further amplifies the performance degradation of such methods. Based on these observations, we propose Visual Grounding Scraper (VGS), a novel multi-stage framework that advances the promising paradigm of generating reusable wrappers. VGS emulates how humans find information on a web page: VGS first visually scans the web page to find the region where the information is, and then identifies the specific items within that region. This strategy bypasses the noise inherent in raw HTML by focusing on the rendered web page, thereby generating a reliable wrapper. Our method establishes state-of-the-art results not only on