Web Execution Bundles: Reproducible, Accurate, and Archivable Web Measurements
Florian Hantke † , Peter Snyder ‡ , Hamed Haddadi, Ben Stock † † \dagger CISPA Helmholtz Center for Information Security ‡ \ddagger Brave Software ⋆ \star Imperial College London [email protected] , [email protected] , [email protected] , [email protected] · 2025
Read Web Execution Bundles: Reproducible, Accurate, and Archivable Web Measurements by Florian Hantke † , Peter Snyder ‡ , Hamed Haddadi, Ben Stock † † \dagger CISPA Helmholtz Center for Information Security ‡ \ddagger Brave Software ⋆ \star Imperial College London [email protected] , [email protected] , [email protected] , [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-15911.txt.
Introduction
Florian Hantke and co-authors build a tool for measuring the web that others can trust and reuse in this 2025 paper. Web studies usually hand-roll their own crawlers, each of unknown accuracy. Their tool, WebREC, records browser events more faithfully than current options. A companion format, dot-web, archives the sessions for later replay.
Repeating famous studies showed WebREC matched the true baselines more closely. The team then checked generality against a survey of crawling research. Seven in ten surveyed papers could run on WebREC unchanged. Nearly half could work from dot-web archives alone, with no fresh crawl. The authors offer both as shared infrastructure for web science.
Catalog record
| CREATOR | Florian Hantke † , Peter Snyder ‡ , Hamed Haddadi, Ben Stock † † \dagger CISPA Helmholtz Center for Information Security ‡ \ddagger Brave Software ⋆ \star Imperial College London [email protected] , [email protected] , [email protected] , [email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 79 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-15911
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Web Execution Bundles: Reproducible, Accurate, and Archivable Web Measurements Florian Hantke † , Peter Snyder ‡ , Hamed Haddadi, Ben Stock † † \dagger CISPA Helmholtz Center for Information Security ‡ \ddagger Brave Software ⋆ \star Imperial College London [email protected] , [email protected] , [email protected] , [email protected] Abstract Recently, reproducibility has become a cornerstone in the security and privacy research community, including artifact evaluations [ 2 , 48 , 35 ] and even a new symposium topic [ 2 ] . However, Web measurements lack tools that can be reused across many measurement tasks without modification, while being robust to circumvention, and accurate across the wide range of behaviors in the Web. As a result, most measurement studies use custom tools and varied archival formats, each of unknown correctness and significant limitations, systematically affecting the research’s accuracy and reproducibility. To address these limitations, we present WebREC, a Web measurement tool that is, compared against the current state-of-the-art, accurate (i.e., correctly measures and attributes events not possible with existing tools), general (i.e., reusable without modification for a broad range of measurement tasks), and comprehensive (i.e., handling events from all relevant browser behaviors). We also present .web , an archival format for the accurate and reproducible measurement of a wide range of website behaviors. We empirically evaluate WebREC’s accuracy by replicating well-known Web measurement studies and showing that WebREC’s results more accurately match our baseline. We then assess if WebREC and .web succeed as general-purpose tools, which could be used to accomplish many Web measurement tasks without modification. We find that this is so: 70% of papers discussed in a 2024 web crawling SoK paper could be conducted using WebREC as is, and a larger number (48%) could be leveraged against .web archives without requiring any new crawling. 1 Introduction Security and privacy Web measurement is an enormous field that lacks accurate, general, and standardized measurement tools and archival formats needed for reproducible measurements and general-purpose datasets. We argue that existing tools in the field operate at the wrong level of abstraction and granularity. In the best of cases, this mismatch invites error; in many cases, it makes certain categories of error unavoidable. As a demonstrative example of how the most common current Web measurement tools and Web archive formats invite errors, consider the following example. Say a researcher wanted to measure how the Web behaved in 2010 (the first year the HTTPArchive project [ 72 ] provides archives). The common way such measurements are done is to use a browser automation framework like Puppeteer [ 17 ] to automate a browser and load a website from a structured HTTP archive such as a HAR file. Current implementations of Puppeteer use the current version of Chromium by default and are incompatible with very old browser versions, meaning that websites archived in 2010 will be executed in a 2024 (or later) version browser. One large change that occurred in the Web platform between 2010 and 2024 is that browsers changed how they handle certain “mixed content” resources, or insecure sub-resources included on a secure website. Before 2015, Chromium would load insecure JavaScript included on a secure page, but (in alignment with other browsers) after 2015, Chromium would not load it on a secure page. The net effect of this change is that measuring how the Web behaved in 2010 by using modern Chromium versions will cause several subtle but categorical measurement errors. It will under-count JavaScript sub-resource requests while over-counting the number of failed HTTP requests, among other errors. This is just one example. Current measurement tools can potentially cause many other errors of this kind, as well as other entirely different classes of errors. Using current archival formats for historical Web measurements will cause comparable problems. More broadly, the lack of accurate, general-purpose Web measurement tools and archive formats weakens the field of Web security and privacy measurement in at least the following ways: i. by encouraging, or sometimes requiring, measurement errors; ii. by making it difficult to compare results across measurement studies (since differing results could be introduced by changes in measurement tools, measurement vantage point, etc.); iii. by preventing accurate and reproducible historical measurements; and iv. by consuming an enormous amount of researcher time on developing similar-but-bespoke measurement tools for each task. Research areas benefit greatly from having common, reusable datasets. For example, such datasets encourage incremental improvements, and allow the results to be easily, objectively, and accurately compared across papers. A primary reason why the field of security and privacy Web measurement lacks such datasets is that the field lacks a general-use, accurate format for archiving Web measurements, which would allow a common dataset to be reused across measurement tasks. By comparison, such datasets exist in other areas of computer science research, such as machine learning and system performance. In this work, we aim to improve the state of security and privacy measurement in two ways. First, by presenting WebREC, a system for accurately measuring how browsers execute websites, in a manner robust to page circumvention, and covering a broad enough range of browser behaviors (e.g., rendering, network, script execution, etc.) that WebREC can be used without modification for extensive measurements. And second, with .web , a novel archival format for recording how a browser fetched, rendered, and executed a website. WebREC and .web bring the following benefits for precise and reproducible Web measurement. First, they provide comprehensiveness where current tools do not. WebREC is deeply integrated with Chromium and is built with industry-assisted knowledge of browser architecture to ensure both comprehensive and correct results (see Section 2.2.4 ; this differs significantly from current Web measurement tools). Second, WebREC and .web enable accurate archiving. WebREC records both the resources that were captured during page execution, along with a fine-grained execution timeline. .web archives enable researchers to ask a wide range of questions about how websites behaved at the time-of-archiving in a way fundamentally not possible with most current tools. Third, WebREC and .web are general tools, and allow researchers to measure a wide range of browser behaviors without requiring modifications or customization. By aiming to be general and reusable, WebREC benefits from iterative improvements and corrections, in a way uncommon to most measurement tools (which, historically, tend to be single-use and discarded after each project). More broadly, in this work, we make the following contributions to the goal of accurate and reproducible security and privacy Web measurements: 1. a comprehensive overview of the Web measurement and archiving tools commonly used in research, along with the limitations why each is insufficient for the goal of generating accurate, reusable, achievable Web data; 2. the design of WebREC and .web , a reusable tool and format for measuring and archiving how browsers execute websites accurately and robustly to allow reproducible measurements beyond what current tools and archival formats provide; 3. multiple empirical evaluations of WebREC’s and .web ’s accuracy , compared against existing commonly used tools, finding that WebREC more accurately matches baseline counts of events; 4. an empirical measurement of WebREC’s generality , by surveying recent security and privacy Web measurement papers and evaluating which could have been performed with WebREC directly, or .web archives, avoiding the need to create new, custom