Visca : Inferring Component Abstractions for Automated End-to-End Testing

Parsa Alian, Martin Tang, Ali Mesbah · 2025

Read Visca : Inferring Component Abstractions for Automated End-to-End Testing by Parsa Alian, Martin Tang, Ali Mesbah (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-04161.txt.

Introduction

Parsa Alian and co-authors turn webpages into component maps so AI can write end-to-end tests in a 2025 paper. Test generators that use language models need the right context about a page, and raw page dumps serve them poorly. Their tool, VISCA, first cuts pages into candidate segments with a fresh heuristic.

A multimodal model then labels each segment and pulls its context into a fixed set of interface parts. That tidy map feeds test writing far better than past context styles. Tests built this way covered most target features, beating the best prior method by a wide margin. The authors frame components, not pixels, as the right unit.

Catalog record

CREATORParsa Alian, Martin Tang, Ali Mesbah
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-04161

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Visca : Inferring Component Abstractions for Automated End-to-End Testing Parsa Alian, Martin Tang, Ali Mesbah Abstract Providing optimal contextual input presents a significant challenge for automated end-to-end (E2E) test generation using large language models (LLMs), a limitation that current approaches inadequately address. This paper introduces Visual-Semantic Component Abstractor ( Visca ), a novel method that transforms webpages into a hierarchical, semantically rich component abstraction. Visca starts by partitioning webpages into candidate segments utilizing a novel heuristic-based segmentation method. These candidate segments subsequently undergo classification and contextual information extraction via multimodal LLM-driven analysis, facilitating their abstraction into a predefined vocabulary of user interface (UI) components. This component-centric abstraction offers a more effective contextual basis than prior approaches, enabling more accurate feature inference and robust E2E test case generation. Our evaluations demonstrate that the test cases generated by Visca achieve an average feature coverage of 92%, exceeding the performance of the state-of-the-art LLM-based E2E test generation method by 16%. I Introduction Before deploying applications, developers need to thoroughly test them to ensure the quality and integrity of the features. End-to-end (E2E) testing, in particular, focuses on ensuring that all components of an application work together correctly and the features designed for the end-user perform smoothly. While manual E2E test creation by human testers can yield high-quality, semantically rich tests, this approach is expensive, time-consuming, and scales poorly with application complexity, necessitating automated solutions. Traditional automated E2E testing techniques, often relying on application modeling [ 1 , 2 , 3 , 4 ] or Reinforcement Learning (RL) [ 5 ] , aimed to reduce manual effort. However, the generated tests lack the quality and semantic coherence of human-written ones, often manifesting as random sequences of actions due to their reliance on criteria that do not inherently capture user-centric goals. Recent advances in Large Language Models (LLMs) have significantly expanded the potential for automating software testing across a variety of domains, including unit, API, and accessibility testing [ 6 , 7 , 8 , 9 , 10 ] . Building on this momentum, LLMs are also explored for E2E test generation. An emerging direction in this space involves leveraging LLMs to generate test cases that cover application features [ 11 ] . Feature inference in current techniques, such as AutoE2E [ 11 ] , involves extracting HTML snippets of user actions and supplying them to an LLM to identify the underlying features. While intuitive, this strategy is hindered by insufficient contextual information. The HTML representation of an interactive element often lacks the surrounding semantics needed for the model to infer its role within the application. As a result, the absence of meaningful context can lead to inaccurate feature interpretations and limit the effectiveness of the generated E2E tests. To address the challenge of finding optimal context for LLM-based feature inference in E2E testing, we introduce Vi sual- S emantic C omponent A bstractor ( Visca ). Our method transforms a given webpage into an abstract structured component abstraction , which is analogous to the modular component models widely adopted in modern UI development frameworks such as React [ 12 ] , VueJS [ 13 ] , and Svelte [ 14 ] . By encapsulating logically related structures, content, and behaviors, this higher-level component-centric abstraction captures the context of UI actions much more effectively than isolated raw HTML elements. Our insight is that this contextual abstraction, in turn, facilitates more accurate feature inference and the generation of more robust and meaningful E2E test cases. The contributions of this work are as follows: • Defining a standard vocabulary for describing the webpages as an abstraction. • A heuristic-based segmentation technique, resulting in segments that are suitable for transformation to component abstractions. • A novel method for transforming an unstructured webpage Document Object Model (DOM) into a hierarchical, semantically enriched component abstraction, identifying logical UI units and their semantic context. • A feature inference technique that leverages the rich contextual information embedded within the component abstraction to more accurately identify the application features and generate test cases for them. Our evaluations demonstrate the multi-faceted effectiveness of Visca . The core component abstraction module achieves a 91.5% accuracy in preserving the structure and content of original applications. Within this, our novel heuristic-based segmentation algorithm attains a precision of 83.4%, outperforming the state-of-the-art VIPS algorithm by 26.9% in this regard. Ultimately, for end-to-end test generation, Visca achieves an average feature coverage of 92%, surpassing the leading feature-driven E2E testing baseline by a 16% improvement. These results collectively highlight the significant benefits of using detailed component abstractions to enhance contextual understanding for automated LLM-driven E2E test generation. II Motivation A critical factor in the success of LLM-based systems is the context provided to the model; its nature and scope significantly influence whether an LLM performs a task effectively. For instance, AutoE2E utilizes the HTML of individual actions as its input context for feature inference. However, relying on such highly localized context presents considerable challenges. Consider Amazon’s shopping cart page, depicted in Figure 1 . On this page, users can “delete a product from the cart.” The simplified HTML for the “Delete” action is shown in 1 . Fig. 1 : Amazon’s shopping cart page ⬇ 1 <span class= "size-small action-delete" > 2 <input value= "Delete" class= "color-link" /> 3 </span> Listing 1: Delete action’s HTML from Amazon’s cart page Based on this action’s HTML alone, the precise target of the “Delete” is ambiguous. This snippet lacks the surrounding information necessary to determine its purpose. Furthermore, since multiple items are in the cart, their respective “Delete” buttons share identical or similar HTML. This makes it challenging for an LLM to distinguish which action corresponds to which item or to infer distinct features. In such cases, the provided context is too narrow , potentially leading to inaccurate or incomplete feature inference. To mitigate narrow context, another solution can be to provide the LLM with the entire page’s HTML. While this theoretically grants access to all information needed to understand element relationships and infer features, it introduces new problems. Webpage HTML can span tens of thousands of lines of code; for example, the Amazon cart page ( Figure 1 ) alone comprises roughly 36,000 lines of HTML, translating to approximately 348,000 LLM tokens. Feeding such extensive data to LLMs risks overlooking crucial information, increases the likelihood of model hallucination, and faces context window limitations. In this scenario, the context becomes too broad , potentially obscuring relevant signals within excessive noise. The challenge of selecting an optimal context is compounded by HTML’s inherent design, primarily for presentation, not for semantic understanding. While the DOM provides a hierarchical structure, it offers no explicit cues for the ideal contextual boundary of any given interactive element. This ambiguity in defining the contextual scope makes it challenging to isolate a DOM segment that is both semantically complete for the task and efficiently processable. To solve the problem of finding optimal context, one might consider segmentation to partition a webpage into distinct, visually and structurally coherent regions (e.g., headers, item lists, forms). Such segments could potentially offer more appropriately scoped, coherent contextual units for an LLM to analyze UI actions and infer features. However, despite their potential, existing segmentation

Read the complete text