Assessing LLMs for Front-end Software Architecture Knowledge

Luiz Pedro Franciscatto Guerra, Neil Ernst · 2025

Read Assessing LLMs for Front-end Software Architecture Knowledge by Luiz Pedro Franciscatto Guerra, Neil Ernst (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-19518.txt.

Introduction

Phone apps are often built in set layers, and one iOS pattern splits those layers in strict ways. Luiz Pedro Franciscatto Guerra and Neil Ernst test in 2025 how well a large language model knows that pattern, named VIPER. They grade the model by a school ladder that runs from recall up to making new work.

The model shone at the top of the ladder: it judged designs well and made sound new ones. It slipped at the low rungs, where exact recall of fine points was the task. That split is odd but useful. It hints that chat tools could cut design costs, while fine facts still need a human check. The team offers its test frame for other design patterns.

Catalog record

CREATORLuiz Pedro Franciscatto Guerra, Neil Ernst
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE21 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-19518

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Assessing LLMs for Front-end Software Architecture Knowledge Luiz Pedro Franciscatto Guerra, Neil Ernst Abstract Large Language Models (LLMs) have demonstrated significant promise in automating software development tasks, yet their capabilities with respect to software design tasks remains largely unclear. This study investigates the capabilities of an LLM in understanding, reproducing, and generating structures within the complex VIPER architecture, a design pattern for iOS applications. We leverage Bloom’s taxonomy to develop a comprehensive evaluation framework to assess the LLM’s performance across different cognitive domains such as remembering, understanding, applying, analyzing, evaluating, and creating. Experimental results, using ChatGPT 4 Turbo 2024-04-09 , reveal that the LLM excelled in higher-order tasks like evaluating and creating, but faced challenges with lower-order tasks requiring precise retrieval of architectural details. These findings highlight both the potential of LLMs to reduce development costs and the barriers to their effective application in real-world software design scenarios. This study proposes a benchmark format for assessing LLM capabilities in software architecture, aiming to contribute toward more robust and accessible AI-driven development tools. I Introduction Specialized architectures like MVC (Model-View-Controller) offer rapid development advantages but can face scaling issues [ 1 ] . Advanced architectures such as VIPER (View-Interactor-Presenter-Entity-Router) provide a consistent structure for building software but demand significant learning and implementation time [ 1 , 2 ] , posing challenges for new developers in communities favoring simpler approaches. Large Language Models (LLMs) could provide assistance with these problems. They are becoming excellent tools for developers (e.g. Microsoft’s Copilot and Sourcegraph’s Cody), excelling at generating repetitive code and documentation, but struggle with complex tasks [ 3 ] . Despite their growing utility, limited research explores how LLMs support front-end architectures like VIPER, as most studies focus on general project-level architecture or unrelated domains [ 4 , 5 ] . This raises the question as to whether LLMs can comprehend project trees, identify underlying patterns, and assist in implementing new features while maintaining architectural consistency. We examine the following research questions: • RQ1 : To what extent do LLMs demonstrate understanding of fundamental front-end architectural patterns, using VIPER as a specific example? • RQ2 : Can LLMs accurately outline the structure of a VIPER module for that feature, specifying the roles and interactions of each component (View, Interactor, Presenter, Entity, Router), given a partial representation of the project of an iOS app (file tree)? • RQ3 : How does the quality of the solutions to architectural design challenges proposed by LLMs (in terms of adherence to VIPER principles, logical consistency, and scalability) compare to those provided by human iOS developers with varying expertise? • RQ4 : How accurately can LLMs infer missing components, predict potential dependencies, or identify architectural inconsistencies? To answer these questions: (a) We created an example iOS project. The project is written in the Swift programming language using the UIKit UI framework and implementing the VIPER architecture. (b) We created a set of questions using a knowledge taxonomy to ensure a broad coverage of questions. (c) We presented these questions to ChatGPT using its API. (d) We analyze and grade the AI’s responses. Since a knowledge taxonomy was used, comparative analysis can then be used between the different cognitive levels. This paper benefits from the primary author’s extensive experience with VIPER, gained from industry practice. We selected the VIPER architecture [ 2 , 6 ] , for being complex and incorporating a number of established patterns, being possible to generalize different responsibilities to other patterns (e.g. observer pattern instead of Presenter tactic). II Related Literature Recent advances in LLMs have shown promise in automating various software engineering tasks, but their application to front-end software architecture remains largely unexplored. Ozkaya et al. [ 7 ] highlight the limited attention given to how design and architecture tasks can be effectively accomplished using generative AI-based software development tools. They emphasize the necessity of human expertise to assess not only the correctness of tool recommendations but also their fitness for purpose in code generation. In the context of software engineering team automation, Lin [ 8 ] explores a scenario where each role in the software creation process is assumed by an LLM agent. However, the evaluation primarily focuses on the final generated code rather than the actual performance of the architect in the project. Although the literature touches on various aspects of architecture and LLMs, direct applications to software architecture are still sparse. Jahic and Hossain [ 9 , 10 ] discuss system architecture, providing insights into how LLMs might be integrated into broader architectural frameworks. Li [ 11 ] explores the application of LLMs in chip architecture, suggesting potential parallels for software architecture. Wei [ 12 ] investigates the use of LLMs to generate software requirements, which indirectly influences architecture, particularly when following clean architecture principles. Several studies focus on the use of LLMs in generating UML diagrams [ 4 , 13 , 14 , 15 ] , Rasheed [ 16 ] mentions the architecture of generated code, though it is often referenced indirectly in prompts rather than being a primary focus. Mzid [ 17 ] examines the detection of design patterns using LLMs. II-A VIPER Architecture We selected the VIPER architecture [ 2 , 6 ] , for being complex and with comparatively little online documentation that an LLM could have been trained on. VIPER is a backronym to its five responsibilities. It is composed of five layers: the View displays data and captures user input, Interactor handles business logic, Presenter acts as a mediator, handling presentation logic, Entity represents the data modelling, and Router manages the navigation between screens. It incorporates a number of established patterns (e.g. delegate), which could be generalized and adapted to other patterns (e.g., the observer pattern being utilized in the Presenter tactic). Additionally, it is a complex architecture with a large learning curve, contains multiple responsibilities and is aimed at large projects, as it has multiple advantages over other more common architectures, having good performance, testability, and modifiability metrics [ 1 ] . III Assessing Architectural Knowledge Multiple knowledge models have been developed to understand how developers comprehend software. These models share common elements, including “telltale code snippets and conjectures about the programming goals, and activities that bring the developer closer to the complete mental model” [ 18 ] . The programmer’s knowledge base typically consists of ”domain expertise; coding knowledge, beacons, and plans; syntactic and semantic knowledge; domain goals; rules of programming discourse; and schemas” [ 18 , 19 ] . However, these software comprehension models are usually based on how developers understand complex code or build knowledge around source code for a project, and given the novelty of LLMs, they are not built or tested against it. III-A Defining Architectural Knowledge Architectural knowledge encompasses a broader scope than just code comprehension. According to Clements et al. and Ozkaya [ 20 , 7 ] , architectural knowledge can be considered the union of five factors: Architecture design (overall structure of the software system); Design decisions (choices made during the architecture design process); Assumptions (attributes assumed to be true during the architecture design); Context (environment where the system will be used, e.g. hardware, operating system, network infrastructure); and other factors (such as cost of development

Read the complete text