Campus AI vs Commercial AI: A Late-Breaking Study on How LLM As-A-Service Customizations Shape Trust and Usage Patterns DOI: XXXXXXX.XXXXXXX ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Human-centered computing Empirical studies in HCI CCS: Human-centered computing User studies CCS: Human-centered computing
Leon Hannig, Annika Bush, Meltem Aksoy, Steffen Becker, Greta Ontrup · 2025
Read Campus AI vs Commercial AI: A Late-Breaking Study on How LLM As-A-Service Customizations Shape Trust and Usage Patterns DOI: XXXXXXX.XXXXXXX ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Human-centered computing Empirical studies in HCI CCS: Human-centered computing User studies CCS: Human-centered computing by Leon Hannig, Annika Bush, Meltem Aksoy, Steffen Becker, Greta Ontrup (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-10490.txt.
Introduction
Leon Hannig and co-authors set up a study of whether a university-branded AI service shapes trust and use among its users, in a 2025 paper. Many schools now offer chat models dressed in their own name and look, not just the raw product. The team argues that such surface changes, often overlooked, can sway how people feel about a tool.
The paper is a short prelude to a larger field study at a German university, where students and staff will compare the campus service with a leading commercial one. Its aims are to spark debate and gather feedback to sharpen the main study. The results should guide groups weighing a branded AI rollout.
Catalog record
| CREATOR | Leon Hannig, Annika Bush, Meltem Aksoy, Steffen Becker, Greta Ontrup |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-10490
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Campus AI vs Commercial AI: A Late-Breaking Study on How LLM As-A-Service Customizations Shape Trust and Usage Patterns DOI: XXXXXXX.XXXXXXX ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Human-centered computing Empirical studies in HCI CCS: Human-centered computing User studies CCS: Human-centered computing Field studies CCS: Security and privacy Privacy protections Leon Hannig, Annika Bush, Meltem Aksoy, Steffen Becker, Greta Ontrup Abstract . As the use of Large Language Models (LLMs) by students, lecturers and researchers becomes more prevalent, universities – like other organizations – are pressed to develop coherent AI strategies. LLMs as-a-Service (LLMaaS) offer accessible pre-trained models, customizable to specific (business) needs. While most studies prioritize data, model, or infrastructure adaptations (e.g., model fine-tuning), we focus on user-salient customizations, like interface changes and corporate branding, which we argue influence users’ trust and usage patterns. This study serves as a functional prequel to a large-scale field study in which we examine how students and employees at a German university perceive and use their institution’s customized LLMaaS compared to ChatGPT. The goals of this prequel are to stimulate discussions on psychological effects of LLMaaS customizations and refine our research approach through feedback. Our forthcoming findings will deepen the understanding of trust dynamics in LLMs, providing practical guidance for organizations considering LLMaaS deployment. 1. Introduction As Large Language Models (LLMs) become increasingly prevalent in professional and educational settings, organizations face mounting pressure to develop coherent strategies for AI integration ( Kapania et al. 2024 ) . Current research shows high adoption rates of commercial off-the-shelf LLMs such as ChatGPT among students and university staff for various professional and academic tasks ( Zhang et al. 2024 ; Zhou et al. 2023 ; von Garrel and Mayer 2023 ) . In order to guide the informed and secure use of AI, many organizations, including universities, corporations, and government agencies, are exploring the deployment of LLMs as-a-Service (LLMaaS) as an achievable and cost-effective way to adopt AI in-house ( La Malfa et al. 2023 ) . This trend is driven by several practical considerations: the opportunity to offer access to all university members while maintaining institutional oversight and alignment with organizational policies or the ability to provide dedicated technical support ( Paavola 2024 ) . One of the main benefits of AI as-a-Service (AIaaS) solutions is the flexibility they offer in customizing various aspects ( Diaferia et al. 2022 ) , such as fine-tuning models or displaying user interfaces in corporate design. Customizations are important because they can, e.g., improve model performance for the specific organizational context or enhance security and privacy safeguards ( Sundberg and Holmström 2023 ; Lewicki et al. 2023 ) . However, even choices that do not impact model performance or security, such as branding a user interface, could lead to significant changes in how the system is perceived and used. From a psychological perspective, users rely on various cues to assess the trustworthiness and consequently align their use of AI ( Schlicker et al. 2023 ; de Visser et al. 2014 ) . Users interpret information about and from the system in terms of its performance, purpose, and processes ( de Visser et al. 2014 ) . We argue that end users may interpret customizations of LLMs as indicative of the system’s capabilities, goals, and underlying processes, even if the design choices (e.g., corporate branding) do not directly impact functionality. While organizations invest in LLMaaS, we lack systematic empirical evidence on how customization choices within the adoption process influence users’ trust, usage patterns, and overall acceptance of these systems compared to commercial off-the-shelf alternatives (hereinafter referred to as “commercial LLMs” ). This is especially relevant for customization choices that are salient to the user (e.g., user interface) but have no real technical impact on the pre-trained data/model/algorithm/infrastructure ( Diaferia et al. 2022 ) . We aim to close this research gap by conducting an empirical study that answers the following research question: RQ: How do customizations of LLMaaS impact end users’ trust, perception, and usage of LLMs in professional and academic contexts compared to commercial AI alternatives? This late-breaking work presents the research motivation, hypotheses, and study design of an ongoing large-scale field study. The purpose of this “prequel” is twofold: first, we compile a comprehensive overview of LLMaaS customization options to guide the hypothesis formulation, aiming to stimulate further discussion and research into the psychological effects of customization choices on end users. Second, we seek feedback from our professional peers on our proposed field study to enhance its effectiveness. Outcomes from our large-scale field study will enrich both theoretical insights into trust in organizational AI systems and practical applications of AI strategies across different sectors. 2. Background 2.1. LLMs (as-a-Service) at Universities LLMs, such as ChatGPT, have become increasingly prevalent in academic settings, with significant adoption among university students. For example, von Garrel and Mayer 2023 report that nearly two thirds of German students use LLMs for tasks that include concept clarification, literature review, and translation. Similarly, Amani et al. 2023 demonstrate the widespread use of LLMs for personalized learning purposes at Texas A&M University. Researchers have reported using LLMs ( Kapania et al. 2024 ; Syed et al. 2024 ) , e.g., for language editing or literature search ( Abdelhafiz et al. 2024 ) . Thus, universities are caught between enabling students and staff to access and use AI in an informed way to leverage the benefits while also protecting data, privacy, rights, and ethical principles. One way to appropriately shape the use of AI is for universities to introduce “their own” LLMs, the same way as organizations in the private sector do (e.g., WärtsiläGPT by the Finnish company Wärtsilä Oyi; ( Paavola 2024 ) ). However, the introduction of AI poses a massive challenge for universities, due to a lack of internal technical abilities and knowledge about the deployment of AI and the high costs of developing and maintaining a sufficient IT infrastructure ( Lins et al. 2021 ) . LLMs as-a-Service (LLMaaS) provide a practical solution by offering easy access to pre-trained models via cloud providers ( Wang et al. 2024 ) . This approach is a subset of AI as-a-Service (AIaaS), broadly defined as “cloud-based systems providing on-demand services to organizations and individuals for deploying, developing, training, and managing AI models” ( Lins et al. 2021 , p. 424) . Often referred to as “no-code or low-code” AI, these systems are recognized as a means of democratizing AI, enabling organizations to adopt and afford AI technologies without requiring extensive technological expertise ( Sundberg and Holmström 2023 ) . Moreover, LLMaaS solutions emphasize enhanced data protection and security, offering a viable option for institutions like universities to leverage AI’s advantages while adhering to stringent compliance and security standards ( Paavola 2024 ) . The primary benefits of AIaaS include reducing complexity, increasing automation, and enabling customization ( Lins et al. 2021 ) . One proposed advantage of LLMaaS is that organizations can tailor various aspects of LLMs to better align with their unique requirements and objectives. In this context, customizations are defined as configurations of parameters at various architectural layers to improve the fit between the system and organization-specific needs ( Diaferia et al. 2022 ) . Table 1 outlines customization